Transforming visual accessibility
OpenAI
Be My Eyes is baking GPT-4 into its app to describe the world to blind and low-vision users. Point your camera at anything and get an instant, detailed answer back — no human volunteer required.
Based on reporting by OpenAI — read the original for the full story.
Summary, retelling and take written by AI under human oversight; images are AI-generated illustrations. How we work · Report an error
For years, Be My Eyes has connected blind and low-vision people with sighted volunteers over video call, for everything from reading expiration dates to matching socks. It's a clever idea that depends on a very human resource: someone else's time, attention, and patience at 2am on a Tuesday. Now the company is testing a replacement, or at least a supplement, for that human layer, using GPT-4's ability to understand images.
The new feature, called Virtual Volunteer, lets users snap a photo and ask questions about it directly inside the app. Point it at an open refrigerator and ask what you could cook with what's inside. Aim it at a treadmill display and ask how to change the incline. Point it at a park and ask what's around, and the tool will describe the scene with enough context to actually be useful, not just list objects it detects. That's the real shift here: GPT-4 isn't just labeling a photo, it's reasoning about it in a way that maps to a real request.
Be My Eyes has been testing this with users for weeks before rolling it out more broadly, and the company is framing it less as a novelty and more as a step toward genuine independence. Volunteers are great, but they're finite, and there's something to be said for not having to wait for someone to pick up before you find out if the leftovers in the fridge have gone bad. An AI tool that's available instantly, at any hour, changes the calculus for a lot of small daily tasks that otherwise pile up.
It's also a fairly unglamorous, practical showcase for GPT-4's multimodal skills. Most of the demos we've seen around image understanding have been about generating captions or answering trivia. This is quieter and more useful: a tool built for people who have very specific, recurring needs, not a party trick. And Be My Eyes says human volunteers aren't going anywhere. The AI handles the routine stuff so people can be saved for the situations that need a real human touch, or just a friendly voice.
My take — AI-written commentary, not fact-checked reporting
This is the kind of AI application I actually want to see more of: narrow, unglamorous, and solving a specific problem for a specific group of people who've been asking for exactly this for years. It won't get the headlines that a flashy chatbot demo gets, but it'll matter more to the people using it every single day, and that's a trade I'll take every time.
Read more about this at: OpenAI