With VizWiz, the blind can take a picture, ask a question, and get an answer back from a real person in seconds.
What’s the News: With the web as their eyes, the blind will able to read menus, identify canned foods, and tell whether that park has any free benches without having to walk over. That’s the vision of a team of computer scientists behind an iPhone app called VizWiz, which lets people take a photo of whatever’s perplexing them, record a question like “What denomination is this bill?” and send it off to real people online who’ll respond in a matter of seconds with “That’s a 20.”
How the Heck:
- Blind people have workarounds for the kinds of tasks the sighted use their eyes for—folding dollar bills in certain shapes, keeping the cans of tomatoes separate from the cans of beans, and so on—but these measures often require the input of a sighted person at some point, and they’re not very efficient. An app like this would give the blind more independence.
- Many simple tasks, like reading an address off a letter, are phenomenally difficult for computer intelligences. So the scientists are working with Amazon’s Mechanical Turk, a system that employs people around the world to do odd jobs computers can’t, like choosing the best picture for a product website or reading signs in photographs.
- Then, to solve the problem of speed, the team wrote a program called quikTurkit that works to recruit people even before the question is sent, so there’s always someone on hand to answer. In the latest version of VizWiz, the average turnaround time on a question was 27 seconds. Not bad.
The Future Holds: VizWiz, which is being tested by teams of blind volunteers, hasn’t left the lab yet. But the volunteers are fans: it would be “very useful,” one said (via New Scientist), “because I get so frustrated when I need sighted help and no one is there.” Though an in-depth study [pdf] on VizWiz was released last year, there’s no word yet from the scientists on when this will hit the market. Soon, one hopes.
The App That Lets Users Lend Their Eyes, And Blind People See Things In A New Way
It’s always nice when you can lend a hand to someone in need, but this new app takes that idea to another level: Thanks to Be My Eyes, you’ll now be able to actually lend your eyes to a visually impaired person.
The new iOS app provides a video stream, similar to Apple’s FaceTime video calling, that connects someone visually impaired with someone able to see and willing to help out. The app’s inventor, Hans Jørgen Wiberg, was inspired by FaceTime and how some of his blind friends were using it for visual help.
“With FaceTime, you can’t call a random volunteer,” says Wiberg. “That’s where I came up with the idea of making this group of volunteers who can easily answer a question whenever they have time and if not, someone else will step in.”
The examples the company shows in its product video include visually impaired users getting help with things like reading an expiration date, figuring out what a photograph looks like, and reading signs in an unfamiliar location. The app is more about helping with particularly difficult moments, rather than long periods of assistance.
Opening the app, you’re prompted to select if you are sighted or if you need help (assuming you have accessibility features turned on), and then sign up for an account. If you sign up as a helper, that’s it, you wait until you get a notification to be someone’s eyes. You can share that you’re using the app across social media to gain points in the app, which you also gain for successfully helping someone. The points are used to create a rating system for determining the best helpers. If you need help seeing you’ll be taken to a screen that says “connect to the first helper available.” The screen automatically enables voiceover (reading text on the screen) from this point on as well.
You wouldn’t be blamed for thinking that it’s a little incompatible that someone who can’t see—or has trouble with their vision—would be using a touch-screen mobile phone to begin with. That’s not the case at all; Apple has been particularly aggressive with its accessibility features. “From what we find, a clear majority of blind smartphone users are on iPhone—in Europe almost exclusively,” says cofounder Thelle Kristensen. “IOS has had accessibility functions for the blind since iOS 3 that came with iPhone 3GS in 2009 and has since gotten continuously better.”
Be My Eyes is currently only available for iOS, but Kristensen hopes to add Android in the future through grant funding. The nonprofit also has a GitHub repository for developers interested in speeding the process along for another mobile operating system like Android or Windows Phone.
Of course, beyond the platform issues one of the main challenges a service like this faces is having someone available to help when they’re needed. “We do not have a clear picture of how many helpers are needed, but our estimate is that if we have two helpers per blind person we should be good,” explains Wiberg. “And right now it is more like 10 to 1.”
Within 10 hours of the app’s launch, the service already had about 4,000 helpers and about 500 visually impaired people signed up. In the small-scale testing done prior to launch there weren’t any problems reaching helpers when they were needed. Wiberg is visually impaired himself and came up with the idea for the app when he was a consultant in the Danish blind community. He first gave a TEDx talk about Be My Eyes back in September of 2013.
Be My Eyes is just one of the ways mobile devices are helping assist those in need. Apple, for one, touts its accessibility features, not only for vision, but also for hearing, physical and motor skills, and learning and literacy. And there are other third-party developers building things like a Braille keyboard or light sensor. The Light Detector app, for instance, uses the phone’s camera to turn natural or artificial light into sound so someone knows if an area is dark or light.
For now, Wiberg does not have plans to evolve Be My Eyes to products like Google Glass, but he hasn’t written off the idea entirely: “We would love to try it out,” he says.