New app connects visually impaired with sighted assistants

Share

From his couch at home in New York City, Sam Marcus helped a woman bake corn muffins in Jefferson City, Mo., last week.

The woman, who is blind, needed someone to read the instructions on a box of muffin mix. She connected with Marcus through Be My Eyes, a new mobile app that matches visually impaired people with sighted volunteers who assist with everyday challenges via a live video connection.

Whether it's reading an expiry date on a carton of milk, picking a laundry setting or finding out if your shirt works with your pants, the revolutionary app crowdsources help for the blind so they can do the simple things that sighted people take for granted. Visually impaired users point their iPhone cameras at what they need seen, and volunteers from all over the world lend their eyes for a few minutes.

The free app was launched in mid-January from Copenhagen, and in a matter of days it attracted nearly 75,000 sighted helpers and 5,500 blind users, making more than 12,000 assists.

"She put a box into view of the camera. She asked if it was turned the right way and it wasn't, so she flipped it," Marcus, a 24-year-old software developer, said about his stint as baker's aide. "Then I read the instructions on the back and broke it down for her. She was happy, and I was happy that I helped someone."

Hans Jørgen Wiberg, a Copenhagen-based craftsman who suffers from tunnel vision, invented the app after his blind friends told him they used Apple's video-calling service FaceTime to connect with friends for a "pair of eyes." The downside was that friends are sometimes unavailable. Sighted helpers on Be My Eyes currently outnumber the blind by more than tenfold, meaning there's little trouble in getting someone on the horn.

Aside from help with daily tasks, Wiberg hopes to provide another added benefit – more pure social time. "When a sighted person visits a blind person, they will very often be met with, 'Could you please help me with this?'" he says. "Now they can get the small tasks out of the way and then have the neighbour over for coffee – and just coffee, not helping."

The "do-good" app is just one of many new digital services for the blind: Advances in technology and the near-universal use of social media have spurred substantial development in applications that can help the sight-impaired. TapTapSee scans barcodes, reads labels and compares photographs against an extensive image database, promptly returning the best match. VizWiz users take a photograph and record one question about it. The shot (be it of a can of tomatoes or a denomination of currency) is swiftly sent off to anonymous web contractors or object recognition software. The service can also friendsource the question to the user's own Twitter followers, Facebook friends or a specified e-mail contact. Another app, CamFind camfindapp.com) is a visual search engine that helps the blind shop online. And Talking Goggles (tagline: "a camera with speech") uses optical character recognition to read small text on DVDs, magazine covers, books and other printed items.

The social aspect of vision-assisting apps evokes a scene from the 2001 film Amelie that sees the philanthropic heroine guiding a blind man across a Paris street while narrating for him what she sees: prices at the butcher and cheese shops, a grocer sampling melon, lollipops in a bakery window, a local eccentric in her husband's military garb, and what the florist's eyes look like – crinkly.

When Be My Eyes users need help, they hit a button that reads "connect to next available helper." This prompts the app to send a text message to a random helper; a live video connection is established when the volunteer answers the call. If a request isn't answered within 10 seconds, the app moves on to another assistant, repeating that process until the call is picked up. The app is free on iPhones.

For a steak dinner earlier this week, Jennie Emery and her husband Matthew, both blind, used Be My Eyes to connect with a stranger who helped them pick a Bordeaux from their wine rack. "This nice guy answered and said, 'How can I help you tonight?'" said Emery, an outpatient counsellor in Kankakee, Ill. "He sounded very laid back," she recalled, and he told her to inch her hand down a few bottles to find a selection of reds.

Emery sees herself using the app again to quickly scan envelopes and toss junk mail at home, or at a hotel to discern between the complimentary bottles of shampoo and conditioner – tasks that she calls "little conveniences that you kind of take for granted."

Wiberg said that there is currently no screening process for volunteers, but users are encouraged to rate and review their experiences, and report any abuses. "There are a lot of good people who want to help," he said. "And you can lie on your couch and do something good."