BrainPort device lets users see with tongue

Mark Pappas walked slowly through a foam obstacle course at Pitt holding his hands out in front… Mark Pappas walked slowly through a foam obstacle course at Pitt holding his hands out in front of him so he wouldn’t bump into any of the soft shapes. For the first time in 12 years, he could see his own arms in front of him — thanks to his tongue.

A small, flat mouthpiece in the 36-year-old Ohioan’s mouth transmitted images from the world around him to a low-resolution, black-and-white image he could “see” through the nerves on his tongue.

One of 30 people to participate in the Pitt study of the new BrainPort vision device, Pappas said he thinks the device will help blind people, especially when trying to see branches or signs sticking out over a sidewalk, which his cane cannot feel. Pappas also has hope that the device will eventually be able to help him get back into golf, a sport which he used to love before he lost his vision.

He did have trouble with the obstacle course though, which consisted of walking down a 7-by-40-foot hallway with various foam obstacles set up at different heights. Since he is used to walking with his hands out in front of him so as not to bump into things, he said his arms caused interference with the BrainPort device, and he kept thinking they were obstacles in the course.

But despite this problem, Pappas said he was “just amazed at what I was able to do.”

Participants in the study said that they loved the device, which Lighthouse International developed in 2009 and is now being tested on Pitt’s campus, but it likely wouldn’t replace existing aids like canes. It’s also likely to spend several more years in testing before patients might see them in mass production.

Pappas lost his sight 12 years ago after contracting a pituitary adenoma tumor the size of a golf ball. He heard about the BrainPort vision device back in 2009, and when his sister found out that a related study was being conducted at UPMC, he immediately applied. He said he was very lucky to have been chosen.

Dr. Amy Nau, director of optometry and low-vision services at the Eye Center of UPMC, was given a $1 million research grant from the state of Pennsylvania to test the device. She said the purpose of the BrainPort device is to try to determine some device that can be used to allow artificial vision, , and she deemed the device a “perfect fit” for her study.

Nau said that the device works by having its users wear a pair of sunglasses that have a miniature camera attached to the nose. The camera captures the visual scene in front of the user, which is transferred to a converter box where it is processed into a different kind of image.

Electrical pulses, which Nau said feel like Pop Rocks candy on the tongue, are then sent to a small flat mouthpiece which transmits the shape of the object that the video camera captures. This allows the user “see” what is in front of them.

“If you take a picture of a cup, you feel the shape of the cup on your tongue,” Nau said.

A hand-held control boosts the intensity of the pulses from mild to stronger, zooms in or out, and increases the contrast of objects, making them easier to sense. The pictures run on white and black contrast, so if there are black letters on a white table, the black letters vibrate on the tongue.

The study consisted of various tests which were designed to see how effective the BrainPort device was. The tests, which were all conducted once without the device and once with it, consisted of reading letters from computer screens, picking up objects and completing an obstacle course.

Mike Fuscardo, a 54-year-old former police officer, also heard about the study from his sister and was thrilled when he was chosen to participate last October.

Fuscardo, who lost his sight on the job in 1992 after 10-and-a-half years of service, traveled from West Virginia to UPMC to take part in the study.

“[The BrainPort device] was interesting and took some getting used to, but it was neat,” Fuscardo said.

Nau, along with the participants, knows the device still needs some improvement.She said that the device is still a research prototype and will require training and practice for users to be able to know how to use it.

“To be more ‘out of the box and start using it’ it needs more improvements,” Nau said.

Fuscardo said that the prototype he used was more of a beginning model, and needs to be improved from the 400-pixel camera that is currently on it so that the wearer can get a clearer picture.

Henry Payton, a 57-year-old participant from Arizona who came to Pitt in April, agreed that the pixels on the camera need to be improved and that the device needs to become wireless, but overall he loved the device.

Payton, who lost his sight in a race-car accident in 2005, heard about the device from his son’s friend after he read about it in an engineering magazine.

“I think eventually people will forget you can’t see with your eyes because you’ll be seeing with your tongue,” Payton said.

Fuscardo doesn’t believe the device will ever be able to take the place of a cane, but said that it can definitely help in tasks around the house. He said it will also be really helpful to decipher the men’s stick figure from the women’s on bathroom doors.

Nau, the only researcher on the project, is going to run two more trials for the device.

The next study will be more long-term and will hopefully be completed in February 2012, she said. In this study, the participants take the device home and track how they use it and whether or not they have any safety issues. So far, six participants have completed the study, and five more are set to enroll.

The other study is a one-year FDA safety study that will start in either July or August and will consider improvements for the device. For this study, Nau will collaborate with Carnegie Mellon’s Robotics Institute to test things like facial recognition that will make the camera “kind of smart” and enhance ability for users’ independence.

Nau has high hopes for the device and said that it has “bright and significant promise.”

“Based on results we’ve seen, every patient has been able to do things they haven’t been able to do before: detect doors, press elevator buttons, find a chair, tell if there’s a person in front of them,” she said.