Engineers work on robotic arm for stroke victims

Besides changing the way college students kick virtual soccer balls and discharge alien…

Kenny Ong | Visual Editor

Paul Monroe, a senior electrical and computer engineering major, commands a robotic arm with Microsoft Kinect’s motion controls.

Besides changing the way college students kick virtual soccer balls and discharge alien weaponry, advances in video game technology might soon transform the lives of people recovering from stroke.

Paul Monroe, Pitt senior and electrical engineering major, wants to make an assistive robotic device for stroke patients out of Microsoft Kinect, a motion sensor that, since 2011, has allowed people to play Xbox games with merely their hands.

Under the guidance of Bambi Brewer, assistant professor in the School of Rehabilitation Science and Technology, Monroe has kicked off his semester-long project to mix Kinect, computer programming and robotics to the point where they become something therapeutic.

“It’s taking existing knowledge and combining it in a new way,” Monroe said. “It could help their lives and improve their lives.”

Amid a full slate of classes, science outreach activities and Pitt Marching Band, Monroe heads to Pitt’s Bakery Square research facility three mornings per week on a special mission — to wage war on robots.

Much like the chemist probes the workings of a new reaction or the cosmologist bites at celestial unknowns, Monroe — who has spent several months working at NASA — tries to understand the enigmatic behavior of his research subject: a robotic arm. It’s about the size of a German shepherd and can grasp objects three feet away, but it often doesn’t want to cooperate.

Trying to execute Monroe’s command to move here or there, the robot has a talent for confusing itself. Sometimes it overshoots its target; other times, it attempts the impossible and subsequently shuts down. “It tries to reach through its body to get to its back,” Monroe offered as an example.

Monroe’s task for the fall, in part, is to gain control over these tendencies, using the computer programming languages of C-sharp and C++ as reins and bridle. His immediate goal is to improve “the robustness of the software, so it doesn’t cut itself off whenever you tell it to point.”

The larger goal is to build a robotic assistant that could seamlessly add to and enhance the daily routine of millions of disabled stroke victims. Since stroke patients commonly experience loss of movement in one side of the body or the other, Monroe and Brewer imagine the robotic arm serving a largely supportive purpose, particularly in the kitchen.

For example, it could help stabilize the milk bottle before opening or keep the pan still before stirring.

“We’ve been most interested in the kind of tasks that are symbiotic,” Brewer said.

While holding milk might seem a bit “square one,” getting a robot to effectively perform as a second appendage can get very complicated, very quickly. And that’s just as true for the user as it is for the engineer.

“You can have an awesome robot, but if someone can’t use it, then it’s not going to help them,” Brewer said. “We want to make the robot as easy to use as possible.”

So to reduce the chance of patients abandoning their robot on account of complexity, Monroe and Brewer are avoiding the common — though endlessly frustrating — modes of human-robot interaction (i.e. two-dimensional joystick, a keypad or strap-on devices). Instead, they’re trying motion detection.

That’s where video games come in. The 2011 debut of Microsoft Kinect surely excited gamers yearning to lose the physical middleman in their digital fun, but it also raised the eyebrows of engineers around the world — think of playing the piano with hands hovering over the keys or navigating Powerpoint without clicking. Brewer’s group similarly saw a real-life application, and Monroe is working on making that vision a reality.

By December, Monroe hopes to produce a robotic system that effectively translates three-dimensional movement detected by the Kinect into the appropriate response of the robotic arm (yes, there’s a lot of Calculus III involved). If all goes well, a user waving one hand to simulate a disabled stroke victim should be able to point to where the robot should go, and the robot should obey the command. And once the robot’s at, say, the milk bottle, the user should be able to effortlessly fasten the robot arm around the bottle with one hand.

A tall order, perhaps, but Monroe harbors confidence that he’ll accomplish something meaningful by semester’s end. “Regardless of whether our goals are met, I will still walk away having contributed to it,” he said.

Monroe’s project is by no means the first time researchers have enlisted robots to aid stroke victims, and it will not be the last. The field of robot-assisted stroke recovery is growing and exciting, said Tudor Jovin, director of the UPMC Stroke Institute.

“There are a lot of promising assistive devices that will allow stroke victims to increase their level of functionality through rehab techniques, and robotics are one of these approaches,” Jovin said, citing robotic leg braces and brain-computer interfaces. “It’s the next frontier we need to conquer in the fight against stroke.”

Conquering that frontier might frustrate Monroe at times, but for him the rewards overshadow the obstacles, especially when he considers the robot’s clinical destination.

“To be able to see people use what I’ve built, to use what I’ve worked for a semester to do,” he said. “That’s why the civil engineers want to see their bridges and buildings be built. It’s why the bioengineers want to see their drugs or their artificial hearts be put into place. It’s why engineers stick with being engineers.”