Microsoft Software Lets Researchers Make Serious Use of Its Video-Game Controller

New software released today by Microsoft could help more academics use the company’s gesture-based video-game controller—called Kinect—in research projects to explore new applications in robotics, health care, education, and other fields.

Kinect, which retails for $150 and debuted in November, is an add-on for the Xbox 360 gaming console. It picks up users’ movements and voices via microphones and cameras—hence the device’s tagline, “Your body is the controller.”

Microsoft unveiled a beta version of a software development kit for the Kinect sensor today that will let any developer build tools that use the controller.

Anoop Gupta, a “distinguished scientist” at Microsoft Research, said in a Webcast announcing the software that the kit will give developers and coding enthusiasts access to the device’s raw audio and visual streams, expanding Kinect beyond its entertainment roots.

Developers will be able to take existing Kinect devices and hook them up to a PC and create software applications that “make the world of computing more natural and intuitive for us,” Mr. Gupta said. “It’s about how gestures and speech all come into play.”

Howard Jay Chizeck, a professor of electrical engineering at the University of Washington, is working on providing surgeons with a sense of touch in telerobotic surgery using a modified version of the Kinect system.

Surgeons must often operate endoscopically—working through small tubes and using computer monitors for visual reference. Adding a virtual sense of touch could help the doctors be more precise.

“The Kinect is a very low-cost depth cam compared to what’s available,” Mr. Chizeck said, noting that the new software will make it easier for scientists to use Kinect in their projects. “It’s one more arrow in the quiver.”

He specifically referred to Kinect’s ability to accurately track human movement as particularly exciting for researchers.

Even without the software released today, developers have already managed to use Kinect to create new tools—by hacking it. One example is NAVI, or Navigational Aids for the Visually Impaired, which was created at the University of Konstanz, in Germany. The developers connected Kinect to a helmet, allowing a visually impaired wearer to navigate a hallway using computer voice commands.