XBox Kinect in the OR: Kinect supports gesture interaction with 3D imaging of the patient, while operating.

Here's an interesting use of technology for health - the Xbox Kinect in the OR!

Thanks to Harry van der Veen for the link!


RELATED
Kinect sensor poised to leap into everyday life
Niall Firth, NewScientist, 1/17/13

For the tech-curious:
PrimeSense (Company that developed the 3D depth sensor that powers the Kinect, the sensor in Ava, a healthcare robot by iRobot, and more.)

OpenNI (Framework for the development of 3D sensing middleware libraries and applications.)

NiTE: Natural Interface Technology for End User (Perception algorithms layer for 3D computer vision, allows for hand locating, tracking, analyzing scenes, and tracking skeleton joints.)

ICT MxR Lab's Response to Google's "April Fools Day" Gmail Motion, using OpenNI and FAAST

I recently purchased a Kinect and have been chomping at the bit to do something creative with it.  I have been  too busy with work to dive into a project. Fortunately,  great minds have been paving the way!


I was almost fooled by Google's April Fools prank, "Gmail Motion", a mythical application designed to allow Gmailers the opportunity to conduct email tasks through gestures alone. 


Robert Kosara, an assistant professor of computer science at UNC-Charlotte, and author of the EagerEyes visualization blog,  recently shared a link to a video of  a working prototype of the Gmail Motion concept.  The video features Evan Suma, a post-doc at USC who earned his Ph.D. at UNC-Charlotte.  



"This morning, Google introduced Gmail Motion, allowing users to control Gmail using gestures and body movement. However, for whatever reason, their application doesn't appear to work. So, we demonstrate our solution - the Software Library Optimizing Obligatory Waving (SLOOW) - and show how it can be used with a Microsoft Kinect sensor to control Gmail using the gestures described by Google. This project uses OpenNI coupled with FAAST and was made by Evan Suma and the folks at Mark Bolas' MxR Lab at the USC Institute for Creative Technologies."

Below is the video that almost had me fooled on April 1st  : )
>

RELATED
FAAST Video Gallery
Download FAAST from the USC ICT FAAST website.
University of Southern California Institute for Creative Technologies (USC ICT) 

The FAAST website has loads of  additional "how-to" information. The list below is just a sample of what can be found on the site:

To use FAAST, you will need to download and install the following software:

1. OpenNI Unstable Build for Windows v1.0.0.25
2. PrimeSense NITE Unstable Build for Windows v1.3.0.18
   During NITE installation, enter the free license key from OpenNI: 0KOIk2JeIBYClPWVnMoRKn5cdY4=
3. Hardware drivers for your sensor (only one of the following)
   • Microsoft Kinect: SensorKinect Module – download Win32 installer from Bin folder
   • PrimeSensor: PrimeSensor Module – this does NOT work with the Kinect

FAAST should then run out-of-the-box; no additional installation or setup is necessary. If you encounter an error on startup, you may also need to install the Microsoft Visual C++ 2008 SP1 Redistributable Package.

FAAST project members are Evan A. Suma, Belinda Lange, Skip Rizzo, David Krum, and Mark Bolas

Interested in the OpenNI Initiative? OpenKinect? To learn more, read Josh Blake's Interview of Tamir Berliner of PrimeSense

Interview: Tamir Berliner of PrimeSense

Josh Blake, Deconstructing the NUI, 12/9/10

Josh Blake recently interviewed Tamir Berliner, one of the founders of PrimeSense.  If you haven't heard, Microsoft's Kinect was based on work by PrimeSense, and licensed their technology. PrimeSense provides consumer electronics with natural user interaction capabilities. The good news is that the company recently released open-sourced middleware for natural interaction and depth-camera drivers. It will be interesting to see how this will play in the near future!

In the interview, Tamir discussed a number of topics related to postWIMP technologies.  He also announced the newly created  OpenNI, "an industry-led, not-for-profit organization formed to certify compatibility and interoperability of Natural Interaction (NI) devices, applications, and middleware."   It is good to see this level of support for the cause!

Here is a quote from the interview that I especially liked:

"I believe that till today the devices we’ve been using, made us learn greatly lot about them before we could use them and gain their value. I’m pretty sure everyone who is reading this has got at least 3 remotes sitting on his living room table, and at least once a week needs to help someone use their computer/media center/phone/etc. It’s time for that to change and it’s up to us, the technologists to make this revolution happen, it’s time for the devices to take the step of understanding what we want and making sure we get that, even without asking if it’s a trivial task as opening a door when we approach, closing the lights when we leave the room, even making sure we have hot water to shower with when we return from work or wake up in the morning, depends on what we normally do." -Tamir


RELATED
Here are a couple of videos from the OpenNI website that demonstrate OpenNI-compliant applications:

OpenNI-compliant real time skelton tracking by PrimeSense


OpenNI-compliant real time SceneAnalyzer by PrimeSense

FYI: 

Josh Blake is the author of the Deconstructing the NUI blog. Over the past couple of years, he's explored natural user interfaces and interactions through his work on applications designed for Microsoft Surface and Win7 with Windows Presentation Foundation.

About a month ago, Josh organized OpenKinect, an on-line community to support collaboration among people interested in exploring ways to use Kinect with PCs and other devices.  An example of this effort is the open source code, libfreenect, which includes drivers and libraries for Windows, Linux, and OS X. 

The Natural User Interface Revolution
Josh Blake, 1/5/09


Kinect for Xbox 360: The inside story of Microsft's secret 'Project Natal'  (long, but worth reading) David Rowan, Wired UK, 10/29/10


People of libreenect

OpenNI User Guide (pdf)