I've been following the evolution of Microsoft's Kinect, and recently discovered a few interesting videos that show how far the system has come. According to Josh Blake, the founder of the OpenKinect community and author of the Deconstructing the NUI blog, the Kinect for Windows SDK v1.7 will be released on Monday, March 18th, from http://www.kinectforwindows.com. More details about this version can be found on Josh's blog as well as the official Kinect for Windows blog. It is possible to create applications for desktop systems that work with the Kinect in interesting ways, as you'll see in the following videos. I think there is potential here for use in education/edutainment!
Below is a video of Toby Sharp, of Microsoft Research, Cambridge, demonstrating Kinect Fusion. The software allows you to use a regular Kinect camera to reconstruct the world in 3D.
KinEtre: A Novel Way to Bring Computer Animation to Life According to information from the YouTube description, "KinÊtre is a research project from Microsoft Research Cambridge that allows novice users to scan physical objects and bring them to life in seconds by using their own bodies to animate them. This system has a multitude of potential uses for interactive storytelling, physical gaming, or more immersive communications."
The following videos are quite long, so feel free to re-visit this post when you have time to relax and take it all in!
Kinect Design Considerations This video covers Microsoft's Human Interface Guidelines, scenarios for interaction and use, and best practices for user interactions. It also includes a preview of the next major version of the Kinect SDK.
Kinect for Windows Programming Deep Dive This video discusses how to build Windows Desktop apps and experiences with the Kinect, and also previews some future work.
Note: Although I recently received my developer kit for Leap Motion, another gesture-based interface, I haven't lost interest in following news for Kinect.
How does MaKey MaKey work? It is powered by a board that can support 6 keyboard keys, and mouse control. It runs on top of Arduino, an open-source electronics prototyping platform that supports multi-modal interactive input and output. I see endless possibilities and fun maker-crafting with my little grandson!
In the following video, musician/visual artist j.viewz uses his MaKey MaKey kit to hook up fruits and veggies to his music system. Watch j.viewz play a bunch of grapes! The strawberries sound nice.
Now that I'm set to experiment with Leap Motion, I started thinking about Google Glass - I know if I visit my grandson wearing them, he'd figure he should, too. Why not? He expects to play with my iPad for at least a short while during our visits. I can see the potential for active educational game applications with this device. I wonder if Google Glass will follow the path of cell phones into classroom settings. Once banned, many schools are embracing their use in BYOD (Bring Your Own Device) programs. BYOGG? TIme will tell. Here is a video from Google provides a view of what the Google Glass experience might be like for a variety of people:
Here is an example of a mother of a 2-month-old infant, wearing Google Glass as she shares special moments:
I'm guilty of BTV (baby TV)- in the form of BetaMax and VHS tape recordings. If Google Glass was around when my kids were babies, I'd probably do the same. Here is another example of "Project Glass": I work with a number of students who are non-verbal and have severe autism. I think there is potential for use with children and adults disabilities. Some ideas that come to mind: Facial expression translator/decoder (for people with autism spectrum disorders) Two-way sign language translator Augmented device for the visually-impaired, elderly, etc. Accessible games, active games Travel guide, museum guide, health care/hospital stay guide Exercise companion InfoVis advisor Shopping trip/fashion advisor for people like me who hate shopping
My Leap Motion developer kit arrived last week. I carefully unboxed the small device and tried out the demo apps that came with the SDK. I'm doing more looking than leaping at this point. I'd like to create a simple cause-and-effect music, art and movement application for my 2-year-old grandson, knowing that he'll be turning three near the end of this year. It would be nice if my app could provide young children with enough scaffolding to support gameplay and learning over a few years of development. Now that I'm a grandmother, I've spent some time thinking about what the evolution of NUI will mean for young children like my grandson. Family and friends captured his first moments after birth with iPhones, and shared across the Internet. Born into the iWorld, he knows how to use an iPad or smart phone to view his earlier digital self on YouTube, without ever touching a mouse or a physical keyboard.
The little guy is pretty creative in his method of interacting with technology, as I've informally documented on video. He was seven months old when he first encountered my first iPad. It was fingers-and-toes interaction from the start. In the first picture below, he's playing with NodeBeat. In the second picture, he's 27 months old, experimenting with hand and foot interaction, on a variety of apps.
My grandson is new to motion control applications, so I'm just beginning to learn what he likes, and what he is capable of doing. A couple of weeks ago, we played River Rush, from the Kinect Adventuresgame. He loved jumping up and down as he tried to hit the adventure pins. Most of the time, he kept jumping right out of the raft! (I think next time we'll tryKinect Sesame Street TV, or revisit Kinectimals.)
One of the steps I'm taking to prepare for my Leap Motion adventure is take a look at what people have done with it so far. There are at least 12,000 developer kits released, so hopefully there will be some interesting apps to go along with the retail version of Leap Motion when it is released at Best Buy on May 19th of this year. One app I really like is Adam Somer's AirHarp, featured in the video clip below:
I also like the idea behing the following app, developed by undergraduate students: Social Sign: Multi-User sign language gesture translator using the Leap Motion Controller (git.to/socialSign) "Built at the PennApps Spring 2013 hackathon, Social Sign is a friendly tool for learning sign language! By using the Leap Motion device, the BadApples team implemented a rudimentary machine learning algorithm to track and identify American Sign Language from a user's hand gestures."
"Social Sign visualizes these hand gestures and broadcasts them in textual and visual representations to other signers in a signing room. In a standard chat room fashion, the interface permits written communication but with the benefit of enhanced learning in mind. It's all about learning a new way to communicate."-BadApples Team
There are a few NUI-focused tech companies that have experimented with Leap Motion. Today, I received a link to the following videoclip Joanna Taccone, of Intuilab, featuring their most recent work: Gesture recognition with Leap Motion usingIntuiFace Presentation
"Preview of our work with the Leap Motion controller. In the same spirit as our support for Microsoft Kinect, we have encoded true gesture support, not just mouse emulation, for the creation of interactive applications by non-programmers. The goal is to hide complexity from designers using our product, IntuiFace Presentation (IP). Through the use of IP's trigger/action syntax, designers simply select a gesture as a trigger - Swipe Left, Swipe Right, Point, etc. - and associate that gesture with an action like "turn the page" or "rotate the carousel". As you can see in this video, it works quite well. :-) We will offer Leap support as soon as it ships." -IntuiLab
Below is a demonstration of guys playing Drop Cord, a collaboration between Leap Motion and Double Fine. From the video, you can tell that they had a blast! Here is an excerpt from the chatter: "The thing is that everyone just looks cool..Yeah, I know, it doesn't matter what you are doing...it's got the right amount of speed-up-slow-down stutter-y stuff...it is like a blend of art and science.."
According to the website, Drop Chord is a"A music-driven score challenge game for the Leap Motion controller, coming soon for PC, Mac, & IOS from the creators of Kinect Party.." The following video is a demonstration of the use of Leap Motion to control an avatar and other interaction inSecond Life:
Below are a few more videos featuring Leap Motion:
Control Your Computer With a Chopstick: Leap Motion Hands On (Mashable) The Leap Motion Experience at SXSW 2013 LEAP Motion demo: Visualizer, Windows 8, Fruit Ninja, and More...
SOMEWHAT RELATED Kinect for Windows Academic: Kaplan Early Learning "3 years & up. Hands-on play with a purpose -- the next generation way. This unique learning tool uses your body as the game controller making it a great opportunity to combine active play and learning all in one. Use any surface to actively engage kinesthetic, visual, and audio learners. Bundle includes the following software: Word Pop, Directions, Patterns, and Shapes." Comment: I've been an enthusiastic supporter of natural-user interfaces and interaction for years - back in 2007 I worked on touch-screen applications for large displays as a graduate student, and became an early member of the NUI group. I'm also a school psychologist, and from my experience, I understand how NUI-based applications and technologies, such as interactive whiteboards and touch-tablets, such as the iPad can support the learning, communication, and leisure needs of students who have significant special needs. It looks like Leap Motion and similar technologies have the potential to support a wide range of applications that target special populations, of all ages.
