ICT MxR Lab's Response to Google's "April Fools Day" Gmail Motion, using OpenNI and FAAST

I recently purchased a Kinect and have been chomping at the bit to do something creative with it.  I have been  too busy with work to dive into a project. Fortunately,  great minds have been paving the way!


I was almost fooled by Google's April Fools prank, "Gmail Motion", a mythical application designed to allow Gmailers the opportunity to conduct email tasks through gestures alone. 


Robert Kosara, an assistant professor of computer science at UNC-Charlotte, and author of the EagerEyes visualization blog,  recently shared a link to a video of  a working prototype of the Gmail Motion concept.  The video features Evan Suma, a post-doc at USC who earned his Ph.D. at UNC-Charlotte.  



"This morning, Google introduced Gmail Motion, allowing users to control Gmail using gestures and body movement. However, for whatever reason, their application doesn't appear to work. So, we demonstrate our solution - the Software Library Optimizing Obligatory Waving (SLOOW) - and show how it can be used with a Microsoft Kinect sensor to control Gmail using the gestures described by Google. This project uses OpenNI coupled with FAAST and was made by Evan Suma and the folks at Mark Bolas' MxR Lab at the USC Institute for Creative Technologies."

Below is the video that almost had me fooled on April 1st  : )
>

RELATED
FAAST Video Gallery
Download FAAST from the USC ICT FAAST website.
University of Southern California Institute for Creative Technologies (USC ICT) 

The FAAST website has loads of  additional "how-to" information. The list below is just a sample of what can be found on the site:

To use FAAST, you will need to download and install the following software:

1. OpenNI Unstable Build for Windows v1.0.0.25
2. PrimeSense NITE Unstable Build for Windows v1.3.0.18
   During NITE installation, enter the free license key from OpenNI: 0KOIk2JeIBYClPWVnMoRKn5cdY4=
3. Hardware drivers for your sensor (only one of the following)
   • Microsoft Kinect: SensorKinect Module – download Win32 installer from Bin folder
   • PrimeSensor: PrimeSensor Module – this does NOT work with the Kinect

FAAST should then run out-of-the-box; no additional installation or setup is necessary. If you encounter an error on startup, you may also need to install the Microsoft Visual C++ 2008 SP1 Redistributable Package.

FAAST project members are Evan A. Suma, Belinda Lange, Skip Rizzo, David Krum, and Mark Bolas

Supertouch Group: Use an iPhone or iPad to Send Balls into Kinect Dodgeball

See how they inter-"kinect"!



"The SuperTouch group mission is to generate wonder through simple, fun, entertaining and educational uses of interactive technology and HCI."


RELATED


iPAD + Kinect + Processing

IPAD + Kinect + Processing = stellar foundation for interactions from SuperTouch Group on Vimeo.

Another close encounter with in-store digital display marketing at Best Buy, bad internet TV controllers, bowling with the Kinect, and more...

Not long ago I visited a Best Buy. While I was there, I wanted to play around with the Kinect, but the demo system hadn't arrived.  As I wandered around the store, I encountered quite a few digital displays, part of  Best Buy's recent in-store digital media marketing effort.  I later shared my experience in a blog post,  Close Encounter with "Best Buy On": Example of a multi-channel marketing approach using in-store digital media that includes an on-line magazine.  


During today's visit to Best Buy,  I noticed that there were more display centers in various departments in the store, and many of the displays had useful and informative content. The Kinect demo was up and running, too.


Within the store,  I noticed a strong emphasis on HDTVs with internet capabilities. I  was hoping that the new Internet HDTVs would come with user-friendly touch-screen controllers, or at least an app for use on touch-screen smartphones, iPads, or other touch-screen tablets. 

What I found was disappointing.  On display were traditional-looking multi-button remote controllers, controllers that looked like PC keyboards,  and of course, Sony's confusing multi-featured contribution to the Internet TV scene.


None of the controllers seemed to be easy-to-use, or capable of supporting web-navigation and other web interactions on HDTVs from a distance, especially when the goal is to watch movies and video from a recliner in a darkened family room.  


What sort of user-centered design or usability studies were in place during the hatching of Sony's Internet TV controller?!  (For more about bad remote controls, one of my pet peeves,  see  "Oh! No! Sony's "Mother of Remote Controls" for Google TV, 74 Buttons and Counting")


Below is a partially annotated slideshow of pictures I took of some of the displays and other things I encountered at Best Buy.  Enjoy!



Close Encounter with Kinect Bowling
It wasn't easy trying to bowl and take video with a phone at the same time! 
(Please excuse the shaky video effects and the view of my fingers.)

PO-MO, a creative group that combines digital art, interaction, movement, and play to create engaging surfaces and spaces.

I recently learned more  PO-MO, a relatively new start-up tech company based in Winnipeg, Canada. According to the company's information, PO-MO "specializes in interactive digital display solutions, including gesture and motion based interactivity, interactive display content creation and management, and large interactive display and projection services for advertisers, educators, and events."  


Po-Motion was a finalist in an elevator pitch video contest last fall. It has several advantages over potential competitors.  The system is easy to use, and priced within the range that is affordable for schools, museums, and other cost-conscious groups who would like to provide technology-supported immersive interactive experiences for people of all ages.  The PO-MOtion software designed for interactive floors and walls starts at $39.99, and works on any computer, using any USB web camera and a projector. Other applications make use of Kinect sensors.


I especially like one of PO-MO's recent projects, the Impossible Animals Museum Exhibit, created using Unity 3-D, for the Manitoba Children's Museum.  How does it work?  Children create a colored egg using crayons and paper, which is then scanned into the exhibit and digitally embedded into the system, which includes an interactive wall and floor.  When the egg is touched, it is activated to hatch, and then becomes a motion reactive animal.  The environment includes things like water, landscapes, and even a spaceship.  The system has a "reset world" button for museum staff to use when needed.  

Impossible Animals Exhibit

Impossible Animals Interactive Museum Installation from PO-MO Inc. on Vimeo.



The following video explains how the PO-MO system works:

PO-MO is also involved in promotional projects, assisting retailers, ad agencies, and brand managers with creative ways to engage customers and clients:

Ragpackers Kinect-based Window Display

Ragpickers Kinect Window Display from PO-MO Inc. on Vimeo.


The following video provides a scrolling description about PO-MO's work, including promising data collected during implementation:

SAP demo final from PO-MO Inc. on Vimeo.

Other products and services provided by PO include mobile app development. I especially like the augmented reality business card depicted in the following video clip:

Augmented Reality Business Card from PO-MO Inc. on Vimeo

Imagine if your local shopping centers, museums, libraries, or even schools offered this level of immersive interaction on a regular basis!

RELATED
The PO-MOtion system has a wide range of uses. It is currently used in an educational setting in a sensory room for students with special needs, something that I'd like to try out in the near future with students at Wolfe School. I plan to share more about this in another post.

PO-MO Case Studies


PO-MO Bios:

Meghan Athavale – Director/CEO, PO-MO Inc.

"Meghan has been a professional designer and animator since graduating from Red River College in 1997. After graduation, she moved to Calgary, where she spent almost two years directing projects at Aurenya Studios, a start-up animation company. In 2001, Meghan was engaged by Community Connections to support community-based IT development projects in rural Manitoba and in Winnipeg’s inner city.  In 2008, Meghan joined Manlab, developing educational interactive games and resources for Immigrate Manitoba. She also launched Meghan PO-MO Project, a sole proprietorship which provided sound reactive visuals for DJs and venues across Canada. In 2009, Meghan was contracted as the User Experience Designer at Tipping Canoe, a multinational internet marketing company.

In 2010, Meghan formed PO-MO Inc. in partnership with Curtis Wachs. She began working exclusively for the company in December, 2010. Today, Meghan is the driving force behind PO-MO Inc."

Curtis Wachs – Technical Director/COO, PO-MO Inc.
"Curtis graduated from Assiniboine Community College in 2003 where he studied object oriented programming. Directly upon graduating, Curtis was hired by Assiniboine Community College to help design and develop software for online classes. Curtis relocated to Winnipeg in 2006 to create interactive training material for sales staff at E.H. Price. During the course of his work, Curt was apprenticed in 3D modelling and animation by Liem Ngyuen, a former Frantic Films resident. In 2008, Curtis joined Manlab, where he created online educational games for Travel Manitoba, Immigrate Manitoba, and other clients. In 2010, Curtis formally joined PO-MO Project, and the company became a partnership. In June 2010, PO-MO Inc. was founded.

Curtis is currently the technical director at PO-MO Inc., overseeing the project management and workflow of contracted and R&D development projects."

"Kinect-based Telepresence with Room Sized 3D Capture and Life Sized Display", Includes Behind-the Scene "how-to" (UNC Chapel Hill)

Jim Spadaccini, of Open Exhibits, recently told me about a project that involves the real-time, interactive 3-D capture of people in a room.  As the viewer moves around the screens, the depth-detecting feature of the Kinect is harnessed to set the stage for a realistic telepresence experience.  


Take the time to view the video, which contains some interesting views of how the system works:





Thanks, Jim, for the link!

RELATED/SOMEWHAT RELATED
Kinect Real-Time Room Telepresence
Kinecthacks, 1/3/12
From the project's website:

Maimone, A. and H. Fuchs. "A First Look at a Telepresence System with Room-Sized Real-Time 3D Capture and Large Tracked Display." The 21st International Conference on Artificial Reality and Telexistence (ICAT) (Osaka, Japan, November 28-30, 2011) [paper] [video]

Maimone, A. and H. Fuchs. "Encumbrance-free Telepresence System with Real-time 3D Capture and Display using Commodity Depth Cameras." The IEEE International Symposium on Mixed and Augmented Reality (ISMAR) 2011 (Basel, Switzerland, October 26-29, 2011) [paper] [video]

Open Exhibits

Monday Musing and Meanderings: Interesting videos and links about emerging technologies.

INTERACTIVE TANGIBLE TABLETOP MUSIC


Reactable Live! SubMixPro Studio Torino

The Reactable featured in the above video is used for DJ-ing in clubs. The one I've played with is at the science museum in my area - I love it. It is fun to improvise on the Reactable with another person.  For more information, see my previous blog posts featuring the Reactable.

AUGMENTED REALITY
While listening to CNBC on my satellite radio on the way home today,  I heard that investing in Qualcomm might be a good idea.   I wonder if this means that Wall Street analysts think that  AR will become mainstream soon...   


The video below shows a variety of creative AR game applications:


Qualcomm has an AR SDK that comes with tutorials, samples, an API reference, and developer forums. The SDK can be downloaded from the Qualcomm AR web-page.

For more information:

GDC 11:  AR- The Augmented Future of Mobile Serious Games?  Eliane Alhadeff, Serious Games Market, 3/7/11

Qualcomm Augmented Reality Research

Qualcomm Extends Augmented Reality with Unity Technologies

Qualcomm's secret mobile weapon:  Augmented Reality Robert Enderle,  TG Daily, 2/24/11

This sort of technology has been around for a while. For an example, take a look at  the 2007 MScape game demo of the AR game, Roku's Reward:

THE KINECT FOR MEDICAL DATA VISUALIZATION 

Here is another use of augmented reality, but this time, harnessing the Kinect:

Magic Mirror for Medical Data Visualization using Kinect

Madhav K. ,  Kinecthacks 

Public Presentation of mirracle

PHOTOELASTIC TOUCH

Transparent Rubbery Interface using a LCD and Photoelasticity
Toshiki Sato, Haruko Mamiya, Taro Tokui, Hideki Koike,  The Unversity of Electro-Communications;  Kentaro Fukuchi, Japan Science and Technology Agency
Abstract (pdf)


ART AND SCREENS

Nomad capacitive brush turns your iPhone into an art canvas for $24.00
Darren Murph, Engadget, 2/22/11

Nomad Brush for the iPad from Don Lee on Vimeo.

Why bother switching from GUI to NUI? - Asked and Answered by Josh Blake; My 2-cents; Stevie B’s Microsoft Research Video; Marco Silva’s NUI-HCI Presentation (and links)

In Chapter 1 of Natural User Interfaces in .NET,  Josh Blake asks and answers a question posed by many people who have been under the spell of keyboard input and GUI/ WIMP interaction: 


Why bother switching from GUI to NUI?  The answer?  Read Chapter 1 (pdf) of the book - the chapter is free.


Here are a few of my personal reasons:  
1.  I want to buy the next version of the iPad or something like it.
2.  I want to buy a new large-screen Internet HD TV.

3.  I want to buy a Kinect.

4.  I do NOT want to interact with my new TV with a Sony remote.  Too many tiny buttons!

5. I do NOT want to interact with my new TV with a keyboard,  because it reminds me of...work.

6.  Most importantly: 

I want to design apps for the people I care about, and others with similar needs:

    My mom.  

    My grandson.

Moms and dads with kids in tow.
People with special needs and/or health concerns, and the people who care and guide them.
Knowledge sharers and (life-long) learners....

RELATED

"Smart" Interactive Display Research

How Natural User Interfaces are changing Human Computer Interaction

 

View more presentations from Marco Silva

My YouTube Playlist:

Post-WIMP Explorers' Club Videos

"Natural user interfaces, gesture interaction, multi-touch, natural interaction, post WIMP examples and more... "

Interactive Multimedia Technology Post:
Interactive Touch-Screen Technology, Participatory Design, and "Getting It": Revisited

RELATED - and somewhat related   

Moms with Apps

Design for Care

SparkOn

NUI Group

UX Matters      "Why don't usability problems get fixed"

Encyclopedia:  Human Computer Interaction, Interaction Design, User Experience, Information Architecture, Usability and More (Interaction-Design.org)

Josh Blake's Blog: Deconstructing the NUI    Book: Chapter 1 (pdf)  Free!
Blake.NUI

"Blake.NUI is a collection of helpful controls, utilities, and samples useful for multi-touch and NUI development with WPF, Surface, and Silverlight."

OpenKinect Wiki

OpenKinect Community Founder Joshua Blake Comes Up With His Own Version of Win 7 Multitouch on Kinect   (Madhav K, KinectHacks 1/14/11)

Kinect SDK Could Open New Worlds for Motion-Controlled Interfaces (Mike Martin, TechNews World, 2/22/11)

Seth Sandler

Seth Sandler's Blog

Seth Sandler's  Ultimate List of Interactive Tables

Dan Saffer

Bill Buxton

Johnny Chung Lee's Procrastineering Blog
Adam Greenfield's Speedbird Blog

 (This is not an inclusive list.)


GUI to NUI Post-WIMP Manifesto:  TBA

Today I hooked up a Wii to the IWB in the school's therapy room. Next - a Kinect?! (Angels are welcome to help us expand the school's "games" programs for students with special needs)

Today, I hooked up our school's Wii to the newly-installed IWB in the therapy room for the first time, and used it during a session for for a social-skills activity with two students who have autism spectrum disorders.  The students helped each other to create their own "Mii" avatars.  I had them play the bowling game, and was impressed how this activity elicited social conversation between the students, with minimal effort on my part.

Most of students in the program at Wolfe are in grades 6-12. They have cognitive, language, and motor delays.  A good number of the students also have autism spectrum disorders. Some have multiple special needs.

We have found that when our students are provided with interactive activities displayed on the large screen of an IWB, they tend to increase their level of attention to their peers and also communicate more with one-another, as well as with the teachers and staff.

By using an IWB for games at school, we are extending the reach of how this technology can be used with students who have more complex special needs.   By providing a means  for our students to learn to play positive games, we help them develop important physical, social, and leisure skills that they can use outside the school setting with non-disabled peers and siblings.  My hunch is that the games will also help promote cognitive/problem solving skills, too.

NOTE:
The funds to purchase the Wii were donated to our school, but did not cover additional games or accessories. We'd like to purchase Rock Band and Wii Fit. We'd also like to expand our program and purchase an Xbox with a Kinect.   An iPad or two would be OK, too : )

If there are any angels out there who'd like to donate funds for our "games" program at Wolfe, please contact me through my Google profile. I will connect you with the appropriate person.

Game designer/developers/researchers/students:
If you are interested in volunteering your efforts to work on a basic cooperative, pro-social game for the Kinect, SMARTTable, or interactive whiteboard, please contact me.

New Radiohead video, Lotus Flower: Will we see Thom Yorke's moves on Kinect's Dance Central or Just Dance 2 anytime soon?

When I saw the new Radiohead video, Lotus Flower, featuring a dancing Thom Yorke, I wondered how his moves might play out in a dance application for the Kinect or Wii.

Wouldn't it be fun to figure out a way to represent the dance "steps" in this video?  Just a thought.  



RELATED
Radiohead Surprises With Early Release
Jon Pareles, New York Times 2/18/11
"Wii Just Dance 2 and Kinect Dance Central: UI and Usability Approaches"

"Hi, Google. My name is Johnny" Johnny Chung Lee leaves Microsoft. (I still wish I could be Johnny Chung Lee for a day.)

"Hi, Google.  My name is Johnny"  Johnny Chung Lee announced on his Procrastineering blog that he's accepted a position at Google as a "Rapid Evaluator".   I'm not sure what he will be doing in this position, but his title is intriguing!

Here are some of my previous posts devoted to the work of Johnny Chung Lee:

I wish I could be Johnny Chung Lee for a Day!  Tracking fingers with the Wii Remote
Video Clips of Projects Inspired by Johnny Chung Lee
More about Project Natal:  Richochet - Great Gaming for Fitness, Johnnie Chung Lee's Contribution


I STILL wish I could be Johnny Chung Lee for a day!

RELATED
Microsoft Kinect Developer Johnny Chung Lee Jumps Ships and Lands at Google
Leena Rao, TechCrunch, 1/18/11
What Microsoft Kinect Defection to Google Means
Rich Tehrani, TMCnet Blog 1/18/11
Microsoft Loses a Top Kinect Researcher to Google
Tricia Duryee,Yahoo! Finance, 1/18/11

Interested in the OpenNI Initiative? OpenKinect? To learn more, read Josh Blake's Interview of Tamir Berliner of PrimeSense

Interview: Tamir Berliner of PrimeSense

Josh Blake, Deconstructing the NUI, 12/9/10

Josh Blake recently interviewed Tamir Berliner, one of the founders of PrimeSense.  If you haven't heard, Microsoft's Kinect was based on work by PrimeSense, and licensed their technology. PrimeSense provides consumer electronics with natural user interaction capabilities. The good news is that the company recently released open-sourced middleware for natural interaction and depth-camera drivers. It will be interesting to see how this will play in the near future!

In the interview, Tamir discussed a number of topics related to postWIMP technologies.  He also announced the newly created  OpenNI, "an industry-led, not-for-profit organization formed to certify compatibility and interoperability of Natural Interaction (NI) devices, applications, and middleware."   It is good to see this level of support for the cause!

Here is a quote from the interview that I especially liked:

"I believe that till today the devices we’ve been using, made us learn greatly lot about them before we could use them and gain their value. I’m pretty sure everyone who is reading this has got at least 3 remotes sitting on his living room table, and at least once a week needs to help someone use their computer/media center/phone/etc. It’s time for that to change and it’s up to us, the technologists to make this revolution happen, it’s time for the devices to take the step of understanding what we want and making sure we get that, even without asking if it’s a trivial task as opening a door when we approach, closing the lights when we leave the room, even making sure we have hot water to shower with when we return from work or wake up in the morning, depends on what we normally do." -Tamir


RELATED
Here are a couple of videos from the OpenNI website that demonstrate OpenNI-compliant applications:

OpenNI-compliant real time skelton tracking by PrimeSense


OpenNI-compliant real time SceneAnalyzer by PrimeSense

FYI: 

Josh Blake is the author of the Deconstructing the NUI blog. Over the past couple of years, he's explored natural user interfaces and interactions through his work on applications designed for Microsoft Surface and Win7 with Windows Presentation Foundation.

About a month ago, Josh organized OpenKinect, an on-line community to support collaboration among people interested in exploring ways to use Kinect with PCs and other devices.  An example of this effort is the open source code, libfreenect, which includes drivers and libraries for Windows, Linux, and OS X. 

The Natural User Interface Revolution
Josh Blake, 1/5/09


Kinect for Xbox 360: The inside story of Microsft's secret 'Project Natal'  (long, but worth reading) David Rowan, Wired UK, 10/29/10


People of libreenect

OpenNI User Guide (pdf)

Interactive Information Visualization for the Kinect? Something like Jer Thop's "Just Landed-36 Hours" might work nicely if revamped!

I follow the O'Reilly Radar blogs and came across a recent post about an information visualization created by blprnt two years ago using Processing. I think it would have great potential if it was re-purposed for use on the Kinect! In the article, Edd Dumbill discusses the advantages of using Processing to create data and information visualizations.  


One example of the power of Processing is an information visualization, "Just Landed -36 Hours, created by Jer Thorp.  Jer gathered tweets from Twitter that included the statement, "just landed", along with location information for each tweet, within a 36-hour period, to create the visualization.


36 Hours- Just Landed is a great 3D visualization of air travel on our planet.  I especially lik the different views that the application provides. As soon as I watched the Just Landed video, I thought it would be great if it could be revamped for use on the Kinect!   (Leave a comment if you know of anyone working on a project in this area.)


Just Landed - 36 Hours from blprnt on Vimeo.


Information about the video from blprnt's Vimeo site:


"I was discussing H1N1 with a bioinformatics friend of mine last weekend, and we ended up talking about ways that epidemiologists model transmission of disease. I wondered how some of the information that is shared voluntarily on social networks might be used to build useful models of various kinds...I'm also interested in visualizing information that isn't implicitly shared - but instead is inferred or suggested...This piece looks for tweets containing the phrases 'just landed in...' or 'just arrived in...'. Locations from these tweets are located using MetaCarta's Location Finder API. The home location for the traveling users are scraped from their Twitter pages. The system then plots these voyages over time...I'm not entirely sure where this will end up going, but I am reasonably happy with the results so far.   Built with Processing (processing.org) You can read more about this project on my blog - blog.blprnt.com"


RELATED
Strata Gems:  Write your own visualizations:  The Processing language is an easy way to get started with graphics
Edd Dumbill, O'Reilly Radar, 12/3/10

Video: DaVinci Surface Physics Illustrator Interface on Xbox Kinect, with gesture interaction, by Razorfish

DaVinci prototype on Xbox Kinect from Razorfish - Emerging Experiences on Vimeo.

RELATED
Razorfish ports DaVinci interface to Kinect, makes physics cool (video)
Time Stevens, Engaget, 12/5/10
Razorfish
(I love this website.)

More gesture and multi-touch interaction! Windows 7 Navigation with Kinect; Product browser by Immersive Labs,

Here are a couple of new natural user interface videos.  The first video, by Evoluce, demonstrates gesture interaction/navigation in Windows 7 applications supported by Kinect. The second video, by Immersive Labs, shows multi-touch product browsing interaction on a large display.

Kinect Treatment of Windows 7, by Evoluce

Evoluce: Leading Surface Technologies


Immersive Labs - Multi-touch Product Browser

Immersive Labs

Drawing with Sounds, an Interactive Musical Drawing App Created with Processing

This is my first creation using the Drawing with Sounds application. This cause and effect application was created with Processing 1.5.1 and produces musical sounds and patterns, along with random abstract shapes that are triggered through movements of a mouse. The application has been adapted for switch users, making it accessible to those who have motor impairments.

Processing is an open-source programming and integrated development environment that was build on the Java programming that has been easy to learn by people who have limited technical backgrounds. It is used by people from from a range of fields, such as art, music, journalism, and data visualization.

The fact that Processing is easy to use makes it great for people with just a little bit of coding knowledge to adopt a variety of open-source applications and tweak a few of the variables to according to the need.   Using the Drawing with Sounds app as an example, I wanted to change the size of the interactive canvas.  By typing in a few keystrokes,  I changed the original size of the screen from 800 x 800 to something larger, as shown below:

I then typed in another variable for the background color, and was pleased with the results:

The application was developed from Andrew R. Brown's tutorial on "SoundCipher", based on the SoundCipher library for Processing.  According to the description of the application, "The sketch is intended for use....as a sensory stimulus using sound, shape and color to create engagement". 

Below is a videoclip created by Keith Manville, demonstrating what happens when you interact with Drawing with Sounds: 

I look forward to sharing more on this topic in future blog posts.  I'll be sure to include basic "how-to" information for my "low-tech" readers and colleagues who desire to learn a bit more about using and creating basic interactive applications that appeal to young people with special needs.  

RELATED

Note:  SEN is the UK acronym for "Special Educational Needs"

Drawing with Sounds Processing sketch

James Winchester, SEN Classroom Blog, 10/31/13

Drawing with Sound

Keith Manville, opensen blog, 10/27/12

"Open-source, Sensory and Interactive Technology in SEN"

ProcessingSEN Wiki

ProcessingSEN Drawing with Sounds Download Page

(An accessible switch-enabled version of Drawing with Sounds can be downloaded from the above link.)

Free sensory applications built in Processing 1.5.1"Download links for sensory applications built in Processing 1.5.1, designed to assist SEN students in engaging in learning through the the use of ICT, interactive “Smart” boards, touchscreen or tablet technologies.  Our experience is that using applications on these devices or with natural user interfaces such as the Microsoft Kinect can increase the opportunities for engagement and social communication with many students."

I will be posting more information about emerging interactive technologies for special needs in the future. In the meantime, take a look at the following resources:

The SHAPE Project, technologies to enhance learning for young people on the Autism Spectrum

Keith Manville, opensen blog, 11/10/12SEN Classroom "Ideas and Tech used in a SEN class"

iPadSEN

GestureSEN Wiki

LeapSEN

KinectSEN Wiki
"Using Kinect in Special Schools for Pupils with Severe Learning Difficulties"
iGazeSEN
OPENSEN Videos