Multi-player multi-touch: "NuMTyPysics", based on Tim Edmond's Numpty Physics (similar to Crayon Physics)

I have Numpty Physics on my Nokia n800 internet tablet, and Crayon Physics on my HP TouchSmart PC. Both are designed for single touch, and are fun to play. Since my TouchSmart can handle duo-touch input, I wondered what Crayon Physics might be like if it supported two players at once.



As you can see from the video, Thomas Perl and his colleagues have figured this out- at least with Numpty Physics!



The music is worth the watch. It's by Triplexity.

Numpty Physics and Crayon Physics both use the Box2D engine. Here is some information from the website:

"NuMTyPYsics are our enhancements to Tim Edmond's NumptyPhysics game. We added support for receiving TUIO messages from tbeta via PyTUIO by embedding an Python interpreter into the NumptyPhysics code. Currently, we simply emulate mouse input by pushing hand-crafted input events (SDL_Event) onto the SDL event queue (SDL_PushEvent). In the future, we plan to do bi-directional communication between the game engine (written in C++) and our multi-touch handling code, which will be written in Python." -Thomas Perl

Note:
I've use Crayon Physics Deluxe with several of the students I work with who have severe autism. It is amazing how well they can figure out solutions for the levels. It would be even better if it could be enabled for duo-touch. It supports joint attention, which is a very important social interaction skill for young people with autism to develop.

More for Multi-touch: NextWindow Plug-in for Natural User Interface's Snowflake Multi-touch Software -and more.

Those of you have an HP TouchSmart, Dell Studio One PC, or NextWindow displays might be interested in the new NUI plug-in that supports NUI Suite Snowflake software. Here are the features of the plugin, according to information from the Natural User Interface website:

• Detailed user manual included with FAQ
• Developed on fast and reliable C++ platform
• Intuitive
• Customizable
• Gesture recognition library
• TUIO/OSC (Open Sound Control) support (sending and receiving events)
• Low level API
• Hardware accelerated rendering
• Support for wide variety of media types
• Advanced window handler that supports scaling and rotation
• Suitable for Windows® XP and Windows® Vista (Mac OSX and Linux can be developed on request)
• Audio support
• Single, dual support
• Multi-threaded resource handler (For fast data visualization)

"NUI has partnered up with NextWindow™, an international leader in the development of optical multi-touch technology and the manufacturer of optical multi-touch screens, overlays and OEM touch components."

"NextWindow™'s integrated technology allows for natural and intuitive interaction of digital content on flat TFT, LCD and Plasma solutions."

"The NUI NextWindow™ plug-in can be used with any programming language that supports TUIO, i.e. C/C++/C#, Java, Flash, Python, VVVV etc, meaning that software developers can run their own applications on NextWindow™, utilizing the NUI NextWindow™ plug-in."

Comment:
I became a fan of NextWindow touch-screen displays in early 2007 when I worked on a couple of touch-screen projects in my HCI and Ubicomp classes at UNC-Charlotte.

I've been using my HP TouchSmart PC at work with students with disabilities. I'm experimenting with the NUI Suite SnowFlake on my TouchSmart, and found that interacting with the Particles application delighted students with severe autism. The activities provided opportunities to establish joint attention. I also noticed an increase in the number of vocalizations and/or verbalizations among the students. Of course, this was NOT a scientific study.

RELATED
Definition of Joint Attention from UConn:

"Joint Attention is the process of sharing one’s experience of observing an object or event, by following gaze or pointing gestures. It is critical for social development, language acquisition, cognitive development…"

Establishing joint attention is an important step in the development of social interaction skills among young people who have autism spectrum disorders.

More about joint attention:

Joint Attention Study Has Implication for Understanding Autism
Science Daily, 9/29/07

Asperger-Advice: Joint Attention

Autism Games: Joint Attention and Reciprocity

Why is joint attention a pivotal skill in autism?
Tony Charman

Philos Trans R Soc Lond B Biol Sci. 2003 February 28; 358(1430): 315–324. doi: 10.1098/rstb.2002.1199.

Interacting and Communicating with HP TouchSmart Notes: Photo, Video, Audio, and More

The HP TouchSmart Notes application can be used in a variety of interesting ways. I have an HP TouchSmart PC, and I like how easy it is to slide the notes about the screen using this multi-modal application. It allows the user to take capture pictures, video, or audio to create notes, as well as traditional "stickies" that allow for typing or finger-writing input. You can even draw or write over photos. It is an application that has many accessibility features. It also supports many of the principles of Universal Design for Learning (UDL).

The various notes can be put together in a variety of ways, and allow for video or photo storytelling and interesting ways of leaving messages- even a song or two.

I did some hunting and found the more information about some non-verbal students with autism who are learning verbal skills through this technology at Hope Technology School in Palo Alto, California.

Video of student using the HP TouchSmart Notes application at Hope Technology School:






Here are a few "How Two" videos that demonstrate the TouchSmart Notes features and interactions:

HP TouchSmart Voice Notes


TouchSmart Text Notes


The pictures below are from the HP TouchSmart Notes Application Review , by Peter Redmer 11/14/08



Text/Drawing Note Creation Panel


Icons for selecting input method for notes



Photo notes input panel:



You can draw on your photo-notes:

HP TouchSmart PC Supports Communication for Students with Autism

Autism Tech | Consumer Health Video Library - Heath Video

Posted using ShareThis

Hope Technology School

Video modeling software for visual learners and those who have autism spectrum disorders

Activity Trainer, video modeling software from Accelerated Educational Software, supports the following activities:

• Academic
• Communication
• Daily Living
• Non-Verbal Imitation
• Recreation
• Social
• Vocational

A free 30-day trial of the software can be downloaded from the Accelerations Educational Software website.

On the Accelerations Educational Software website, you can find other products, such as Storymovies, which is the product of a collaboration between Carol Gray (social stories), Mark Shelley, and the Special Minds Foundation.

Link to post: Assisting young people with autism spectrum disorders through the use of digital media

If you have an interest in digital media and interventions for young people with autism disorders, take a look at my recent TechPsych post:.

Cognitive Bursts; Sense of Self: Assisting young people with autism spectrum disorders through the use of digital media.

Hug Shirt and Hug from a Distance: Wearables for people with autism spectrum disorders? (Cross post from Tech Psych)

"Hug from a Distance" is a vest that can be electronically controlled to give someone a hug over the Internet. The vest was created by Floyd Meuller, who has a rich background in interactive technologies, multimedia, and interdisciplinary research.

Children with autism spectrum disorders who calm with pressure might benefit from this type of vest.



Another wearable technology is the Hug Shirt.

"Wearers hug themselves, then using Bluetooth technology and their cell phone, they can send it to someone else wearing a Hug Shirt that simulates the feeling of the hug," says Francesca Rosella, creative director of London-based CuteCircuit, which developed the interactive top. "It copies the strength, length, temperature and heart rate of the hug."

It would be interesting to research the effectiveness of these wearable technologies with children and teens who have autism spectrum disorders.

Johnny Lee's Wii Hacks: Can the Wii be configured to provide immersive learning experiences?

Thousands of people have been following Johnny Lee's progress as he hacks Nintendo's Wii system. Johnny Lee is a graduate student in the Human-Computer Interaction Institute at Carnegie-Mellon, and he has come up with the most amazing, creative uses for the various components of the Wi.

Take a look at Johnny's recent video of his work creating desk-top VR using a large-screen display:

"Head Tracking for Desktop Virtual Reality"


Although the Wii is a game system, the possibilities for the Wii as a platform for immersive educational experiences in the classroom - as well as in the family room - are endless, provided that a few people in educational technology are willing to contribute to this effort at the same level demonstrated by Johnny!

Since more classrooms are getting large-screen displays in the form of interactive whiteboards, there might just be a chance for teachers to use a few more tools to invigorate and engage their "digital native" students!

I think that learning activities utilizing the power of the Wii and Johnny's present (and future) hacks might also be very effective in reaching the needs of students who have disabilities. Visual and kinesthetic learners would also benefit, as well as kids and teens who are considered to have hyperactivity.

Johnny provides the code and instruction for all of his hacks on his website at http://johnnylee.net

You can subscribe to project updates at http://procrastineering.com

Johnny Lee's work is creative interactive multimedia technology at its best!

Keepon, BeatBot's robot: Potential to promote social behavior in children with autism spectrum and developmental disorders.

The BeatBots project develops rhythmically intelligent robots for research and entertainment. In the video below, the responsive robot, Keepon, dances to the Spoon song, "Don't You Evah".

Keepon was developed by Hideki Kozima and programmed by Marek Michalowski, from Carnegie Melon.


(The video is available to the public under the terms of the Creative Commons Attribution-NonCommercial 3.0 license.)


According to an article by David Templeton in the Pittsburgh Post-Gazette:

"While the videos have proven Keepon's human appeal, the double-bubble yellow fellow's key role is to interact with children. Keepon has shown promise in encouraging social behavior in children with developmental disorders, including autism."

"Keepon's simple appearance makes children comfortable, and its lifelike movement makes it attractive to them," Mr. Michalowski said. "This combination creates an environment in which social interaction is encouraged.""

"While the robot can dance to almost any song, it also can identify visual and other sensory rhythms, helping to prove how rhythm and synchronization in body language are paramount in human interaction. As such, Keepon has schooled roboticists in how to improve human interaction with robots."

An example of Keepon's attentive and emotive actions:



Another example of Keepon's entertainment value - Spoon's "I turn my camera on":



More videos can be found on the BeatBots website.

iSocial: A Virtual Social Space for Youth with ASD, built with Croquet

iSocial is a 3D environment developed to support conversation and positive social skills for young people who have autism spectrum disorders. iSocial is a work in progress, designed by a team of researchers at the University of Missouri.

iSocial runs on Croquet, an open-source 3D SDK/tool designed to build 3D virtual collaborative environments. (EduSim applications also run on Croquet.)

iSocial Video

My Mind is a Web Browser: Temple Grandin's description of visual thinking

Temple Grandin, author of Thinking in Pictures and Other Reports From My Life With Autism (Vintage Books) 1996, has a website where she's posted interesting accounts of the way she thinks and perceives the world. As a school psychologist, many of the students I've worked with are visual learners and seem to think and perceive the world in a similar manner. For people who are auditory-verbal thinkers and would like to learn more about visual thinking, the article is a good start.

http://grandin.com/inc/mind.web.browser.html

Preschool Playroom - teaching play skills through video modeling for preschoolers with autism

Have you heard of the NECC (New England Center for Children)?

NECC offers a video modeling program to help teach play skills to preschoolers with autism. The videos come with progress monitoring charts. Please post me a comment if you have worked with the NECC Preschool Playroom as a teacher, parent, or related service provider.

First attempt at a touch-screen "Poetry Picture Share" application

This was my first attempt at a "poetry picture share" application. It was designed for use on a multi-touch table and can be accessed remotely so people in different places can move things around on the screen.   The video shows how the application works on a NextWindow Human Touch interactive large-screen display.

Version 2 will be posted soon. I am planning on adapting this application for use with students with special needs, such as those who have autism or other communication disorders.

Embedded Touch-Screen Computer Games for Primates at the Zoo!

Interactive computer games still very common in our public schools, the day-time habitat for most young human primates. Non-human primates at several zoos now have access to "ubiquitous" touch-screen computer games, embedded within their habitats, as part of a research program spearheaded by Atlanta's Center for Behavioral Neuroscience.

The games were developed with assistance from IBM volunteers to help assess reasoning, learning, and memory. For related information and links, see the TechPsych blog entry.



I would love to use (and develop) interactive multimedia computer games, delivered on touch-screens, to help me with my assessment duties as a school psychologist!

If you or someone you know is doing research on this subject in the schools or with school-aged populations, please let me know.

RELATED
Researchers Have Orangutans Play Video Games

MindHabits Demo: Interactive game for identifying facial expressions and increasing self-esteem.

Mind Habits was developed by Dr. Mark Baldwin, a social psychologist, and his colleagues at McGill University.

Visit the on-line demo of Mind Habits , which includes on-line versions of several games.

Vist Mind Habits