Struktable Multi-touch Installation at TOCA ME Design Conference

Struktable Multitouch Installation from Gregor Hofbauer on Vimeo.


Strukt is a design studio in Vienna, Austria, that specializes in interactive and generative design for a variety of purposes, such as interactive environments and installations, ambient intelligent environments, games, and multi-touch tables, screens, and walls. The video is a demonstration of applications that were presented at the March 2009 TOCA ME Design Conference in Munich, Germany. The applications were developed using vvvv. (More information regarding vvvv can be found at the end of this post.)




INFO FOR THE TECH-SAVVY OR TECH-CURIOUS:

According to information from the vvvv website, vvvv is a "toolkit for real time video synthesis. It is designed to facilitate the handling of large media environments with physical interfaces, real-time motion graphics, audio and video that can interact with many users simultaneously. vvvv is a visual programming interface. Therefore it provides a graphical programming language for easy prototyping and development. vvvv is real time, where many other languages have distinct modes for building and running programs, vvv only has one mode, run-time. vvvv is free for non-commercial use."

VVVV Screenshots
VVVV's Propaganda Page
Other projects using VVVV
Struktable: the 70-inch Multitouch Table

STRUK ON A SPHERE: Interactive installation at a Mercedes Benz conference

Pattie Maes TED Talk: Sixth Sense - Mobile Wearable Interface and Gesture Interaction (for the price of a cell phone?!)

In the following video, Pattie Maes and Pranav Mistry,of MIT's Fluid Interfaces Group, demonstrate SixthSense, wearable technology that incorporates a video camera, a projector, a digital camera:


From the SixthSense website:

"The SixthSense prototype is comprised of a pocket projector, a mirror and a camera. The hardware components are coupled in a pendant like mobile wearable device. Both the projector and the camera are connected to the mobile computing device in the user’s pocket. The projector projects visual information enabling surfaces, walls and physical objects around us to be used as interfaces; while the camera recognizes and tracks user's hand gestures and physical objects using computer-vision based techniques. The software program processes the video stream data captured by the camera and tracks the locations of the colored markers (visual tracking fiducials) at the tip of the user’s fingers using simple computer-vision techniques. The movements and arrangements of these fiducials are interpreted into gestures that act as interaction instructions for the projected application interfaces. The maximum number of tracked fingers is only constrained by the number of unique fiducials, thus SixthSense also supports multi-touch and multi-user interaction."

Photo from the SixthSense website

Of course it is similar to Minority Report, but I don't consider it to be a problem, because it supports multi-touch and multi-user interaction, and it is a "mobile" application of useful technology.

The interaction that had me at "hello world".

Boxee and Digital Convergence

This post is an update to a previous post, Digital Convergence & Interactive Television

Have you heard about Boxee?  I first learned of it today when I came across an interesting post on the Boxee Blog of a debate with Mark Cuban, author of the post, Why Do Internet People Think Content People Are Stupid? on Blog Maverick.

This debate led me to the following related posts:
Boxee CEO: Consumers Will Get a la Carte Online
(Ryan Lawler, Contentinople: Networking the Digital Media Industry)
Bewkes Defends TV Everywhere (Post by Georg Szalai on the Hollywood Report of how Time Warner would provide subscribers to view cable TV network content online.)

What is boxee? Sort of a social-network-internet-cable-TV-Wii application....
"On a laptop or connected to an HDTV, boxee gives you a true entertainment experience to enjoy your movies, TV shows, music and photos, as well as streaming content from websites like Netflix, CBS, Comedy Central, Last.fm, and flickr."

quick intro to boxee from boxee on Vimeo.
From the boxee blog:

"boxee is the developer of the first “social” media center. boxee plays media from your computer and other devices in your home network, as well as connect you to various Internet sources that allow you to stream or download movies, tv shows, music and photos. boxee is based on the xbmc open-source project. we have been working with team-xbmc since early 2007.

We are in the process of alpha testing boxee, so this is why you need to get an invite to participate.. sorry. the alpha is for Mac OS X 10.4 (and above), Ubuntu and Apple TV (a Windows version will come out towards the end of the year"

Web Meets World, Embedded Systems, Interactive Wireless Networks, SmartGrid City, and More

I've been on a wireless sensor networks & embedded systems kick lately, further motivated by the two weeks I spent with my father in the ICU.

Here are a couple of related posts on my Technology Supported Human-World Interaction blog:

The UX of SmartGridCity: Control Your SmartHouse Remotely, Online

"Web Meets World": User Experience of Embedded Systems and Interactive Wireless Sensor Networks..."

Interview of Henry Jenkins on Games-Based Learning and Importance of Collaboration, Learning Ecology, and Media Literacy

Henry Jenkins will be moving from MIT to USC Annenburg School of Communication and the Cinematic Arts School. He'll be teaching courses such as "Transmedia Storytelling" and "New Media Literacies".

"...there is a learning ecology now, one that takes place outside of the classroom in the after school world...and right now, schools are cutting themselves out of the learning ecology by blocking games, by blocking YouTube, by putting filters on the computers. They block off ways the students are technologically connected, from the best ways of learning..and they leave those students who are trapped behind the participation gap from having access to the experiences that prepare the technically literate for the future."

"..Good teachers are fighting a valiant battle just to be able to access the materials of YouTube .. the other day we discovered that students could not access online resources about Moby Dick, the great American novel, because it had the word "Dick" in it."

My high school blocks websites about games. TeacherTube is blocked, too!

In the above video, a school that integrates the use of games within the curriculum is mentioned. The school is Quest to Learn, scheduled to open for the 2009-2010 school year. It is known as "New York's school for today's digital kids".

"Quest supports a dynamic curriculum that uses the underlying design principles of games to create highly immersive, game-like learning experiences for students. Games and other forms of digital media also model the complexity and promise of “systems.” Understanding and accounting for this complexity is a fundamental literacy of the 21st century."




Institute of Play (A partner of the Quest to Learn school)
Quest to Learn Press Links
A Win-Win Scenario: "Game School" Aims to Engage and Educate
Eliza Strickland, Wired 8/6/07

More for Multi-touch: NextWindow Plug-in for Natural User Interface's Snowflake Multi-touch Software -and more.

Those of you have an HP TouchSmart, Dell Studio One PC, or NextWindow displays might be interested in the new NUI plug-in that supports NUI Suite Snowflake software. Here are the features of the plugin, according to information from the Natural User Interface website:

• Detailed user manual included with FAQ
• Developed on fast and reliable C++ platform
• Intuitive
• Customizable
• Gesture recognition library
• TUIO/OSC (Open Sound Control) support (sending and receiving events)
• Low level API
• Hardware accelerated rendering
• Support for wide variety of media types
• Advanced window handler that supports scaling and rotation
• Suitable for Windows® XP and Windows® Vista (Mac OSX and Linux can be developed on request)
• Audio support
• Single, dual support
• Multi-threaded resource handler (For fast data visualization)

"NUI has partnered up with NextWindow™, an international leader in the development of optical multi-touch technology and the manufacturer of optical multi-touch screens, overlays and OEM touch components."

"NextWindow™'s integrated technology allows for natural and intuitive interaction of digital content on flat TFT, LCD and Plasma solutions."

"The NUI NextWindow™ plug-in can be used with any programming language that supports TUIO, i.e. C/C++/C#, Java, Flash, Python, VVVV etc, meaning that software developers can run their own applications on NextWindow™, utilizing the NUI NextWindow™ plug-in."

Comment:
I became a fan of NextWindow touch-screen displays in early 2007 when I worked on a couple of touch-screen projects in my HCI and Ubicomp classes at UNC-Charlotte.

I've been using my HP TouchSmart PC at work with students with disabilities. I'm experimenting with the NUI Suite SnowFlake on my TouchSmart, and found that interacting with the Particles application delighted students with severe autism. The activities provided opportunities to establish joint attention. I also noticed an increase in the number of vocalizations and/or verbalizations among the students. Of course, this was NOT a scientific study.

RELATED
Definition of Joint Attention from UConn:

"Joint Attention is the process of sharing one’s experience of observing an object or event, by following gaze or pointing gestures. It is critical for social development, language acquisition, cognitive development…"

Establishing joint attention is an important step in the development of social interaction skills among young people who have autism spectrum disorders.

More about joint attention:

Joint Attention Study Has Implication for Understanding Autism
Science Daily, 9/29/07

Asperger-Advice: Joint Attention

Autism Games: Joint Attention and Reciprocity

Why is joint attention a pivotal skill in autism?
Tony Charman

Philos Trans R Soc Lond B Biol Sci. 2003 February 28; 358(1430): 315–324. doi: 10.1098/rstb.2002.1199.