Want to make some multi-touch? Try PyMT- Python Multitouch. Featured in Make. (via Sharath Patali)

Sharath Patali, a member of the NUI-Group, has been working with Python Multitouch, otherwise known as PyMT, to create multi-touch applications.  He shared a link to a recent post in Make, featuring PyMT.  Sharath is the author of the UI Addict blog, and is currently doing his internship at NUITEQ (Natural User Interface Technologies).

I've been told that the beauty of PyMT is that it makes it "easy" to create multi-touch prototype applications using very few lines of code, which is great for trying out different ideas in a short period of time.  It helps if you already know Python!


PyMT - A post-WIMP Multi-Touch UI Toolkit from Thomas Hansen on Vimeo.

"PyMT is a python module for developing multi-touch enabled media rich applications. Currently the aim is to allow for quick and easy interaction design and rapid prototype development. PyMT is written in Python, based on pyglet toolkit."


PyMT Programming Guide


PyMT Website

Note: 
Christopher, author of The Space Station blog, is a member of the NUI-Group, and is building his own multi-touch table running his PyMT-based applications. Christopher is a student in Koblenz, Germany, studying computational visualistics, known as information visualization in the US.

Cyan Design's Multi-touch Experiments, Tutorials, and More (Collider Exhibition Series- Interactivity and New Media)

Chris Yanc, of Cyan Design, has been creating multi-touch applications for a while. His work, "Multi-touch Experiments" was included in the Collider Exhibition Series in Akron, Ohio. Below is a video of his experiments, a video of a demo app for an interactive touch conference map,  info about his work with the 36 Views of a Bridge project, and links to his tutorials and code.


Collider Series - Multitouch Table Experiments from Chris Yanc on Vimeo.


About the Collider Exhibition Series (info taken from the website):
"The Collider Exhibition Series examines the impact, implications and inspi- ration of the phenomenon generally categorized under the umbrella term New Media within the design practice and fine arts.Collider: Interactivity and New Media is an initial exploration into this realm. The exhibition seeks to provoke an awareness of the pervasive nature of New Media as it is applied in every function of our society and immerges as a forum for the highest expressions of our contemporary culture. It explores the results of collisions between humans and machines, biology and com- putations, art and technology, thought and process. What is New Media? And what are the implications to artists and designers when worlds, cul- tures and even identities collapse, build and collide."


Chris Yanc's Multi-Touch Demo App:  Tokyo Game Show 09 Conference Map

multitouch demo app - tokyo game show 09 conference map from Chris Yanc on Vimeo.

Chris was involved with the 36 Views of a Bridge at the Bridge Project in Cleveland, Ohio.  His post, 36 Views of a Bridge, explains in detail how his work was created, and also observations of groups of people interacting with the multi-touch table that was part of this project.

The sliding list widget demonstrated in the above video was created using Flash and can be downloaded from Chris's website: Sliding Lists ~ TUIO Flas App Widget.  The link will take you to "how-to" information, a code example,  and also to a link to download the widget.

Chris was kind enough to share his repository of tutorials and code:
http://www.cyancdesign.com/tutorials/   His tutorial page has lots of good links!

I'll be highlighting more work by innovative individuals, groups, and companies from time-to-time in future posts.

BP Oil Spill Multi-touch Map Mashup by Ideum

BP Gulf of Mexico Oil Spill Multi-touch Exhibit

In response to the BP oil spill in the Gulf of Mexico, the team at Ideum has created a free version of their multi-touch-enabled Google Map and Flickr mashup application to educational organizations, including museums and aquariums.  The information included on the map includes oil spill and fishing restriction data from the National Oceanic and Atmospheric Association (NOAA), along with pictures taken at the scene in and around the region of the spill.

Ideum is a company that "designs and creates interactive computer-based exhibits, multitouch installations, rich Internet applications, and social networking sites for museum and Web visitors. We work closely with museums, nonprofit organizations, and socially responsible companies to create memorable visitor experiences."   Ideum uses their own multi-touch framework called GestureWorks to create interactive applications.

Jim Spadaccini is the director and founder of Ideum. He formerly was the Director of Interactive Media at the Exploratorium in San Francisco.

Ideum is working on a version of the program optimized to work on 3M's  22" multi-touch system.  A single-touch version for the web might be available if there is interest.

RELATED

BP Oil Spill in the Gulf of Mexico Flickr Group

Ideum's oil spill-related pictures on Flickr

More from Multitouch Barcelona!

Multitouch Barcelona is up to more great work!

Multitouch Space Invaders XL

Multitouch Space Invaders XL from Multitouch Barcelona on Vimeo.

Guten Touch, by Multitouch Barcelona

GUTEN TOUCH from Multitouch Barcelona on Vimeo.

"Designed for the Red Bull Music Academy 08, Guten Touch is an interactive installation that involves people into a natural relationship with technology. A two projected display system plus a 3m x 2m multitouch wall showcase applications designed to engage us into human friendly experiences rather than flashy and jaw-dropping visualizations. Space Invaders hit by foam balls, pixel paintings created with brushes and digital objects held by hands try to blur boundaries between real and digital."

Multi-touch Barcelona (new site)
"Multitouch Barcelona is a recently born interaction design group that explores natural communication between people and technology. They design touch sensitive environments where real world interactions move to a digital context. Interfaces where people touch, play, move, feel...Where senses play their natural role, where everything just 'happens" as it does in the real world. "




RELATED
Offf 2009: International Festival for the Post-Digital Creation Culture

Update on Ideum: Multitouch, Multiuser Table at the California Science Center

Ideum is a company that has been around since 1999.  It focuses on the design and creation of "computer-based exhibits, multitouch installations, rich Internet applications, and social networking sites for museum and Web visitors."   Gestureworks is the software developed by Ideum, which is an authoring solution for Adobe Flash and Adobe Flex.

Below is some information, including a video clip, of one of Ideum's most recent installations a multiuser, multitouch table at the California Science Center.

Case Study:  Ideum's  L.A. Zone Multi-touch, Multiuser Table (Jim Spadaccini)

Here is an excerpt from Jim Spadaccini's post:
"This multiuser, multitouch table exhibit in the L.A. Zone allows visitors to explore a satellite image and map of the LA Basin and view overlays in thematic areas that explore fire, air, water, and earth. In addition, points of interest are placed on the map in the form of images and videos. All of the content and navigational controls are available in English and Spanish. The exhibit software runs on a custom-built 50" multitouch table that supports 60+ simultaneous points of input."

The Ideum team has spent a great deal of attention to the way groups of museum visitors interact with their multitouch applications. In the past, they've noticed that on a 50" surface, visitors would experience interaction conflicts. In the present application, more than one visitor can manipulate the content without changing the interaction of others.  Spadaccini points out that multi-user control of touch screen exhibits is new, so interface design concepts developed for single-user interaction are no longer useful.  Social interactions around a multiuser display are now something to consider.

In my opinion, what the Ideum team learns through their design process can be quite useful to those of us planning to develop multiuser educational applications for interactive surfaces and displays. What I liked about this exhibit is that it has a monitor that mirrors the display, so people can view the interaction from afar.  This feature would be useful in classroom settings that have an interactive table and an interactive whiteboard. The interactive whiteboard as the secondary display so the teacher and other students could see what the group at the table was exploring.



Multitouch Google Maps and Flickr Mashup Built with Flash



Update: A video showing how Ideum's multi-touch table can withstand all sorts of heavy-duty treatment:




Note:I have been pretty busy over the past couple of months- I have much that I haven't yet blogged about.  More to come!

Quick Link: 3M Invests in Perceptive Pixel, Jeff Han's Multitouch Tech Company

3M Invests in Perceptive Pixel

"3M, through its 3M New Ventures business, has invested in Perceptive Pixel Inc., a developer of advanced multi-touch solutions based in New York City. Terms of the transaction were not disclosed.Founded by multi-touch pioneer Jeff Han in 2006, Perceptive Pixel is dedicated to the research, development and production of multi-touch interfaces for the knowledge worker. The company's hardware and software products enable users to manipulate complex datasets through a new class of intuitive, powerful and visually rich interface techniques. The combination of its technologies with those of 3M will create incredible new opportunities for both companies."

"To see Perceptive Pixel multi-touch solutions in action on 3M Projected Capacitive Technology, see the video at http://www.3m.com/touchPPI. For more information about 3M MicroTouch products, visit www.3M.com/touch. For an overview of popular touch technologies and terminology, visit www.touchtopics.com."

RELATED
3M Touchscreen Products and Systems

Catching up with multitouch pioneer Jef Han Ina Fried, Cnet 10/22/10

Jeff Han's 2006 Ted Talk



Jeff Han, 2007



Thanks to Seth Sandler for the link!

Interactive Mobile Multimedia

A study in January of 2005 by Nokia indicated that there was a demand for interactive mobile multimedia services. That study was conducted about five years ago, before anyone had heard of the iPhone!   The technology to support interactive mobile multimedia has come a long way since then, and many of the new applications support multi-touch, or at least duo touch interactivity.

I'm very much interested in figuring out how to design web-based interactive content (and apps) that can be optimized for touch (and multi-touch/gesture) screens of various sizes, from SmartPhones/iPhones to the large interactive whiteboards that are now in a multitude of classrooms.

From my experience as a school psychologist, I know that there are many teens who have graduated from traditional cell phone to the next level.  If they don't have an iPhone, they have a smart phone.  I don't have the statistics on this, but my personal observations tell me that there are teens who come from families who are from lower economic status who are somehow able to own 3G smartphones.

What a great opportunity to provide casual interactive multimedia educational games to support student learning!  The games and activities could be assigned as homework from time to time, and with the appropriate LMS (Learning Management System), the teacher would have instant access to student progress.  In addition, the students would be provided with immediate feedback about their "work", which we know is an important factor in learning.

It is difficult to figure out the best path to forge, since nearly every week someone announces a new platform, technology, and programming approach!

At any rate, here are a few interesting things related to this topic that I'd like to share.  Many of these concepts are in the experimental phase, but are worth some attention.


SciLor's Open-Source Programs
SciLor's HD2/ Leo Multi-touch Demo v2 12/3/09 using VB.Net and Windows Mobile

SciLor's Comments: 
I have manged it to get Multitouch running in an vb.net app :)
There are still some bugs, which have to be resolved:
-Stop auto alignment!
-Identify the "Touches"


Google I/O 2009: Mastering the Android Media Framework


AT&T Interactive Mobile Website


Satellite-Terrestrial Network Delivering Mobile Video with Interactive Services - ICO mim





RELATED
Adding Multi-Touch to Your Windows Mobile Application's User Interface
Wei-Meng Lee,  DevX.com 3/24/08
Previous Post:  The new 3G iPhone:  Expanding the Possibilities of Interactive Multimedia Communication (Interactive Multimedia Technology,  6/9/08)
Note: I just skimmed the following articles - when I have more time, I'll post more of my reflections related to this set of topics.

Rethinking context framworks: Enabling cross-domain, cross-device experiences (pdf)

Daniel Stewart, Nitya Narasimhan,  Position Paper, CMPPC Workshop, Pervasive 2007

Interactive Mobile Multimedia Needs IP and Circuits
Brough Turner,  Internet Telephony, 9/09
Primetime for Mobile Television:  Extending the entertainment concept by bringing together the best of both worlds (pdf)  IBM Institute for Business Value
A Holistic Approach to Enhance Universal Usability in m-Learning
Vlado Glavinic, Sandi Ljubic, Mihael Kukec, 2008 The Second International Conference on Mobile Ubiquitous Computing, Systems, Services and Technologies
Seamless Mobility: A Continuity of Experiences across Domains, Devices, and Networks (pdf) 2005
Raghu Rau, Senior Vice President of Marketing, Motorola
Ambient Networks:  Cooperative Mobile Networking for the Wireless World
Norbert Niebert, John Wiley and Sons LTD  4/07 
Mobility management challenges and issues in 4G heterogeneous networks (link to ACM pdf)
Sadia Hussain, Zara Hamid and Naveed S. Khattak, InterSense '06.  Proceedings of the first International Conference on Integrated Internet Ad hoc and Sensor Networks
Mobile Multimedia: Tune in to Digital Convergence (pdf)
DVB-Scene (a trade magazine) 3/2008
HP OpenCall Media Plafform: a cost-effective, agile IP media server (pdf)
Whitepaper, Hewlett-Packard Development Company, L.P.  2/2009
A survey of wireless multimedia sensor networks (pdf)
Ian F. Akyildiz, Tommaso Melodia, Kaushik R. Chowdhury, 2006  Science Direct  Elsevier


Mobile Multimedia Companies/R&D
Stantum  (UMPC)

Movidity
MobIME: Mobile Internet for Media and Entertainment

Ideum's GestureWorks vs Adobe AIR 2 and Flash Player 1.0 comparison of multitouch and gesture support

Jim Spadaccini, of Ideum, shared information about his company's product, Gestureworks, highlighting how it provides better multi-touch and gesture support than Adobe AIR2 and Flash Player 10.1. Gestureworks supports multiple-point drag, rotate, and scale at the same time. In the video, the application is demonstrated on an HP Touchsmart 600 and a 3M multitouch screen.

Adobe AIR 2 and Flash Player 10.1  vs Gestureworks 1.0: A direct comparison of multitouch and gesture support


"A direct comparison between the built-in support for multitouch found in Adobe Flash Player 10.1 beta / Adobe AIR 2 and that of the Gestureworks multitouch framework for Flash. More about this comparison can be found on the Gestureworks website (http://www.gestureworks.com) and the Ideum website (http://www.ideum.com)  There is a blog post with more about this comparison and links to all of the example files at: www.ideum.com/2010/01/true-multitouch-wi th-adobe-flash/ "

True Multitouch with Adobe Flash - Jim Spadaccini


GestureWorks Supported Gestures

Example of Ideum's GestureWorks multi-touch, multi-user design for an exhibit a the Vancouver Aquarium:

Musings about NUI, Perceptive Pixel and Microsoft, Rapid Creative Prototyping (Lots of video and links) Revised

It just might be the right time for everyone to brush up on 21st century tech skills. iPads and touch-phones are ubiquitous. Touch-enabled interactive whiteboards and displays are in schools and boardrooms.  With Microsoft's Windows 8 and the news that the company recently acquired Jeff Han's company, Perspective Pixel, I think that there will be good support - and more opportunities- for designers and developers interested in moving from GUI to NUI.    


In the video below, from CES 2012, Jeff Han provides a good overview of where things are moving in the future.  We are in a post-WIMP world and there is a lot of catching up to do!

CES 2012  Perceptive Pixel and the Future of Multitouch (IEEE Spectrum YouTube Channel)



During the video clip, Jeff explains how far things have come during the past few years:

 "Five and 1/2 years ago I had to explain to everybody what multi-touch was and meant. And then, frankly, we've seen some great products from folks like Apple, and really have executed so brilliantly, that everyone really sees what a good implementation can be, and have come to expect it.  I also think though, that the explosion of NUI is less about just multi-touch, but an awareness that finally people have that you don't have to use a keyboard and mouse, you can demand something else beside that.  People are now willing to say, "Oh, this is something I can try, you know, touch is something I can try as my friendlier interface"."

Who wouldn't want to interact with a friendlier interface?  Steve Ballmer doesn't curb his enthusiasm about Windows 8 and Perceptive Pixel.  Jeff Han is happy how designs created in Windows 8 scales for use on screens large and small. He explains how Windows 8 can support collaboration. The Story Board application (7:58) on the large touchscreen display looks interesting.

I continue to be frustrated by the poor usability of many web-based and desk-top applications.  I like my iPad, but only because so many dedicated souls have given some thought to the user experience when creating their apps.  I often meet with disappointment when I encounter interactive displays when I'm out and about during the day.  It is 2012, and it seems that there are a lot of application designers and developers who have never read Don Norman's The Design of Everyday Things!

I enjoy making working prototypes and demo apps, but my skill set is stuck in 2008, the last year I took a graduate-level computer course.  I was thinking about taking a class next semester, something hands-on, creative, and also practical, to move me forward. I can only do so much when I'm in the DIY mode alone in my "lab" at home.  I need to explore new tools, alongside like-minded others.  


There ARE many more tools available to designers and developers than there were just four years ago.  Some of them are available online, free, or for a modest fee.  I was inspired by a link posted by my former HCI professor, Celine Latulipe, to her updated webpage devoted to Rapid Prototyping tools. The resources on her website look like a good place to start for people who are interested in creating applications for the "NUI" era.  (Celine has worked many interesting projects that explore how technology can support new and creative interaction, such as Dance.Draw.) Below is her description of her updated HCI resources:

"New HCI resource to share: I have created a few pages on my web site devoted to Rapid Prototyping tools, books, and methods. These pages contain reviews of various digital tools, including 7 different desktop prototyping apps, and including 8 different iPad apps for wireframing/prototyping. I hope it's useful to others. Feel free to share... and please send me comments and suggestions if you find anything inaccurate, or if you think there is stuff that I should be adding. I will be continuing to update this resource." -http://www.celinelatulipe.com (click on the rapid prototyping link at the top)

IDEAS
Below are just a few of my ideas that I'd like to implement in some way. I can't claim ownership to these ideas- they are mash-ups of what comes to me in my dreams, usually after reading scholarly publications from ACM or IEEE, or attending tech conferences. 

• An interactive timeline, (multi-dimensional, multi-modal, multimedia) for off-the-desktop interaction, collaboration, data/info analysis exploration.  It might be useful for medical researchers, historians, genealogists, or people who are into the "history of ideas".  Big Data folks would love it, too. It would handle data from a variety of sources, including sensor networks. It would be beautiful to use.
• A web-based system of delivering seamless interactive, multi-modal, immersive experiences, across devices, displays, and surfaces. The system would support multi-user, collaborative interaction.  The system would provide an option for tangible interaction.
• A visual/auditory display interface that presents network activity, including potential intrusions, malfunctions, or anything that needs immediate attention that would be likely to be missed under present monitoring methods. 
• Interactive video tools for creation, collaboration, storytelling.  (No bad remote controllers needed.)
• A "wearable" that provides new ways for people to express and communicate creatively, through art, music, dance, with wireless capability. (It can interact with wireless sensor networks.)*
• An public health application designed to provide information useful in understanding and sepsis prevention efforts. This application would utilize the timeline concept describe at the top of this list. This concept could also be useful in analyzing other medical puzzles, such as autism.

Most of these ideas could translate nicely to educational settings, and the focus on natural user interaction and multi-modal i/o aligns with the principles of Universal Design for Learning, something that is important to consider, given the number of "at-risk" learners and young people who have disabilities.

I welcome comments from readers who are working on similar projects, or who know of similar projects.  I also encourage graduate students and researchers who are interested in natural user interfaces to and move forward with an off-the-desktop NUI project.  I hope that my efforts can play a part in helping people make the move from GUI to NUI!  



Below are a few videos of some interesting projects, along with a list of a few references and links.

SMALLab (Multi-modal embodied immersive learning)

PUPPET PARADE: Interactive Kinect Puppets(CineKid 2011)



MEDIA FACADES: When Buildings Start to Twitter

HUMANAQUARIUM (CHI 2012)

 

NANOSCIENCE NRC Cambridge (Nokia's Morph project)






 
Examples: YouTube Playlists
POST WIMP EXPLORERS' CLUB
POST-WIMP EXPLORER'S CLUB II

Web Resources
Celine Latulipe's Rapid Prototyping Resources 
Creative Applications
NUI Group: Natural User Interface Group
OpenFrameworks and Interactive Multimedia: Funky Forest Installation for CineKid
SMALLab Learning

Microsoft Research Natural User Interface 

OpenExhibits: Free multi-touch + multiuser software initiative for museums, education, nonprofits, and students.
OpenSense Wiki 
CINEKID 2012 Website 
Multitouch Systems I Have Known and Loved (Bill Buxton)
Windows 8
Perceptive Pixel
Books
Natural User Interfaces in .NET  WPF 4, Surface2, and Kinect (Josh Blake, Manning Publications)
Chapter 1 pdf (Free)
Brave NUI World: Designing Natural User Interfaces for Touch and Gesture (Daniel Wigdor and Dennis Wixon)
Designing Gestural Interfaces (Dan Saffer)
Posts

Microsoft's"Magic Wall" Tech on Windows 8 Laptops and Tablets by Next Year

Bill Snyder, ReadWrite Web, 7/20/12

I noticed some interesting tools on the Chrome web store - I plan to devote a few more posts to NUI tools in the future.

For the Tech Curious: Multi-Gesture Net: A Multi-touch and Multi-gesture Research Blog

Laurence Muller, M.Sc. is a scientific programmer at the Universiteit van Amsterdam who develops scientific software for multi-touch devices. His blog, Multigesture.Net, provides good information regarding multi-touch and gesture interaction hardware and software applications.


Laurence links to the DYI tabletop computing bootcamp that was held at
IEEE Tabletops and Interactive Surfaces 2008. From there, you can find a linked list of the organizers of the events, and additional information.

Picture below is from MTC Multi-touch Console:


Here is a link to the group's libavg wiki that includes open-source code and "how-to" instructions.

If you are interested in multi-touch and multi-gesture computing from an academic point of view, Florian Echtler, of the Technische Universitat Munchen has a series of publications listed on his website. Here is the abstract of one of his papers. He is on the right track. I especially like the fact that he's thought about widget layers. (I have, too, but they are only sketches in my idea book.)

TICH: Tangible Interactive Surfaces for Collaboration between Humans (Sourceforge website, with links to libtisch.

F. Echtler, G. Klinker
A Multitouch Software Architecture
NordiCHI 2008: Using Bridges, 18-22 October, Lund, Sweden. (bib)

"In recent years, a large amount of software for multitouch interfaces with various degrees of similarity has been written. In order to improve interoperability, we aim to identify the common traits of these systems and present a layered software architecture which abstracts these similarities by defining common interfaces between successive layers. This provides developers with a unified view of the various types of multitouch hardware. Moreover, the layered architecture allows easy integration of existing software, as several alternative implementations for each layer can co-exist. Finally, we present our implementation of this architecture, consisting of hardware abstraction, calibration, event interpretation and widget layers."

Multi-touch Linux on a Stantum SlatePC & more

-Stephane Chatty, Benjamin Tissoires; Video by G. Tabart


Information from the YouTube video:
"This video, shot by ENAC's Interactive Computing Laboratory, demonstrates how one can use out-of-the-box Linux applications with multitouch displays, using the lab's kernel drivers and modified X.org evdev driver. See http://lii-enac.fr/en/projects/shareit/xorg.html for technical details. This work was done during project ShareIT, carried out with Thales Avionics, IntuiLab and Stantum, and sponsored by Aerospace Valley. The ShareIT project explores the use of multitouch interaction for future commercial aircraft cockpits."


The multi-touch software shown in the video was created by IntuiLab.  The Slate PC multi-touch hardware was built by Stantum.  Later in the video, the software is shown running on a MERL DiamondTouch


RELATED
Project ShartIT, ENAC:

ENAC Interactive Computing Laboratory, University of Toulouse, France


Multitouch Interaction: Some Background (from ENAC)


List & Description of some multi-touch devices (from ENAC)


IntuiLab Showroom Pictures


Stantum: Notes from Guillame:
"Key to the expansion of multi-touch is finding the killer app."  

London Design Festival Winners - Visualization Category: Multi-Touch Barcelona; Experiential Category: LM3Labs

Multitouch Barcelona and LM3Labs were among the winners in the London Design Festival!

Visualization Category:  Multi-Touch Barcelona

Human Interface (A Real Human!)

Hi from Multitouch Barcelona on Vimeo.
 

"With a client list that includes Red Bull, you know these guys aren’t messing around. Multitouch Barcelona designs "experiences that merge real and digital into a creative environment where people are invited to touch, play, move, feel as they do in the real world."  Clearly enjoying a tongue-in-cheek methodology, the group’s philosophy seems to grow out of an affinity for natural interaction. The recent work showcased at Offf (a festival for, um, "post-digital creation culture") perfectly shows both their whimsy and tech chops. For example, the Human Interface, a person you use as a computer by asking him to carry out tasks such as email using a keyboard made from cardboard and tubes that carry email around.  Nothing short of hilarious and yet strangely alarming." -Grand designers:  the world's best design work Paul Armstrong,  Wired UK,    8/17/09



I love Multitouch Barcelona. Especially their Space Invaders project.
(I posted this video previously, but I like it so much I decided to post it again!)

Multitouch Space Invaders XL from Multitouch Barcelona on Vimeo.




Another company I follow,  LM3Labs, won in the Experiential category:

Showreel:

Update from Multi-Touch Barcelona and Guten Touch: Multi-Touch Space Invaders

Multitouch Space Invaders Basic Demo from multitouch-barcelona on Vimeo..

For more sights and sounds from this group, visit the Red Bull Music Academy Barcelona 2008 website.

For more about multi-touch technology, including DYI instructions for creating multi-touch tables and displays, open-source code,and tutorials, visit the NUI Group website.

Sensory-Mind's Ring Wall, an interactive multi-touch wall you don't even have to touch!

Sensory-Minds is a small company in Germany that is focused on research and design in the field of Natural User Interfaces. If you visit the SENSORY-MINDS website, you'll find that it has been designed for touch interaction.

ring°wall from SENSORY-MINDS on Vimeo.

Information about the Ring Wall from Sensory-Mind's Vimeo site:

".....The two-piece ring°wall consists of a LED display and a multitouch information-wall and impresses by its size: a total surface of 425 square meters, which equals more than 6000 computer displays, is the biggest of its kind. An interactive World emerges out of 34 million pixels generated by 15 high definition projectors and is supported by 30 directional speakers.


Multitouch sensors basing on laser technology, convert the usage of the natural user interface into an experience. By direct touching, more than 80 users can simultaneously get informed about news and activities around the ringworld.


The interactive wall is not only a central information system, but also an innovative advertising tool and medium for public viewings."

Heiko Hoffman of Sensory-Minds recently joined the NUI-Group.  Here is a response to a question on the NUI-group forum about the way the system's sensors work:


"The sensor works like spinning radar gun, or like a wiper in a car, that means like a radar gun you get the distance to the object.  It’s not quiet difficult when you know the position of the sensor and the degree from the beam to get the X,Y position.  Yes, you don’t need to touch the surface but we arranged it that the radar beam is very close to the screen so it seems that you have to touch the surface.

At the moment the system (has) limitations because each sensor (gets) coordinates and this means that you got shadows.This is no problem because the people interact very fast. We are working on a system where the data from the sensors were put together and that would be the solution for this problem."


Note:

I'll be writing a few more posts updating the work of past and present NUI-group members as well as news from the commercial multi-touch & gesture community.  Be sure to check back soon, or subscribe to this blog!

Stantum's Multi-touch Slate PC, Windows 7 Certified

The Stantum Slate PC is based on the Dell Inspiron Mini netbook platform. The 10.1 Slate accepts handwriting, gestures, and text-based input. Is in the "proof-of-concept stage.

Slate PC Proof of Concept (pdf) Includes specifications. Stantum also offers Multi-Touch Development Kits (MDK's) that include a controller board featuring PMatrix, a controller chip. and a touch panel.

How PMatrix Works (from the Stantum website):


RELATED

Stantum's unlimited multitouch meets Mini 10 in a fight to the death
(Josephe L. Flatley, Engadget, 11/17/09)

JAZZ MUTANT
Jazz Mutant is the Music & Media product division of Stantum.  I offers multi-touch control surfaces for audio and media - Dexter and Lemur.

More information about Stantum and other multi-touch focused companies to come!

Multi-touch application developed by an all-women team from South Dakota School of Mines and Technology for the Imagine Cup Competition

According to a recent article from the Microsoft News Center, "statistics from high schools and universities suggest that percentage won’t change any time soon. Only 17 percent of Advanced Placement (AP) computer science test-takers in 2008 were women, even though women represented more than half of all AP test-takers. At the college level, fewer than one in five computer and information science degrees were awarded to women."


Microsoft's Imagine Cup competition is a way to encourage female students that they can use technology to help make the world a better place, and that computer science is a field that provides an outlet for creativity a innovation.

One of the teams that participated in the Imagine Cup Competition was "Team Blob". The young women in this team attend South Dakota School of Mines and Technology. Their work can be seen in the video presentation of Team Blob's Multi-touch Designer, which allows teachers to create multi-touch presentations for their students. The application was written in C# using Windows Presentation Foundation.


The team developed a interactive timeline to highlight history's famous women in math and science, and demonstrated it on a 40-inch multi-touch table to girls who visited their university campus. The time-line can be seen in the video clip at about 3:34.

"Team Blob members, from left, Lori Rebenitsch, Robyn Krage and Jaelle Scheuerman demo their application that aims to bring emerging multitouch technology into K-12 classrooms. The all-woman team is from the South Dakota School of Mines and Technology."

Musical Multitouch/Gestural Interfaces by Osmosis

I've been pleasantly surprised by the increase of interesting multi-touch and gesture-based applications developed for musical interaction on large displays.  This topic is dear to my heart -  I took a computer music technology class back in 2003, and think it would have been great if this sort of thing was available back then.  Who wants to point and click around a music app for hours on end?!


The video below shows what is new from Osmosis, a company based in NY that focuses on the development of multi-touch and gesture based software for a range of uses, including music applications.


Performance Systems for Stage/Studio from Osmosis on Vimeo.

Transparent Stage System Specs
Design
• Floating, transparent HD displays from 32"
• Haptic surface with tempered glass backing
• High gain image with wide viewing angle
• Rugged aircraft-grade aluminum build
• Enclosed projector and computer
• Minimalist style, compact footprint
• Disassembles for easy transportation


Interactivity
• Projected capacitive foil or IR bezel options
• Up to 32 simultaneous touch points
• Precise, responsive touch tracking (3mm)
• Immune to external light conditions
• Use of fingers, gloves or stylus


Technical
Display:
• DLP portable projector
• 1280×800 HD resolution
• 2500 ANSI-lumen, 1800:1 contrast
Computer:
• Mini-ITX, Core 2 Quad, 4GB RAM
• ATI Radeon X1250 graphics card
• Wireless keyboard and mouse
• Windows 7 Pro

Stuart McClean, the founder of Osmosis, shared the following information about his company:

"Osmosis is a consulting firm based in the NYC area with deep experience in interactive technology. Although we cater to a range of markets, we’re especially passionate about music production and performance. Working closely with artists, we build customized interactive systems for stage and studio. HCI technology is integrated into a range of designs including stands, desktop rigs, tables, carts and vertical screens. Interfaces are tailored to specific artist needs and combine controls, generative audio and visuals, instruments, and gestural input. Our unique and flexible systems take full advantage of multi-touch interaction and offer seamless control of Ableton Live, Traktor, or other DAWs via midi and OSC..."





For more information about the applications developed by the Osmosis team, take a look at their showcase page.  

Unlocking the Future of Cities through Multi-Touch Interactive Visualization at RENCI (UNC-Charlotte)

Here is a link to an article that was in the SciTech section of my morning paper today!


Unlocking the Future of Cities:  UNCC scientists work across disciplines to predict how urban areas will use open land. Tyler Dukes, Charlotte Observer, 10/31/10

"As part of a three-year, $286,000 grant from the National Science Foundation, the group of scientists from UNC Charlotte is researching the complex relationship between the Queen City and its surrounding forest and pastoral lands. Using a combination of social, natural and computer science, they're working to build an interactive map-based simulation capable of showing the impact of future development and policy on land use....It's a project requiring Meentemeyer's team to peel back multiple layers of cultural and economic values surrounding land in the South. The research will have implications beyond the Charlotte area...By allowing the public to explore those possibilities visually on anything from a laptop to a touch-screen table, the research team is hoping its work will mean more informed decisions about how people use the land around them."  -Charlotte Observer

Image Source: Charlotte Observer


Wouldn't this be a great tool to use to support collaborative learning projects in the schools?


RELATED
RENCI at UNC-Charlotte has a Multi-touch Table in the Visualization Center
RENCI Visualization Center Update
Visualization Resources at RENCI UNC-Charlotte
RENCI at UNC Charlotte
Multi-Touch at RENCI
Research by Touch:  RENCI Multitouch Table Gives Computer Science Research an Intuitive Interface

For Techies and the Tech Curious: Multi-Touch/Gesture from the NUI-Group

If you are a new visitor to this blog and interested in interactive multimedia, you'll need to know more about the NUI-Group. Natural User Interface or ~ NUI Group is an interactive media group researching and creating open source machine sensing techniques to benefit artistic and educational applications.
(For related information, please read my recent post, Usability, Accessibility, and User Experience in a Win7 Environment.)

Seth Sandler, of the NUI-Group, sent out a great email with links and resources for people who are interested in multi-touch/gesture interaction, hardware, and/or software development. The list of NUI-Group members who have completed projects is listed below, with links to project websites as well as related threads on the NUI-Group forum.

(The information can be found on the NUI-Group Wiki, which boasts a nice icon based front-page)

Thanks, Seth, for organizing this wealth of information!

The following projects are divided up by type. (Links to information about the various types of multi-touch and gesture systems can be found near the end of this message.)

FTIR - Frustrated Total Internal Reflection

Name: Seth (cerupcat)
Project Name: AudioTouch
Project Website: http://ssandler.wordpress.com
Project Thread: http://nuigroup.com/forums/viewthread/1352/ http://nuigroup.com/forums/viewthread/2309/

Name: bassmang5
Project Name: Æ-table
Project Thread: http://nuigroup.com/forums/viewthread/3144/

Name: Daniel (Zin)
Project Name: Prometheus
Project Thread: http://nuigroup.com/forums/viewthread/2612/
Name: Carsten (carschdn)
Project Name : aTRACKtive
Project Thread: http://nuigroup.com/forums/viewthread/3223/
LLP Laser Light Plane



Name: Denis Santelli(dsan)
Project Website: http://www.touchwall.fr/
Project Thread: http://nuigroup.com/forums/viewthread/3051/

LED-LP Light-Emitting Diode Laser Plane

Name: Nolan - (PeauProductions)
Project Name: PeauProductions (LCD)
Project Website: http://peauproductions.blogspot.com/
Project Thread: http://nuigroup.com/forums/viewthread/3291/

DI - Diffused Illumination

Name: Seth (cerupcat)
Project Name: MTmini
Project Website: http://ssandler.wordpress.com/MTmini
Project Thread: http://nuigroup.com/forums/viewthread/1731/

Name: Fairlane
Project Name: ORION v2
Project Website: http://orionmultitouch.blogspot.com/
Project Thread: http://nuigroup.com/forums/viewthread/1709/

Name: Abdullah (EfeNDy)
Project Name: EfeNDy’s Diffused Illumination MT
Project Thread: http://nuigroup.com/forums/viewthread/3176/

Name: Sandor
Project Name: EXPO REAL 2008
Project Website: http://vimeo.com/2240537
Project Thread: http://nuigroup.com/forums/viewthread/3576/

Name: Matthew (MatthewW)
Project Name: Design Garage
Project Website: http://www.gotuasciencecenter.org/
Project Thread: http://nuigroup.com/forums/viewthread/3730/

Touchless

Name: Jimi Hertz
Project Name: Touchless Wall
Project Website: http://sassexperience.org/projettouchwall.html
(try: http://sassexperience.org/multitouch/inprogress.html )
Project Thread: http://nuigroup.com/forums/viewthread/2414/
YouTube: http://www.youtube.com/watch?v=KCFbWPf37jw

Other Types:
DSI Diffused Screen Illumination




RELATED LINKS

Tips for Success

• Read the FAQ
• Know Your Terminology
• Understand the Techniques
• Start Small
• Study Completed Community Projects
• Plan, plan, plan

Resources

• NUI Group Tools

• NUI Group Wiki
• Play a Part - Contribute
• Frameworks and Libraries
• Multitouch Business Directory

Thanks to all NUI-Group members who have been working so hard at this mission!

I'll end this post with a YouTube video created by Jimi Hertz, a NUI-Group member. "MULTI-TOUCHLESS WALL HOW TO?"

I especially like the music!