Sensory-Mind's Ring Wall, an interactive multi-touch wall you don't even have to touch!

Sensory-Minds is a small company in Germany that is focused on research and design in the field of Natural User Interfaces. If you visit the SENSORY-MINDS website, you'll find that it has been designed for touch interaction.

ring°wall from SENSORY-MINDS on Vimeo.

Information about the Ring Wall from Sensory-Mind's Vimeo site:

".....The two-piece ring°wall consists of a LED display and a multitouch information-wall and impresses by its size: a total surface of 425 square meters, which equals more than 6000 computer displays, is the biggest of its kind. An interactive World emerges out of 34 million pixels generated by 15 high definition projectors and is supported by 30 directional speakers.


Multitouch sensors basing on laser technology, convert the usage of the natural user interface into an experience. By direct touching, more than 80 users can simultaneously get informed about news and activities around the ringworld.


The interactive wall is not only a central information system, but also an innovative advertising tool and medium for public viewings."

Heiko Hoffman of Sensory-Minds recently joined the NUI-Group.  Here is a response to a question on the NUI-group forum about the way the system's sensors work:


"The sensor works like spinning radar gun, or like a wiper in a car, that means like a radar gun you get the distance to the object.  It’s not quiet difficult when you know the position of the sensor and the degree from the beam to get the X,Y position.  Yes, you don’t need to touch the surface but we arranged it that the radar beam is very close to the screen so it seems that you have to touch the surface.

At the moment the system (has) limitations because each sensor (gets) coordinates and this means that you got shadows.This is no problem because the people interact very fast. We are working on a system where the data from the sensors were put together and that would be the solution for this problem."


Note:

I'll be writing a few more posts updating the work of past and present NUI-group members as well as news from the commercial multi-touch & gesture community.  Be sure to check back soon, or subscribe to this blog!

News from NUITEQ and 3M

Various multi-touch and natural-user interface/interaction companies have been collaborating, and results are promising. 

Here is the result of a collaboration between 3M and NUITEQ, the company based in Sweden:

3M Multi-Touch &; NUITEQ Suite Software Video

In this video, you'll see SnowFlake software used in a variety of contexts, such as education, architecture, health, financial consultation, product browsing, games, playing with special effects, and more.












What a difference since 2007, when I shared what I was doing in my graduate classes on this blog!

IntuiLab's Interfaces: Multi-touch applications/solutions for presentation, collaboration, GIS, and commerce

This is a company I've been meaning to write about!   Here is the plug from the IntuiLab website:

"Founded in 2002 and headquartered in Toulouse, France, IntuiLab is a leader in the design and development of surface computing-based applications. Through IntuiFace, the company’s portable, scalable and extensible software surface computing platform, IntuiLab delivers and deploys applications that bring tangible returns on investment to its clients by providing their customers and users with a more natural, immersive and memorable interactive experience. IntuiLab is a Microsoft and Adobe partner and has clients in a wide variety of industries such as retail, food and health, banking, aerospace and defense, telecoms and hospitality."

"We are able to deliver such benefits to our customers thanks to a unique blend of skills (our multidisciplinary IntuiTeam), technology (the IntuiFace Surface Computing Software Platform), process (the IntuiSign design process) and a wide range of partners."

R.U.S.E. on Intuilab's Interface

IntuiLab lets you blow things up with your hands (Jimin Brelsford, CrunchGear 10/09/09)
 It is not all for fun and games:


Amazon Multi-touch Clinet on the IntuiFace Platform

"This video shows an example of how to access to an online retail catalogue (such as Amazon) from a Surface Computer rich client, and create, browse and merge lists of queried or selected items in a natural way." -Intuilabs



 Multi-user Web Browsing on a Windows 7-based 19" 3M Multi-touch




Pictures from the Intuilab website:

If you take a look at Intuilab's "the team" page, you'll find that they look like...graduate students! They probably are, or were, given the size of list of published papers. This tells me that they must have their heart, mind, and souls poured into the business!

IntuiLab's Partners

I should brush up on my French and pay them a visit!

The Convergence of TV , the Internet, and Interactivity: Updated and Revised.

Yesterday I read an interesting article about the future of television on Experientia's Putting People First blog:
Herkko Hietanen: The social future of television.


In this article, Herkko Hietanen, a researcher at Helsinki Institute of Information Technology, is interviewed about his thoughts  about the future of TV.  He observes that "TV is broken" and thinks that "social television" is a concept that needs to be seriously addressed.  "Herkko ends with the observation that social television isn’t a new concept. We’ve seen lots of experimentation with split screens, which allow chat alongside live broadcast. “But television is a lean-back experience,” Herkko offers – you don’t want to share screen estate with your friends. Instead, he believes that social interactions will be before and after the show."


So what's happening now?  I'm not sure if the people on the technology end of the interactive/social TV scene have thought very deeply about how this will play out in our homes and social networks.  Right now, the only way I can access the Interactive TV channel my satellite carrier is through the user-unfriendly remote, which looks something like this.    



My experience with the interactive TV channel on DISH Network has been frustrating. Why should I be forced to use a complicated remote-control system to interact with content?  Why should I be forced to experience a poorly-designed navigation system?   It is common knowledge that remote control systems are poorly designed, despite the fact that companies such a EchoStar have been involved with interactive TV for at least a decade now. 


What puzzles me is that things have not evolved very much, at least in terms of TV and interaction design. Here is an example -the following picture is a screen shot from a recent promo video about Playin' TV, an interactive TV offering that is the result of a collaboration between Dish Network and Echostar.  From what I can gather from the video, the only way to play the games through the user-unfriendly remote control!

Interactive TV innovations from DISH Network:  Playin'TV- Dish Network-Echostar- Promo October 2009 - Play Games on your TV!

(A list of games available for Dish Network subscribers can be found on the DishGames website.)

From the Playin' TV website, I linked to the Visiware website:

"Expert in casual gaming, Visiware is the world-leading provider of games for pay television. Its game channels are carried on more than 30 cable, satellite and IPTV networks and reach more than 120 million people within 77 countries."   


Visiware is behind Playin'TV, Playin' Casino, MiniKids TV, and Playin'Star. Playin'TV games now available on Internet connected televisions. There must be a better way.  Why not control the games with a Wiimote or iPhone?  Visiware might be working on some changes,  from the information on their User Interface and Design web page:  "It’s time for your New Generation Interface Design : Consumers expect innovative yet simple interfaces Compelling, intuitive U.I. is the key to success (Iphone, WII…)"


Digging Deeper
In the video clip below  Bill Leszinske, GM, from Intel Digital Home, discusses the future of interactive television. Consumers want to take their television experience and augment it with the internet experience.  Bill outlines the different ways this can happen:

• Internet access is built into the television.
• The internet can be accessed through the a set-box from a cable or satellite TV carrier
  
• Interactive internet access can be built into a Blu-Ray box or gaming system
• The technology will support 3D games and social networking.

Consumers want to take their TV experience and augment it with an internet experience.
Intel's Next Generation TV: Social Networking, 3D TV

How will technology support this convergence?


The following articles provide an overview of Intel's chip technology, previously known as "Sodaville", called SoC,  System on a Chip: Intel Unveils "Sodaville" Chip for TV Set-Top Boxes (Mark Hachman, PCMag, 9/24/09)


"But putting PC on a TV doesn't work; we know, we tried it," Kim said. "People want an immersive TV experience on their television." People want the power of the Internet on a TV, but they want it "simple," Kim said...What's needed is a pure Internet development framework, Kim said – and the most popular version of that is Adobe's Flash technology. David Wadhwani, general manager of the platform business unit at Adobe, said that the company has opened Flash and removed all license fees, requiring only that manufacturers to open the platform to third-party developers, as part of the Open Screen initiative.
Wadhwani demoed Flash 10 running on an Intel processor, showing full-screen Flash browsing, not to a Web site, but to a custom screen designed by Disney."

"The Sodaville processor uses an Atom core, and Intel has brought "Moore's Law" to shrink the processor to 45 nanometer technology. The Atom Processor CE4100, as it will be formally called, includes a 1080p video engine not to just decompress streams, but also recorded content supplied from another source, such as a hard drive. Intel doubled the speed of its 2D/3D engine, and added support for MPEG-4. The chip uses either DDR-2 or DDR-3 memory."


Intel Technology,  Processing Power Key to TV Revolution (Intel Developer Forum, 9/24/09)
New Intel chips run Web apps on TV sets (Sodaville) (itbusiness.ca, 9/25/09)   Podcast version
In the following video, Intel's work in the area of 3D Internet is discussed:
Intel Introduces the 3D Internet

Intel is also collaborating with Adobe to innovate mobile media production, which most likely lead to some interesting outcomes:


Adobe CS4 and the New Intel Core i7 Mobile
"Rendering is blazing fast." Mobile rendering on the road...anywhere anytime editing...



RELATED LINKS AND THOUGHTS
I previously posted on this topic a few times:
March 2009

Digital Convergence and Interactive Television;  Boxee and Digital Convergence
December 2008:  An Example of Convergence:  Interactive TV: UXTV 2008

In my opinion, there are many factors to consider when thinking about television as we know it, web-based TV, and interactive television.  Technology exists that can support the convergence of the social web and interactive television, but the key players are coming from different directions and with different agendas.  Television still is a "push" medium, and this concept appears to be embedded in the mindsets of people involved with commercial TV programming.

For example, if you watch an episode of your favorite TV show via a network website,  you are forced to watch commercials all along the way.  If you stop the show and resume it after a break, you might even see the SAME commercial again!    This is annoying, just another example of the "push" mentality.  In my mind, this is a form of banner ad and pop-up litter- or even contamination!  Where is the seamless, engaging, innovative UX here?  (There are some examples of progress, such as the ABC's FlashForward website.)


I'm a subscriber to DISH Network, which offers some interactive TV programming.  I went to the DISH Network website to find out more about it, and this is what assaulted my vision:

The website design looks pretty pushy to me. Does this foreshadow the future of Interactive TV?


Here's a screenshot of another DISH Network website:

From this web page you can link to the following web pages: DISH Remote Access: Sling "Your Browser, Your TV"  - links to product overviews:  Slingbox: "Watch your TV anywhere"  SlingPlayer Mobile:  "Extend your Slingbox experience to a mobile phone"  SlingCatcher: " Extend your Slingbox Experience to a TV"  Accessories: "Make your Sling Experience Complete"


An excerpt from Sling's promotional information:
"Founded in 2004, Sling Media, Inc. is a different kind of consumer electronics company - one that's working to demystify convergence technologies and to create empowering experiences for the digital media consumer. The focus of Sling Media is to embrace - not replace - existing products and standards by enhancing them with hardware and software that make divergent technologies compatible and greatly improve the consumer experience. Because, after all, can't we all just get along?! "


"Sling Media, Inc., a wholly owned subsidiary of EchoStar Corporation (NASDAQ: SATS), is a leading digital lifestyle company offering consumer services and products that are a natural extension of today's digital way-of-life. Sling Media's product family includes the internationally acclaimed, Emmy award-winning Slingbox that allows consumers to watch and control their living room television shows at any time, from any location, using PCs, Macs, PDAs and smartphones and the revolutionary new SlingCatcher, a universal media player that seamlessly delivers broadcast TV, Internet video and personal content to the TV. Sling Media is also the company behind the video entertainment web site, Sling.com, offering consumers a wide variety of popular TV shows, movies and other entertainment free for viewing online or on the TV using SlingCatcher."


I managed to find information about DISH's interactive TV offerings  elsewhere on the web:
DISH Network(R) Premieres Interactive Television Experience for New History Series BATTLES BC
DISH Network (R) Announces Winners of 8th Annual Interactive Television Awards


At any rate, here is a smattering of related articles and video-clips related to the future of TV that I'm presently contemplating:
Interactive TV Today:  "InteractiveTV Today [itvt] is the most widely read and trusted news source on the rapidly emerging medium of multiplatform, broadband interactive television (ITV)" 
TV's Killer App?  Guess What, It May Be An App
Joe Mandese, Media Daily News 10/2/09

Ensequence
Video games, Interactive TV, and Cheats


Interactive TV/Internet at the hospital: Interactive TV Gives Patients Access to Movies and Internet
Skylight Internet Access Patient System

I'll add information about the next generation of remote control technology soon.

More Multi-touch and Gesture-based Natural User Interfaces: Bamboo Wacom Tablet; Multi-touch PresTop Kiosk and Snowflake Suite software

Wacom Tablets Get Multi-Touch, Gestures
(Charlie Sorrel, Wired, 9/24/09)
"For the tech-curious, the new tablets have 512 pressure levels in the pen tip and the active area of the tablet is 5.8 x 3.6 inches, and all lose the in-pack mouse (for obvious reasons). The Touch and the Pen models are both $70, and the Pen & Touch is $100. Also, if you were thinking of buying Photoshop Elements 7 for the same price, get a tablet instead — Elements comes in the box."






Official Wacom Video

"Bamboo Touch is new type of computer input device by Wacom that lets you navigate and perform commands like zoom, scroll, rotate and more with a series of simple finger taps and hand gestures. Bamboo Touch brings Multi-Touch capability to your Mac or PC"

Video from a Wacom user:

A nice alternative to a mouse.  I'm going to get one for my laptop!


Multi-touch Kiosks!
Press release:  Dutch touchscreen supplier PresTop partners with Natural User Interface (NUITEQ)
 



RELATED

I couldn't find any video clips of PresTop's multi-touch interaction. From what I can tell, PresTop multi-touch screens will be using SnowFlake Suite from Natural User Interface Technogies AB.

How-to:SnowFlake Suite Flash multi-touch Interactable component (NUIversity)

Without a single line of code, you can do quite a bit with Snowflake Suite

"This video covers how to make a rotatable and scalable image. The beauty about this is, that we have developed a Flash mouse input simulator, so that there is no need for multi-touch hardware in order to develop your applications. Simply simulate multiple mouse inputs for multi-touch.This project is still in alpha phase and a download will become available with the next release of Snowflake Suite 1.7 for the NextWindow platform and camera based multi-touch solutions."


Below is a video of single-touch interaction for PresTop, from Omnivision:


PresTop  PresTop offers interactive hardware and software solutions that can be used indoors as well as in outdoor environments.

Natural User Interface Surfaces: TouchCo - IFSR technology company, founded by NYU Media Reseach Lab's Ken Perlin and Ilya Rosenburg

Earlier this year, I had a very vivid dream about haptic touch-screen overlays. That's how things go for me sometimes. At any rate, the dream inspired a blog post, "Last night I dreamt about haptic touch-screen overlays...". Since this is one of my interests, I try to keep up with developments in this field when I can.

It is amusing to see how seemingly unrelated things have similar roots. Today I came across information about a cool mobile phone concept for the visually impaired, designed by Samsung. The touch technology involved in the concept phone incorporated something called Electric Activating Plastic, or EAP. 


This sparked my curiosity, and while searching for more information, I came across an article in MIT's Technology Review that caught my eye:


A Touch of Ingenuity: Inexpensive pressure-sensitive pad could make surfaces smarter
(Kate Green, MIT Technology Review September/October 2009)

UPDATE:  As of 2/2010, TouchCo no longer exists. It was bought by Amazon:

TouchCo Bought By Amazon 2/3/2010

"..Now that more and more smart phones and MP3 players have touch-screen interfaces, people have grown accustomed to interacting with gadgets using only taps and swipes of their fingers. But on the 11th floor of a downtown Manhattan building, New York University researchers Ilya Rosenberg and Ken Perlin are developing an interface that goes even further. It’s a thin pad that responds precisely to pressure from not only a finger but a range of objects, such as a foot, a stylus, or a drumstick. And it can sense multiple inputs at once."

Ken Perlin and Ilya Rosenberg developed this system as part of their human-computer interaction work at NYU's Media Research Lab, and with the success of their work, went on to start a new company, Touchco, to commercialize and market this concept, known as IFSR (Interpolating Force-Sensitive Resistance) technology.

"The IFSR sensor revolutionizes the field of multi-touch human-computer interaction by out-performing existing solutions in both price and capabilities. Unlike traditional capacitive sensors, our patent pending system can detect any object - not just a finger - and can determine how much pressure is being applied to every point on a sensor simultaneously. IFSR sensors are natively multi-touch, use less power than capacitive sensors, and are much less expensive to produce, making them a highly disruptive technology with widespread market applications."


The Touchco website has a wealth of information regarding this technology. (Update: The TouchCo website was taken down in February 2010)

Other members of Touchco include Nadim Awad, Fang Cheng, Julien Beguin, and Christopher Nam.

SOMEWHAT RELATED
I know of Ken Perlin through his Games for Learning work, and also through his on-line application, Responsive Face, which I use in my work with students who have autism.




Ken Perlin's Blog
Games for Learning Institute

TechPsych Post:
Samsung Braille TouchPhone Prototype is Cool: Uses Electric Active Plastic for functional tactile feedback.

Ron George's Interaction Design Toolbox

Ron George, an interaction designer, recently posted a comment on my blog. So I visited his blog and found that he has a great set of resources that many of my readers would appreciate!

For starters, take a look at Ron's Interaction Design Toolbox page, and then spend some time browsing around the rest of his site/blog.

Ron works at Microsoft on a team that is exploring Natural User Interface/Interaction (NUI). He previously was on the Surface team.

Microsoft's Project Natal & "Fun" Natural User Interfaces for the Office

This looks great!

I came across the picture below today, and discovered more about it in a CNET article:

"Gesture-recognition technology, such as that seen in Project Natal, has changed the face of gaming, but Microsoft's Craig Mundie (pictured) believes it will also transform the office." (Ina Fried, CNET)


This approach looks like it incorporates visual thinking and conceptualization in a nice and fluid way.

RELATED:

"Microsoft: Future desktop PC's will transform the office" (Ina Fried, CNET UK, 7/31/09).

According to Fried, author of the the linked CNET article, Craig Mundie, at the Microsoft Research Faculty Summit this past July demonstrated the set-up in the above picture. The demonstration "...included hologram-like videoconferencing, a virtual digital assistant, and multiple surface computers, along with voice-, touch- and gesture-recognition technology. The desk in the demo was a multitouch surface computer, and the office's walls were also a display that could easily switch from being a virtual window or collection of digital photos to a corkboard of sticky notes or various workspaces. In one case, Mundie also used Project Natal-like depth cameras to put himself in the middle of an architectural demo, essentially putting himself inside a building that was not yet built."

Craig Mundie of Microsoft on the Future of Software: Digital Assistants, Natural User Interfaces, and Room Computing (Gregory T. Huang, Xconomy Seatle, 7/13/09)

The author of the above article was also at the Microsoft Research Faculty Summit and touched on the concept of natural user interfaces:

“All the things we talk about as natural user interfaces have been largely used one at a time as enhancements to [graphical user interfaces],” Mundie said. Gesture recognition, expressive responses, immersive 3-D virtual environments, and understanding of context—these advances in computing algorithms will lead to software that is “better at anticipating what you might want.”

Xbox's Project Natal Revamped for Offices? (J. Nicholas Hoover, 7/30/09)

When I find a video of the Project Natal/Surface/Natural User Interface Office in action, I'll be sure to post it!

Why this is important:

Natural user interfaces might have the potential to support more efficient and effective collaboration among people in the workplace, and the visual display of information may also support better decision-making and problem solving.

Do you have an HP TouchSmart, Dell Studio One, or NextWindow touch-screen? NUITech's Snowflake Suite upgrade provides a multi-touch plug-in.

If you have a NextWindow touch screen display, HP TouchSmart PC, or Dell Studio One PC, you might be interested to know that NUITech has upgraded their multi-touch software, Snowflake Suite. There is an evaluation download available on the NUITech website created for the NextWindow platform. (The TouchSmart and Studio One have NextWindow touchscreens.)

Here is a short clip of Snowflake 1.6 in action:

Snowflake Suite 1.6 provides users with an opportunity to change the standard content that is delivered with the software, which includes images, videos, 3D models, and backgrounds, so it can be customized according to need.

RELATED

Natural User Interface AB has adopted new branding and a new name, Natural User Interface Technologies AB, or NUITech.

Press Release (8/12/09)

Evaluation version of Snowflake Suite for NextWindow systems (including TouchSmart, Studio One)

Snowflake Suite running on multiple NextWindow 2150 overlays for 22-inch screens:

(The previous version of Snowflake, depicted in the above video, was a finalist for the 2009 Stevie Award, "Best New Product or Service of the Year- Media & Entertainment)



Singapore's Public Utility Board display, Singapore International Water Week, featuring NUITech systems and software.

SOMEWHAT RELATED
Press Release: Next Window Earns Coveted Windows 7 Logo Certification pdf

Surface Flight Tracker Video from fboweb labs / flightwise.com, with background music by Art of Noise for your NUI pleasure.

This flight-tracker application for the Surface, looks fun to use. As I watched the video, I realized that it wasn't the application itself that I liked. It was the music that accompanied the video. The choice of music was from the 80's synth-pop band, Art of Noise

Since I'm a music lover, the music got me thinking.

Wouldn't it be great if productivity/work-related applications like Flight Tracker could be developed to provide a means for incorporating a sound-track?

Several thoughts and ideas flashed into my mind:

• Surface and related natural user interface/interaction (NUI) applications have the potential to transform routine, ho-hum work tasks into activities that are a bit more pleasant. Since people often listen to music while they work, it stands to reason that NUI productivity applications should incorporate a music component, at least as an option.
  

• To support a user-centered music platform for NUI applications, the application could incorporate a "smart" music library within the system, with the capability of integrating music libraries and playlists from user's mobile devices, as well as the web, effortlessly.(Of course, there are privacy/security and firewall issues to address, but that is another story.)
  

• Users could have a choice of listening to their own music playlists (including a shuffle option, selecting from a variety of presets, or go for something like the iTunes genius effect, listening to music generated from an algorithm that takes into account music preferences and user interaction with the productivity application over time.
  

• Since many Surface/NUI applications are designed to support collaborative work and interaction between two or more people, the music situation could get a bit complicated, since people have differing tastes. If co-workers disagreed about the music selection, the program would automatically default to generic elevator music, or silence.
  

• NUI applications might even pave the way for a new genre of music. This concept isn't too far-fetched. Think of all the music we've come to love over the years that was composed for movies and even video games!
  

If you know of anyone that is working on this concept, or would like to collaborate with me sometime in the future on this concept, please let me know. I'm slowly working on an interactive timeline prototype, and I have some ideas about adding a music/sound track component.

RELATED
Art of Noise - Close To The Edit (Version 1):

Where are they now? Revisiting Interactive User Interface Projects

I've been blogging about interactive multimedia and interesting human-computer interfaces/ interaction for a while. I thought I'd revisit some of the projects and prototypes that interested me and find out what happened to all of those visionary graduate students after they earned their degrees. (Feel free to leave a comment and a link if you have any information to update about yourself or someone you know.)

This will be a topic I'll touch on from time-to-time, and since I've covered so many projects, it might take quite a while!

In the meantime, I've posted a video of the TANGerINE Inspirational Cube, a mixed reality interactive multimedia project between the University of Florence, University of Bologna, along with the contributions of other researchers. The video was produced in 2007, I think.


Here is a 2008 picture of some of the people who continued to work with the TANGerINE project at CHI 2008:

Here is the video of "TANGerINE Cities: Collaborative Tangible Sonorization" from Nicola Torpei's Vimeo page:

TANGerINE cities - collaborative tangible sonorization from Nicola Torpei on Vimeo.

And here is a link to a Flickr slideshow from TANGerINE Cities at the Frontiers of Interaction 09 conference, held recently in Rome, Italy: http://www.flickr.com/photos/nicolatorpei/sets/72157619553939994/show/ Below is a video of the conference highlights, set to spaghetti western music:


Somewhat Related:
Note: Stephano Baraldi, one of Nicola Torpei's colleagues who worked on the TANGerINE project, worked for a while at a company called Natural Interaction, now part of iO. To read more about iO, take a look at Alessandro Valli's whitepaper, "Natural Interaction, iO"(pdf). Stephano Baraldi's prototype of the Sensitive Table is mentioned in this paper.

iO's Interactive Climbing Wall


(If you just happen to be interested in spaghetti western music, you can listen to it at Last FM: http://www.last.fm/tag/spaghetti+western Ennio Morricone is one of the top artists of this genre.)

Lonely Planet Travel App by Amnesia-Razorfish for the Surface

I came across this video on the Microsoft Surface blog about a demo created for Lonely Planet by Amnesia-Razorfish, from REMIX 2009 Australia this past June:


Lonely Planet proof-of-concept at REMIX Australia 2009

Following a link from the Surface blog, I found yet another video:



Design in the Round. Creating Compelling User Experiences for Surface

The video provides an overview of the history of human computer interaction and look towards the future of NUI (natural user interface). The Surface is viewed as only as a step towards NUI, which follows a person as they go about the day, interacting with technology via a variety of devices and settings.

"Designing for multi-touch, gesture-based and tangible experience like Microsoft Surface presents a new set of challenges. How do you design for a user interface that doesn't have a top? How to allow for multiple simultaneous users without them getting in each other's way?..."

Interactive Touch Tables are Multiplying! Ideum's new 100" multi-touch museum display; Ubisoft's Ruse on a Surface....

Photo from the Museum Exhibit and Design News/Ideum Blog

The above table is 100", created for Nashville's Adventure Science Center's Sudekum Planetarium, is based on Ideum's mt2 table: "Built for museum environments, the high-resolution mt2 Table supports intuitive gestures, allowing visitors to interact and explore custom exhibits. With multiuser support, mt2 Table serves as digital gathering place, allowing visitors to interact with museum content and each other."

Two computers power this table. Natural User Interface's Snowflake handles the optical support and tracking, and the second computer will be running the EM spectrum application, which is developed in Adobe Flash. It is a work in progress. For more information, visit the Ideum blog.

Here is a video trailer of Ubisoft's R.U.S.E. played on a multi-touch table. Much of the trailer appears to be a simulation. I liked the part where the battleships are traveling over the water. According to a blog post, the table depicted in the trailer is the Intuiface, by Intuilab.

Photos of Ubisoft's R.U.S.E. game, running on a multi-touch table:




Photo from the Ubisoft Blog

Intuiface Information PDF
Intuilabs Website

If you visit the Intuilabs website, be sure you interact with the photos of the team! I couldn't get the pictures to change when I touched the screen of my HP TouchSmart PC, so I think they are activated by a mouse-over. It would be nice if they could touch-enable their website.)

Michael Haller Discusses Multi-touch, Interactive Surfaces, and Emerging Technologies for Learning

I came across an excellent overview of interactive display technologies that hold promise for education. The link below is a research article written by Michael Haller for BECTA, formally known as the British Educational Communications and Technology Agency.

Emerging Technologies for Learning: Interactive Displays and Next Generation Interfaces(pdf)
Becta Research Report (2008) Michael Haller Volume 3 (2008)

"Multi-touch and interactive surfaces are becoming more interesting, because they allow a natural and intuitive interaction with the computer system.

These more intuitive and natural interfaces could help students to be more actively involved in working together with content and could also help improve whole-class teaching activities. As these technologies develop, the barrier of having to learn and work with traditional computer interfaces may diminish.

It is still unclear how fast these interfaces will become part of our daily life and how long it will take for them to be used in every classroom. However, we strongly believe that the more intuitive the interface is, the faster it will be accepted and used. There is a huge potential in these devices, because they allow us to use digital technologies in a more human way." -Michael Haller

Michael Haller works at the department of Digital Media of the Upper Austria University of Applied Sciences (Hagenberg, Austria), where he is the head of the Media Interaction Lab.

Michael co-organized the Interaction Tomorrow course at SIGGRAPH 2007, along with Chia Shen, of the Mitsubishi Electric Research Laboratories (MERL). Lecturers included Gerald Morrison, of Smart Technologies, Bruce H. Thomas, of the University oof Southern Australia, and Andy Wilson, of Microsoft Research. The course materials from Interaction Tomorrow are available on-line, and include videos, slides, and course notes.

Below is an excerpt from the discription of the Interaction Tomorrow SIGGRAPH 2007 course:

"Conventional metaphors and underlying interface infrastructure for single-user desktop systems have been traditionally geared towards single mouse and keyboard-based WIMP interface design, while people usually meet around a table, facing each other. A table/wall setting provides a large interactive visual surface for groups to interact together. It encourages collaboration, coordination, as well as simultaneous and parallel problem solving among multiple people.

In this course, we will describe particular challenges and solutions for the design of direct-touch tabletop and interactive wall environments. The participants will learn how to design a non-traditional user interface for large horizontal and vertical displays. Topics include physical setups (e.g. output displays), tracking, sensing, input devices, output displays, pen-based interfaces, direct multi-touch interactions, tangible UI, interaction techniques, application domains, current commercial systems, and future research."

It is worth taking the time to look over Haller's other publications. Here is a few that would be good to read:

M. Haller, C. Forlines, C. Koeffel, J. Leitner, and C. Shen, 2009. "Tabletop Games: Platforms, Experimental Games and Design Recommendations." Springer, 2009. in press [bibtex]

A. D. Cheok, M. Haller, O. N. N. Fernando, and J. P. Wijesena, 2009.
"Mixed Reality Entertainment and Art," International Journal of Virtual Reality, vol. X, p. X, 2009. in press [bibtex]

J. Leitner, C. Köffel, and M. Haller, 2009. "Bridging the gap between real and virtual objects for tabletop games," International Journal of Virtual Reality, vol. X, p. X, 2009. in press [bibtex]

M. Haller and M. Billinghurst, 2008.
"Interactive Tables: Requirements, Design Recommendations, and Implementation." IGI Publishing, 2008. [bibtex]

D. Leithinger and M. Haller, 2007. "Improving Menu Interaction for Cluttered Tabletop Setups with User-Drawn Path Menus," Horizontal Interactive Human-Computer Systems, 2007. TABLETOP 07. Second Annual IEEE International Workshop on, pp. 121-128, 2007. [bibtex]

J. Leitner, J. Powell, P. Brandl, T. Seifried, M. Haller, B. Dorray, and P. To, 2009."Flux: a tilting multi-touch and pen based surface," in CHI EA 09: Proceedings of the 27th international conference extended abstracts on Human factors in computing systems, New York, NY, USA, 2009, pp. 3211-3216. [bibtex]




P. Brandl, J. Leitner, T. Seifried, M. Haller, B. Doray, and P. To, 2009. "Occlusion-aware menu design for digital tabletops," in CHI EA 09: Proceedings of the 27th international conference extended abstracts on Human factors in computing systems, New York, NY, USA, 2009, pp. 3223-3228. [bibtex]


References from the BECTA paper:

Elrod, S., Bruce, R., Gold, R., Goldberg, D., Halasz, F., Janssen, W., Lee, D., Mc-Call, K., Pedersen, E., Pier, F., Tang, J., and Welch, B., Liveboard: a large interactive display supporting group meetings, presentations, and remote collaboration, CHI ’92 (New York, NY, USA), ACM Press, 1992, pp. 599–607.

Morrison, G., ‘A Camera-Based Input Device for Large Interactive Displays’, IEEE Computer Graphics and Applications, vol. 25, no. 4, pp. 52-57, Jul/Aug, 2005.

Albert, A. E. The effect of graphic input devices on performance in a cursor positioning task. Proceedings ofthe Human Factors Society 26th Annual Meeting, Santa Monica, CA: Human Factors Society, 1982, pp. 54-58.

Dietz, P.H., Leigh, D.L., DiamondTouch: A Multi-User Touch Technology, ACM Symposium on User Interface Software and Technology (UIST), ISBN: 1-58113-438-X, pp. 219-226, November 2001.

Rekimoto, J., SmartSkin: An Infrastructure for Freehand Manipulation on Interactive Surfaces,
CHI 2002, 2002.

Kakehi, Y., Iida, M., Naemura, T., Shirai, Y., Matsushita, M.,Ohguro, T., ‘Lumisight Table: Interactive View-Dependent Tabletop Display Surrounded by Multiple Users’, In IEEE Computer
Graphics and Applications, vol. 25, no.1, pp 48 – 53, 2005.

Streitz, N., Prante, P., Röcker, C., van Alphen, D., Magerkurth, C.,Stenzel, R., ‘Ambient Displays and Mobile Devices for the Creation of Social Architectural Spaces: Supporting informal communication and social awareness in organizations’ in Public and Situated Displays: Social and Interactional Aspects of Shared Display Technologies, Kluwer Publishers, 2003. pp. 387-409.

Morrison, G., A Camera-Based Input Device for Large Interactive Displays, IEEE Computer Graphics and Applications, vol. 25, no. 4, pp. 52-57, Jul/Aug, 2005.

Ishii, H., Underkoffler, J., Chak, D., Piper, B., Ben-Joseph, E., Yeung, L. and Zahra, K., Augmented Urban Planning Workbench: Overlaying Drawings, Physical Models and Digital Simulation. IEEE and ACM International Symposium on Mixed and Augmented Reality ACM Press, Darmstadt, Germany.

Han, Y., Low-cost multi-touch sensing through frustrated total internal reflection, UIST ’05 (New York), ACM Press, 2005, pp. 115–118.

Hull., J., Erol, B., Graham, J., Ke, Q., Kishi, H., Moraleda, J., Olst, D., Paper-Based Augmented Reality. In Proceedings of the 17th International Conference on Artificial Reality and Telexistence (Esbjerg, Denmark,November 28-30, 2007). ICAT ’07. IEEE, 205-209.

Haller, M., Leithinger, D., Leitner, J., Seifried, T., Brandl, P., Zauner, J., Billinghurst, M., The shared design space. In SIGGRAPH ’06: ACM SIGGRAPH 2006 Emerging technologies, page 29, New York, NY,USA, 2006. ACM Press.

Research email: emtech@becta.org.uk
Main email: becta@becta.org.uk
URL: www.becta.org.uk

(This was also posted on the TechPsych blog.)

From the NUITEQ (Natural User Interface) Gallery, via Harry van der Veen

Kids take to multi-touch interaction naturally!

The following photos are from Harry van der Veen's Multi-touch blog. (Harry was one of the founding members of the NUI-Group, and also is the CEO of NUITEQ-Natural User Interface)

The last two pictures are of the HP TouchSmart running NUI Suite Snowflake software, developed by the Natural User Interface Europe AB (NUITEQ) for think LCD, Plasma, and FT displays.

Dell Studio One 19 Touch Zone App by Fingertapps: The Video

Here is the video demonstration of Dell Studio One 19 Touch Zone, developed by Fingertapps, a New Zealand software company:



The Dell Studio One with Fingertapp's multi-touch natural user interface software is due for launch soon, according to Ben Wilde and Dave Brebner, of Fingertapps. Here is a link to a recent Engadget article by Paul Miller: Dell demos multitouch on the Studio One 29 (with additional videos)

NUI-Group's Christian Moore Interview-Podcast on the Interactive Display 2009 Conference website

As I write this post, the 2009 Interactive Displays conference is underway in San Jose, California. The home page of the conference website provides a series of interviews of many of the conference participants and presenters, including Chris Moore, a member of the NUI group. Chris discusses the importance of the open-source NUI (natural user interface) community in moving interactive display and NUI-based technology forward.

Interactive Displays 2009 Conference Website

Interview Participants:
Mats Johansson - EON Reality
Guillaume Largillier - Stantum
Christian Moore- NUI Group
Brent Bushnell- Tapcode
Steven Bathiche - Microsoft
Jerry Bertrand - Microscent
Henry Kaufman- Tactable

If you are at the conference and you have some video clips to share of a presentation, demonstration, or anything else that might be of interest, please give me the link so I can share it on my blog.

Below are some pictures from the companies represented by the people who were interviewed for the Interactive Displays 2009 conference:


Eon Reality


Tactable


Microsoft's Surface


NUI-Group


Tapcode


I don't usually put company logos on my blog, but if you are a visiting my blog searching for links to companies that are involved with interactive display technologies, this will save you some time. (I received no payment for this gesture.)