Seth Sandler's "How to Make a Cheap Multitouch Pad" YouTube video is going viral...

Seth Sandler, a member of the NUI group, has worked very hard at putting together low-cost multi-touch screen surfaces. Seth has a background in music, so his projects focus on interactive music applications. Over 315,000 people watched this video within the week that it was uploaded.

Thanks Seth, for sharing this vision with the world!



Seth's AudioTouch Blog: "An Interactive Multi-user, Multi-touch Musical Table and More"

Hint for high school teachers: This sort of thing would be a great project for an after-school technology club!

Why bother switching from GUI to NUI? - Asked and Answered by Josh Blake; My 2-cents; Stevie B’s Microsoft Research Video; Marco Silva’s NUI-HCI Presentation (and links)

In Chapter 1 of Natural User Interfaces in .NET,  Josh Blake asks and answers a question posed by many people who have been under the spell of keyboard input and GUI/ WIMP interaction: 


Why bother switching from GUI to NUI?  The answer?  Read Chapter 1 (pdf) of the book - the chapter is free.


Here are a few of my personal reasons:  
1.  I want to buy the next version of the iPad or something like it.
2.  I want to buy a new large-screen Internet HD TV.

3.  I want to buy a Kinect.

4.  I do NOT want to interact with my new TV with a Sony remote.  Too many tiny buttons!

5. I do NOT want to interact with my new TV with a keyboard,  because it reminds me of...work.

6.  Most importantly: 

I want to design apps for the people I care about, and others with similar needs:

    My mom.  

    My grandson.

Moms and dads with kids in tow.
People with special needs and/or health concerns, and the people who care and guide them.
Knowledge sharers and (life-long) learners....

RELATED

"Smart" Interactive Display Research

How Natural User Interfaces are changing Human Computer Interaction

 

View more presentations from Marco Silva

My YouTube Playlist:

Post-WIMP Explorers' Club Videos

"Natural user interfaces, gesture interaction, multi-touch, natural interaction, post WIMP examples and more... "

Interactive Multimedia Technology Post:
Interactive Touch-Screen Technology, Participatory Design, and "Getting It": Revisited

RELATED - and somewhat related   

Moms with Apps

Design for Care

SparkOn

NUI Group

UX Matters      "Why don't usability problems get fixed"

Encyclopedia:  Human Computer Interaction, Interaction Design, User Experience, Information Architecture, Usability and More (Interaction-Design.org)

Josh Blake's Blog: Deconstructing the NUI    Book: Chapter 1 (pdf)  Free!
Blake.NUI

"Blake.NUI is a collection of helpful controls, utilities, and samples useful for multi-touch and NUI development with WPF, Surface, and Silverlight."

OpenKinect Wiki

OpenKinect Community Founder Joshua Blake Comes Up With His Own Version of Win 7 Multitouch on Kinect   (Madhav K, KinectHacks 1/14/11)

Kinect SDK Could Open New Worlds for Motion-Controlled Interfaces (Mike Martin, TechNews World, 2/22/11)

Seth Sandler

Seth Sandler's Blog

Seth Sandler's  Ultimate List of Interactive Tables

Dan Saffer

Bill Buxton

Johnny Chung Lee's Procrastineering Blog
Adam Greenfield's Speedbird Blog

 (This is not an inclusive list.)


GUI to NUI Post-WIMP Manifesto:  TBA

Elastic Creative, Delphi Productions, GestureTek, & the Metronic HRS multi-touch table and wall

I came across this on the Elevating Interactivity blog:
Medtronic HRS Conference Tables & Walls -Elevating Interactivity, 11/10/09

Elastic Creative and Delphi Productions collaborated to create the Metronic HRS multi-touch table and wall application.  The hardware system was by GestureTek.

Medtronic is a medical technologies company that has been around for many years. Some of their projects include the Activa deep brain stimulation therapy used to treat Parkinson's disease and other conditions, pain management therapies, and more.






Here are the videos:




Medtronic HRS 2009 Multitouch Table from Elastic Creative on Vimeo.

SOMEWHAT RELATED
The following is a video featuring Elastic Creative's work with Colardo Creative for Immersion, a company that creates systems and technology that integrates touch feedback into products, including medical simulators used in training physicians for minimally invasive surgical techniques.

Immersion Brand Video from Elastic Creative on Vimeo.

BBC Video about Immersion's medical simulation application:



Elevating Interactivity's 6 Levels of Interaction

Struktable Multi-touch Installation at TOCA ME Design Conference

Struktable Multitouch Installation from Gregor Hofbauer on Vimeo.


Strukt is a design studio in Vienna, Austria, that specializes in interactive and generative design for a variety of purposes, such as interactive environments and installations, ambient intelligent environments, games, and multi-touch tables, screens, and walls. The video is a demonstration of applications that were presented at the March 2009 TOCA ME Design Conference in Munich, Germany. The applications were developed using vvvv. (More information regarding vvvv can be found at the end of this post.)




INFO FOR THE TECH-SAVVY OR TECH-CURIOUS:

According to information from the vvvv website, vvvv is a "toolkit for real time video synthesis. It is designed to facilitate the handling of large media environments with physical interfaces, real-time motion graphics, audio and video that can interact with many users simultaneously. vvvv is a visual programming interface. Therefore it provides a graphical programming language for easy prototyping and development. vvvv is real time, where many other languages have distinct modes for building and running programs, vvv only has one mode, run-time. vvvv is free for non-commercial use."

VVVV Screenshots
VVVV's Propaganda Page
Other projects using VVVV
Struktable: the 70-inch Multitouch Table

STRUK ON A SPHERE: Interactive installation at a Mercedes Benz conference

Multi-touch Crayon Physics

Also posted on the TSHWI blog:
Watch how you can draw simple shapes that can instantly turn into a game!


Multitouch Crayon Physics from multitouch-barcelona on Vimeo.
For a better version of this video, see http://www.vimeo.com/980528 For more information, see the RXSurface blog post. On May 18, Multi touch crayon physics will be offered as an alphabeta opensource! The people behind RXSurface are members of the Natural User Interface (NUI) group:
"Natural User Interface or ~ NUI Group is an interactive media group researching and creating open source machine sensing techniques to benefit artistic and educational applications."

Gesture Markup Language (GML) for Natural User Interaction and Interfaces

Quick post:

Gesture ML Wiki 

"GML is an extensible markup language used to define gestures that describe interactive object behavior and the relationships between objects in an application.  Gesture Markup Language has been designed to enhance the development of multiuser multi-touch and other HCI device driven applications." -Gesture ML Wiki

GestureML was created and maintained by Ideum. 

More information to come!

Photo credit: Ideum

RELATED
Ideum Blog

OpenExhibits Free multitouch and multiuser software initiative for museums, education, nonprofits, and students

GestureWorks  Multi-touch authoring for Windows 8 & Windows 7

Ideum's GestureWorks Platform Will Let Developers Create Complex Gestures for Ultrabook Apps  -Alex Williams, Tech Crunch, 11/18/12

5 Perspectives On The Future Of The Human Interface -Alex Williams, Tech Crunch, 11/4/12

NUI Group

More to come!

Blast from the 2009 past: News, Videos, and Links about Multi-touch and Screen Technologies

One of the things I like to do is share updates about the world of multimedia, multi-touch, gesture, screen, surface, and interactive technologies, focusing on off-the-desktop applications and systems. When I started this blog, I had to put forth quite a bit of effort just to FIND interesting things to blog about.  


These days, there are so many sources that focus on emerging - and now commonplace- interactive technologies, my main challenge is to filter the noise.  Where do I begin?

My archives are vast.   I randomly picked the year 2009 and came across one of my previous posts, "News, Videos, and Links about Multitouch and Screen Technologies."   The post is long, and contains a number of videos and links that probably will be of value to a future curator of the history of technology.


I welcome comments from readers who might be able to help me update information about various applications and systems I've featured on this blog in the past. 

The pictures are screenshots from the results of an  image search for "interactivemultimediatechnology".  Over the past 6 years, I've posted quite a few!

More Multi-touch: Multi-touch Table at Schlossmuseum Linz, by Strukt Design Studio

Schlossmuseum Linz / Multitouch Installation from Strukt Studio on Vimeo.

Info from the Strukt website:
"The first game Strukt produced is called “Solar Land”. The visitors can place solar panels on a map of Upper Austria, guessing where they would be most efficient according to the altitude of the sun in the area. After all items are placed, the participants can start a simulation of the insolation over the duration of an entire year. The game shows how much energy is produced during that period of time, and the top simulations are listed in a high-score. The results encourage people to discuss their decisions and to play the game once more, using the knowledge they gained to reach a better score."





-Picture from the Strukt website


Strukt studio is located in Vienna, Austria, and is a design agency that specializes in interactive media for events and exhibitions.  Strukt has an innovative portfolio of interesting work.

More News, Videos, and Links About Multi-touch and Screen Technologies: Eatang's "Holographic" Multitouch, Globacore & Blue Ocean Screens; NextWindow, NUITEQ, FingerTapps, Stantum, GUNZE USA, GestureTek

This post was recently updated and moved. You can find it at by visiting the link below:

UPDATED: More News, Videos, and Links about Multi-touch and Screen Technologies

So how are people using their multi-touch all-in-ones? Medion X9613 will be relased soon in Germany, next, the US?

The Medion x9613 is a Windows 7, multi-touch all-in-one that has all of the features needed for playing games, working collaboratively with a colleague or classmate, watching HD video content, and more.

The following YouTube video is from Gizmodo, via Engadget.


According to Cali Lewis, of GeekBrief TV, this all-in-one will be available in the US. (Cali also previews other interesting gadgets and tech on the GeekBrief TV clip below.)



RELATED
Medion X9613: A Multitouch PC that Longs to Be Your Home Theater
Mark Wilson, Gizmodo


I haven't set eyes on this one yet. If anyone has played with a Medion X9613 a bit, or even used it for work, please leave a comment! How do YOU use your "All-in-One?"

Steven Sinofsky Discusses Multi-touch and the HP TouchSmart, Windows 7, and more at PDC2008

Steven Sinofsky at the PDC2008 Open Space


In this video, Steven discusses Windows 7 and confirms that Windows 7 works with the HP TouchSmart: "The hardware is multitouch." Sinofsky also discusses how there might be a need for multi-touch drivers for the HP TouchSmart.

Take the 25 minutes or so to watch the video. It is worth it, even the more technical aspects, including how events will be managed to minimize boot time.

Dude!
Sinofsky says you gotta say "dude!". Watch the video to get it!

iPhone: Multitouch control of a laptop screen!

The Media Computing Group at RWTH-AACHEN University has developed an application to allow users to control a laptop via a multi-touch iPhone. Although the demo shows how to rotate and resize images, the "multiple views" section is interesting. Like an Etch-a-Sketch, you shake the iPhone to reset the images.

I've been toying with ideas about ways to create user-friendly interactions between screens of all sizes. This approach intrigues me.



You can find more information about the iPhone and Cocoa on the Multi-touch Framework website. The contact for this project is Stefan Hafeneger, a student assistant

Good news about the San Jose Interactive Displays Conference (that I couldn't attend)

Sadly, I was unable to attend the Interactive Displays conference that was held last week in San Jose, California. Jeff Han, the guy in the video clip in my sidebar, was one of the keynote presenters. I really wish I could have seen - and touched - the action!

Update (4/27/09) from Thomas Hansen, a member of NUI Group, who attended IDC:

"..It was certainly an interesting event. I got to talk (albeit briefly) to both Andy Wilson and Jeff Han. Both of whom produce very inspiring and world class UI/HCI research on a consistent basis. Further, it was a great honor to meet some of the other members of the nuigroup community in person, some of whom where inspiring not only because of their amazing intellect and artistic talents, but especially due to their friendliness, benevolence, and maturity."

Here are a few excerpts from people who were fortunate to attend and then write about the experience:

Putting our arms around the future of touch Ina Fried , CNET 4/23/09

"..if you used one of the interactive displays here to show a heat map of this industry, it would glow red hot. That's because touch displays, for years relegated to kiosks and industrial uses, are quickly becoming mainstream. Hewlett-Packard and Dell already have touch-capable machines, while Microsoft is set to make gesture input standard with Windows 7...For his own part, Han said he was inspired by seeing a PBS documentary in the early 1980s that showed Microsoft researcher Bill Buxton, then at the University of Toronto, using multitouch to compose music on a computer. The computer itself was a green screen with an ancient processor and little memory, but the key underlying concept was already there..

"Sometimes it takes that long for these things to marinate and gestate," Han said....And while things are now taking off, Han urged the crowd not to forsake quality in the rush to take advantage of a hot market. "That will ruin it and mess it up for all of us, and that would be a real shame," Han said."

Interactive Displays Conference Highlights Kevin Arthur, Touch Usability 4/22/09

"...He (Jeff Han) showed a great clip from an early 80s TV show called Bits and Bytes that featured a young Bill Buxton demonstrating some of his tablet work at the University of Toronto...Jeff Han's point in showing this clip was not just to share what inspired him as a kid to pursue computer science as a career. He also wanted to make the point that none of this stuff is really new. He urged the audience not to jump on whatever tech is cool this week, but to be aware of the history and to do the research. Be thoughtful and careful about what you're making. He said one of his fears with the multitouch craze is that the waters will be poisoned by bad and poorly conceived implementations. He said "don't add noise" to the ecosystem by using terms sloppily -- like "multitouch"...

The importance of being more thoughtful and mindful of prior work are not exactly new to most of us with design, HCI, or CS backgrounds, but the audience here is largely made up of marketing or other business types, I believe, who sometimes tend to get a bit carried away, you might say. I mean no disrespect to my friends in marketing..."

Interactive Displays Conference San Jose Harry van der Veen, 4/25/09
" Big thumbs up for Pira tech for managing to get so many multi-touch industry professionals (Wacom, Mindstorm, NextWindow, 3M, Jeff Han (Perceptive Pixel), LG, Tyco Electronics, Stantum and more) and hobbyists together."

Human Computer Interaction + Informal Science Education Conference (NUI News)

I recently learned of the HCI + ISE conference, funded by the National Science Foundation and organized by Ideum and Independent Exhibitions that will provide the groundwork for the future of the development and design of interactive computer-based science exhibits.

Science museums have a long history of interactivity, well suited to groups of "explorers", such as families or students visiting on a field trip.  

What is really exciting is that new interactive applications and technologies have the power to transform the way people learn and understand science in a collaborative and social way.  Innovations in the field of HCI - Human-Computer Interaction- such as multi-touch and gesture interaction, are  well-suited to meet the goals of science education for all, beyond the school doors and wordy textbooks. 

Below is a screen-shot of the conference website, a description about the conference, quoted from the site, and some related resources.

About the HCI+ISE Conference

"HCI technologies, such as motion capture, multitouch, augmented reality, RFID, and voice recognition are beginning to change the way computer-based science exhibits are designed and developed. Human Computer Interaction in Informal Science Education (HCI+ISE) is a first-of-its-kind gathering to explore and disseminate effective practices in developing a new generation of digital exhibits that are more intuitive, interactive, and social than their predecessors."

"The HCI+ISE Conference, to be held in Albuquerque, New Mexico June 11-14 2013, will bring together 60 museum exhibit designers and developers, learning researchers, and technology industry professionals to share effective practices, and to explore both the enormous potential and possible pitfalls that these new technologies present for exhibit development in informal science education settings."

"HCI+ISE will focus on the practical considerations of implementing new HCI technologies in educational settings with an eye on the future. Along with a survey of how HCI is shaping the museum world, participants will be challenged to envision the museum experience a decade into future. The conference results will provide a concrete starting point for exhibit developers and informal science educators who are just beginning to investigate these emerging technologies and design challenges in creating these new types of exhibits."

Why HCI+ISE?
"Since the mid-1980s informal educational venues have increasingly incorporated computer-based exhibits into their science communication offerings in an effort to keep pace with public expectations and make use of the expanding opportunities these technologies provide. The advent and popularity of once novel HCI technologies are becoming commonplace: the Wii and Microsoft Kinect now allow for motion capture video games, tablet PCs have multitouch interaction, and smart phones and other devices come standard with voice recognition. Yet many museums are still developing single-touch and trackball-driven, single-user computer kiosks."

"Science museums have a long history of championing hands-on, physical, and inquiry-based activities and exhibits. This vast experience has only just begun to be applied to interactive computer interfaces. Along with seasoned science exhibit developers, the Conference will draw upon individuals outside of ISE who will provide fresh insight into the technologies, design issues, and audience expectations that these visitor experiences present."

Involvement and Findings
"HCI+ISE will bring together a diverse group of practitioners and other professionals to discuss (and in some cases share and prototype) new design approaches utilizing emerging HCI technology. Please see our Apply page to learn how you can participate. Conference news and findings will be distributed through a variety of ISE and museum websites, including this one."

"We welcome your questions and comments about the HCI+ISE Conference."

CONTACTS

Kathleen McLean of Independent Exhibitions
& Jim Spadaccini of Ideum
HCI+ISE Co-chairs

RELATED
Independent Exhibitions

Open Exhibits

"Open Exhibits is a multitouch, multi-user tool kit that allows you to create custom interactive exhibits."
CML:  Creative Mark-up Language
GML: Gesture Mark-up Language
GestureWorks
Ideum

ICE PAD: Interactive Multitouch Ice Sculpture by Art Below Zero (video)

ICE PAD: Interactive Multitouch Ice Sculpture by Art Below Zero


Here is the information about the interactive sculpture from the Art Below Zero YouTube Channel:

"Created by David Sauer & Max Zuleta for the Lake Forest Tree Lighting Festival.This Ice Crystal Display was the 1st to be created in the USA, Transforming 300 pounds of ice into the equivalent of a giant Ipad touch screen. "People always want to touch our Ice Sculptures, This Interactive Display gave them the perfect reason to get their hands cold." said Max Zuleta owner of Art Below Zero. The public response was amazement and interest in the workings of the touch screen in ice. Our favorite guess was "It must work by sensing body heat!"..."

"...The system is known as Rear Diffused Illumination or Rear DI. It works because an Infrared light is shone from the opposite side of the ice wall through the ice. When an object such as a finger, hand, or mitten stops the infrared light it reflects the light back to a custom camera built by Peau Productions. The illuminated objects are then converted to points of interaction using an open source program Community Core Vision which outputs TUIO data streams to a Flash program for animation. We like the look and feel of the Fluid Solver flash application. The output from the computer is then projected into the ice and ice diffracts the light into something beautiful. By this method the user can manipulate a visible light screen via an invisible light that only the camera can see..."


Thanks to Nolan Ramseyer, of PeauProductions, for the link!
PeauProductions Blog: Multitouch and Technology


RELATED
Ubice = Multi-touch On Ice at the Nokia Research Center in Finland (Video + Pic via Albrecht Schmidt)
Art Below Zero

Designing for Multitouch Tables and Surfaces, by Erin Rose, Open Exhibits Blog

If you are interested in exploring collaborative tabletop applications, take a look at the Open Exhibits blog. Erin Rose's recent post, "Designing for Multi-touch Tables and Surfaces", is a good overview of lessons learned over the past couple of years in design, development, and implementation of multi-user interactive tabletop applications.

Although the focus of Open Exhibits is on applications and systems designed for museum exhibits, many of the design challenges hold true for similar applications in other settings, such as classrooms, libraries, and other public spaces.

Erin's post explores each of the following topics in more detail:

• Don't forget that the table is omni-directional.
• Individual control of objects encourages multi-user interaction.
• Promote collaboration, founded in healthy competition.

(Erin Rose is a developer and community liaison for Open Exhibits.)

RELATED
Exhibit Files
Jim Spadaccini
Visitors Explore L.A. in Google Maps and Flickr Mashup.

Top 10 Interactive Multimedia Technology Blogposts, November, 2010

Here are the "top ten" posts for this blog, according to the number of visitors during the month of November, 2010:

Hacked Kinect Multitouch using libFreenect and libTISCH (via Florian Echtler)
11/13/2010


Power to the Pixel Cross-Media Forum Streaming Live from London Today!
10/12/2010


Revised Post 8/1/06: Interactive multimedia for social skills, understanding feelings, relaxtion, coping strategies, etc.
7/12/06


Interactive Touch Screen Technology, Participatory Design, and "Getting It": REVISED
11/11/2010


Microsoft Surface Light and Physics App for Kids at the Smithsonian
11/24/2010


Interactive iPad Apps for Kids with Autism: Could some of these be transformed for multi-touch tabletop activities?
11/06/10


New Version of Surface from Microsoft?
11/10/10


Tech Product Placement & Embedded Advertising: Cisco Telepresence, Surface, Kinect, Windows 7 Phone, Apple, Apple iAd - Videos, Links - plus legal & ethical concerns
11/29/10


Digital Newspaper from News Corp, for the iPad (via physog, Guardian)
10/18/10


Online Physics Games for Interactive Whiteboards and Touch Screens (including mobile devices)
10/18/10

NUI-Group Members: What are they doing now?

Multitouch Media Application Pro v3.0 from Falcon4ever on Vimeo.

MMA Pro is a multitouch photo and video organizer build in Adobe AIR (Flex3) and has new features such as Google Maps, support for uploading pictures on the fly using blue-tooth. For more information, visit Laurence Muller's website, Multigesture.Net. There you can download the application. Make sure you read the install instructions that are included in the readme.text, and also make sure that you have the latest Adobe AIR 1.5.x. Laurence also recommends installing BlueSoleil to handing the pairing of devices and file transfers. (If you've never programmed for Bluetooth, take his advice!)

Laurence Muller (M.Sc.) is a Scientific Programmer at the University of Amsterdam who develops scientific software for multi-touch devices. He is a member of the NUI-Group.

The following video highlights some of the applications from the University of Amsterdam from about a year ago:

Multitouch Applications from Falcon4ever on Vimeo.

Feel free to leave a comment and a link or two if you are a NUI-Group member and like to share your recent projects!

Dell Studio One 19 Touch Zone App by Fingertapps: The Video

Here is the video demonstration of Dell Studio One 19 Touch Zone, developed by Fingertapps, a New Zealand software company:



The Dell Studio One with Fingertapp's multi-touch natural user interface software is due for launch soon, according to Ben Wilde and Dave Brebner, of Fingertapps. Here is a link to a recent Engadget article by Paul Miller: Dell demos multitouch on the Studio One 29 (with additional videos)

RENCI Visualization Center Update & Link to Innovative Interactivity post

I've been meaning to visit a few of North Carolina's RENCI visualization centers, but I just haven't had a moment to squeeze the field trip in my schedule.  Tracy Boyer, who authors the Innovative Interactivity blog, had a chance to visit the center in Chapel Hill


For those of you who can't visit a RENCI center, visit Tracy's blog and read about her first-hand experience as a visitor: RENCI pioneering the visualization industry with innovative interfaces
I found a link to the following project, "The Docuverse",  deep within one of the RENCI websites:


The Docuverse: 1.5 million documents on the screen at once.

"The Docuverse is a method of displaying an entire digital library on a screen at once. The Docuverse takes a digital library of HTML documents and a set of queries and creates a “universe” around these queries. Each “galaxy” corresponds to a library query. Each “star” corresponds to a document. The most relevant documents found through a query are the stars located close to the center of a galaxy. Documents perceived by a search engine as less relevant are out among the arms of the galaxies. The visualization is interactive, allowing the user to zoom in and out of galaxies, change queries, and overlay different kinds of data mining results onto the visualization to show how documents with certain characteristics are distributed throughout the collection."

-RENCI


Below are a couple of videos from RENCI's YouTube channel:





Unity 3D Game Engine running on a multi-projector dome system using JavaScript:

FYI:  C#/WPF App to open Unity plug-in window at custom dimension

RENCI Focus Areas (from the RENCI website):
Biosciences & Health
Computing & Technology
Data & Information Management
Disaster & Environmental Research
Economic Development
Education and Outreach
Humanities, Arts and Social Science
Visualization & Collaborative Environments
Project Archive


RELATED
Multi-Touch at RENCI
Research by Touch:  RENCI Multitouch Table Gives Computer Science Research an Intuitive Interface
RENCI Anchor at Europa Center
RENCI at Duke University
RENCI at ECU
RENCI at North Carolina State University
RENCI at UNC Asheville
RENCI at UNC Chapel Hill 
RENCI at UNC Charlotte
RENCI at UNC Coastal Studies Institute
RENCI at UNC HSL

How to visit RENCI at UNC Chapel Hill