Focused on interactive multimedia and emerging technologies to enhance the lives of people as they collaborate, create, learn, work, and play.
Jan 26, 2010
There is a need for multi-touch/gesture designers/developers!
In my opinion, there will be a need for multi-touch web applications as well as for multi-touch education and collaboration applications for the SMART Table, Microsoft's Surface, multi-touch tablets like the rumored iTablet from Apple, and the multi-touch laptops and all-in-ones (Dell, HP, etc.).
Below are direct links to some of my blog posts related to multi-touch applications and screens. If you are fairly new to multi-touch, I'm sure that looking through some of my blog posts will be helpful. All of the posts have links to resources, and most have photos and video clips of multi-touch in action.
If you are new to this blog, I have a great deal of information, links, photos, and video clips of various multi-touch screens and applications. The best way to find the stuff is to enter in a keyword in the search box for this blog: multitouch, touch screen, gesture, multi-touch, etc. on this blog.
Also do a search on my other blog: The World Is My Interface http://tshwi.blogspot.com
Here are some links:
Do you have an HP TouchSmart, Dell Studio One or NextWindow touch-screen? NUITech's Snowflake Suite upgrade provides a multi-touch plug-in
http://bit.ly/5tdlhc
The following blog post has a video clip that shows someone from Adobe painting with a multi-touch application in development:
More Multi-Touch!: Rumor of the mobile apple iTablet; AdobeXD & Multitouch; 10-finger Mobile Multitouch: http://bit.ly/4S9Upm
Ideum's GestureWorks: http://bit.ly/4C1p7M
Interactive Walls, Interactive Projection Systems, GestureTek's Motion-Based Games: http://bit.ly/6GRGtW
Intuilab's Interfaces: Multi-touch applications/solutions for presentations, collaboration, GIS, and commerce: http://bit.ly/7RK7qN
For software developers:
How to do Multitouch with WPF 4 in Visual Studio 2010: http://bit.ly/7c4YqC
Dec 27, 2009
Touch, Multi-Touch & Gesture Responsive Web & Related Applications (helpful if you have a touch screen or IWB!)
Here's what I have to share today!
Be sure to explore the activities from the Kids section of the National Gallery of Art website, located at the end of this post.
MULTI-TOUCH FIREFOX
Multi-touch on Firefox from Felipe on Vimeo.
Code Snippets from Felipe's Demo (Includes tracking divs, drawing canvas, image resizing, image crop, & pong) Mozilla Wiki
Bringing Multi-touch to Firefox and the Web
Christopher Blizzard, Mozilla Hacks
COOL IRIS
I have a hunch that someone out there is working on a multi-touch version of Cool Iris. Until I can find out the details, take a look at the videos below:
Cool Iris Overview on Google Chrome
Here is a short video of what Cool Iris looks like on an iPhone:
Cool Iris Links
Cool Iris and iPhone
Cool Iris and Developers
Cool Iris Blog
Cool Iris Media/Press
About Cool Iris: "Cooliris, Inc. was founded in January 2006 with a simple mantra: "Think beyond the browser". We focus on creating products that make discovering and enjoying the Web more exciting, efficient, and personal.Our core products include Cooliris (formerly PicLens), which transforms your browser into an interactive, full-screen "cinematic" experience for web media, and CoolPreviews, which lets you preview links instantly. Headquartered in Palo Alto, CA, our team consists of seasoned developers, entrepreneurs, and Stanford computer engineers. Each of us is passionate about serving our users without compromise and seeing that our products deliver the best experience."
BUMPTOP
Bumptop Gets Multi-touch Support on Windows 7
Bumptop Website
You can download Bumptop from the Bumptop website. Here's the description:
"BumpTop is a fun, intuitive 3D desktop that keeps you organized and makes you more productive. Like a real desk, but better. Now with awesome mouse and multi-touch gestures!"
Anand Agarawala's Ted Talk
"Anand Agarawala presents BumpTop, a user interface that takes the usual desktop metaphor to a glorious, 3-D extreme, transforming file navigation into a freewheeling playground of crumpled documents and clipping-covered "walls.""
Discussion about Bumptop on the TED website
NATIONAL GALLERY OF ART KIDS
I've been compiling a list of websites that offer good touch-interaction. One site that is good for children- and children at heart- is the National Gallery of Art's Kids pages. There are a few entries that I had fun playing with students on the new SMARTboards at one of my schools:
FACES AND PLACES - LANDSCAPE

"FACES & PLACES encourages children of all ages to create portraits and landscape paintings in the style of American naive artists. By combining visual elements borrowed from more than 100 works in the National Gallery's permanent collection, this two-part interactive activity offers an overview of American folk art of the 18th and 19th centuries.(Shockwave, 6 MB)."
This one is so fun! You can select different characters and make them dance, run, jump, or even fall. You can design the landscape and add buildings, trees, and animals, and even change the sky pattern. Press "go", and your character will travel around the panorama you've created.
DUTCH DOLL HOUSE

Dutch Dollhouse (Shockwave, 4.6 MB)
"Mix and match colorful characters, create decorative objects, and explore the kitchen, living quarters, artist's studio, and courtyard of this interactive 17th-century Dutch House."
NGA KIDS JUNGLE

"Create a tropical jungle filled with tigers, monkeys, and other exotic creatures. Inspired by the art of Henri Rousseau, NGAkids Jungle is an interactive art activity for kids of all ages. (Shockwave, 930k)"
What I liked about the Jungle application is that each item can be easily customized. On the SMARTBoard, as well as on my HP TouchSmart PC, it is very easy for a student who has limited fine-motor control to create beautiful pictures.
FLOW

"Flow is a motion painting machine for children of all ages. Enjoy watching the changing patterns and colors as you mix pictures on two overlapping layers. Choose designs from four sets of menu icons, or add to the flow by clicking the pencil tool to create your own designs."
This application is a favorite of some of the students I work with who enjoy watching things spin. (You don't have to have an autism spectrum disorder to enjoy playing with Flow!)
National Gallery of Art Student and Teacher Online Resources
Dec 20, 2009
CityScape 1.8: What would it be like on an interactive touch or gesture screen? (Video and link to demo)
Unfortunately, it comes with a high price tag. Even so, wouldn't it be a great tool to use in middle and high school classrooms? I want to get the demo and try it out on a SMARTboard....I can think of quite a few ways that it could be put to work in learning environments.
Nov 23, 2009
GestureTek & Sprint's Interactive Wall: 3D depth-sensing allows wall interaction with a cell phone.
The interactive display can be controlled by a cell phone.
YouTube description/plug:
"GestureTeks 3D depth sensing technology powers an attention-grabbing interactive digital signage system for telecom leader Sprint. The 3D depth sensing interactive display screen, with mobile phone connectivity, tracks peoples body movements, and responds by sending a Sprint promotional message that follows them the entire length of the interactive billboard. The interactive motion-detecting advertising message invites users to create their own personalized interactive wall art on Sprints gesture control screen, by calling Sprint on their mobile phone. GestureTeks 3D tracker is the heart of the system. Installation lead: Mission Electronics. Creative: Goodby Silverstein."
The Instant DJ application looks fun! It allows you to mix the music tracks on the large display with your phone.

Phone Painter: Sprint Center Interactive Wall

Instant DJ

Now Widget

RELATED
Sprint Uses GestureTek 3D Tracking & Control System for New Interactive Digital Signage Campaign
GestureTek Announces 3D Gesture Tracking Initiatives for Sprint and Hitachi; Shares New 3D Patent Information
SOMEWHAT RELATED
GestureFX: Next Generation Pediatrics Business Case (interactive floor for a pediatric clinic's waiting room)
AirPoint Hand-Tracking Unit for Mouse Replacement and "Point to Control" Interactivity
Cross posted on The World Is My Interface blog
Oct 28, 2009
libTISCH, a multi-touch development framework with multi-touch widgets and more!
Florian Echtler announced the first stable releas of libTISCH, a multi-touch development framwork, which can be found on Sourceforge. TISCH stands for Tangible Interaction Surfaces for Collaboration between Humans. libTISCH, a C++ software framework, is included in this project. It provides a means for creating GUIs based on multi-touch and/or tangible input devices.
Here is how it works:

Here is information from libTISCH announcement:
Highlights of this release are, among others, the following features:
- ready-to-use multitouch widgets based on OpenGL
- reconfigurable, hardware-independent gesture recognition engine
- support for widely used (move, scale, rotate..), pre-defined gestures
as well as custom-defined gestures
- hardware drivers for FTIR, DI, Wiimote, DiamondTouch..
- TUIO converters: source and sink
- cross-platform: Linux, MacOS X, Windows (32 and 64 bit)
- cross-language: C++ with bindings for C#, Java, Python
libTISCH has a lot to offer for the multitouch developer. For example,
the textured widgets enable rapid development of applications for many
kinds of multi-touch or tangible interfaces. The separate gesture
recognition engine allows the translation of a wide range of highly
configurable gestures into pre-defined or custom events which are then
acted on by the widgets. While the lower layers of libTISCH provide
functionality similar to tbeta, touche etc. (you can interface existing
TUIO-based software with libTISCH in both directions), it goes far
beyond.
More information about the library and underlying architecture can be
found on http://tisch.sf.net/ and in the Sourceforge wiki at
http://sourceforge.net/apps/
TISCH Project Wiki
RELATED
Florian is on the scientific staff at the Technisch Universitat Munchen in Germany. Be sure to check out his webpage.
I especially like the concept of the MeTaTop: "A Multi-Sensory Table Top System for Medical Procedures" that is linked from Florian's website.
Oct 9, 2009
WIRED's Overview of Touchscreen PC's and Interface Innovations (and some links for the tech-curious)
This is a great opportunity for developers interested in touch, multi-touch, and gesture interaction applications. I think there will be even more opportunity for web developers to create websites that are touch-enabled, or at least optimized for touch screen interaction.
Will the multi-touch web might be right around the corner?
At the moment, there is some confusion about what designers should consider when developing applications. There is not standard system of touch or gesture interaction, and researchers are still very busy figuring things out. Some companies have rushed out and patented gestures, which in my opinion, is like patenting how we breath. But that is another story.
If you are a designer or developer, you might be interested in the Touch First Microsoft Surface Developer Challenge. It is a chance to win your very own Microsoft Surface! Hurry, the deadline is October 12th.
If you want to learn more, read WIRED's Gadget Lab: "Touchscreen PC's Prompt Interface Innovations" for a good overview of what's happening in the Touch PC world.
If you are curious and would like to dig deeper into this topic, take some time to look at some of my blog-posts about multi-touch, touch screen news, innovations, and related musings:
The World Is My Interface: An Introduction (and some links)
Interactive Touch-Screen Technology, Participatory Design, and "Getting It".
Multimedia and Interaction Resources (a mega-list, work-in-progress)
Bump Top 3D Desktop on a Touch Screen: Toss Your Photos to your Facebook Icon!
Ron George's Interaction Design Toolbox
So how are people using their multi-touch all-in-ones? Medion X9613 will be released soon..
Multi-touch, multimedia, multi-modal: Fujitsu LIFEBOOK 54310 has a multitude of possibilties
Windows Vista User Experience Guidelines: "All Windows programs should be touchable!"
Oct 3, 2009
Interactive TV Game Controllers - A variety of permutations for the present and the future (iPhone, iPod Touch, Wii mote, XBox 360 Controller, Gesture...)

How About the iPhone as Controller for Apple TV Gaming Console?
(Read the article's comments section to see what people think about this concept.)

Wii + Internet=More
Video of a real 12-year-old kid navigating through the Wii Internet and Wii News/Earth. He provides a great "think aloud" analysis of his interaction.
"They should make a video game that is geography based on something like this..."
Xbox permutations:
This:

Or maybe even this, if you blog, chat, or tweet:

The following social/interactive applications can be controlled by your Xbox Controller:(The photos were taken from Gizmodo "Microsoft E3 Keynote Archive" and Zatz Not Funny!: Xbox 360: Welcome to the Social?)

Sky Player on Xbox 360 to Launch Mid-October



With the introduction of gesture-based interaction, such as Microsoft's Project Natal, in the future, no controller will be required to interact with your screen, no matter the content.

Project Natal for Xbox 360 could put Microsoft ahead of Wii with controller-free gaming
YOU ARE THE CONTROLLER
XBox Project Natal Website
Motion Sensing Confirmed for 2010 (Sony PlayStation 3)
RELATED
Interactive demonstration of how BBC's Red Button works for interactive TV content. You can press the red button on the mockup, located on the lower right-hand section of the screen.
BBC Red Button Demo
BBC Red Button launches new CBeebies interactive service
Digital Television Group 9/28/09
Social Television and User Interaction
(Scholarly articles on this topic from the ACM Portal)
Stefan Agamanolis (researcher in this area)
Interactive TV Research from UITV.INFO
Sep 15, 2009
Applications for touch, multi-touch, and gesture interaction: What's Marching to Market?
Lenovo ThinkPad T400s with Multitouch Review: Finger Flicking Fun

The Multitouch ThinkPad in Action
Lenovo ThinkPad T400s with Multitouch from Gizmodo on Vimeo.
SCREENSHOT

"The screen is pure beauty. Like a capacitive touchscreen phone, light taps on the display are responsive and you don't have to think about positioning your finger in a certain way to open apps or rearrange windows." - Joanna Stern, Gizmodo, 9/15/09
OTHER EXAMPLES
THE FUJITSU LIFEBOOK T5010
Add the Fujitsu LifeBook t5010 to the Windows 7 Multitouch Laptop List
Gizmodo, Joanna Stern, 9/15/09

DELL LATITUDE XT2

TOUCH DEMO- HP TOUCHSMART TX2 & WINDOWS 7
AIR HOCKEY FOR WINDOWS 7: IDENTITY MINE
FINGERTAPPS
Want to create something multi-touch?
It looks like there will be a growing need for Windows 7 application developers. Here's some code examples using Windows Presentation Foundation and the Windows 7 Multitouch API
Windows 7 Multitouch Application Development, Part I (The Code Project: Kunal Chowdhury)
Other Developer Resources
Touchlib Multi-touch Development Kit (NUI GROUP)
MSDN Windows Touch: Developer Resources
nTrig's Multi-Touch Download for Windows 7
NUITech's Snowflake Suite for NextWindow
TouchKit Multi-touch Development Kit
GestureWorks Flash Multitouch SDK
3M Multi-touch Developer Kit
RELATED
Touch-less Interaction from lm3labs
Create the future you want. with ubiq'window from Nicolas Loeillot on Vimeo.
More about gesture interaction and multi-touch can be found on this blog- do a search!
Aug 8, 2009
More about Project Natal: Richochet - Great Gaming for Fitness, Johnnie Chung Lee's Contribution

(Credit: CNET News)
Ina Fried, in a recent CNET Beyond Binary post, recently reviewed her experience playing Ricochet, a 3D game developed by Microsoft for Natal, the company's new gesture-recognition, controller-less Xbox gaming system. Above is a screenshot from Fried's article, Exclusive: Getting up close and personal with Natal:
Here is the video:
Ina Fried had a chance to spend some time in Redmond, Washington to explore the games in development at Microsoft, and hang out with the people responsible for Project Natal.
In her Beyond Binary article, Fried notes that the Ricochet game provides quite a workout, and this has had a positive effect on the Natal team:
"Since I started working on this project, I've lost almost like 10 pounds," said Kudo Tsunoda, general manager of Microsoft Game Studios and the creative director for Project Natal. "We're going to have the most in-shape development team you've ever seen."
Fans of Johnny Chung Lee will be happy to know that his work at Microsoft contributed to this game in someway, if they don't know this by now!
Who is Johnny Chung Lee? Read my post, "I wish I could be Johnny Chung Lee for a Day!".
RELATED
Speaking of Natal, it should be out next year (Ina Fried, CNET)
Gates: Natal to bring gesture recognition to Windows too
Jul 14, 2009
Mobile Gesture Design at Nokia: Gesture Interaction Around the World
The video explains it all.
(Fritz Desir, thanks for the link!)
May 26, 2009
GestureTek's "Cube": A compact Interactive Gesture-based Display System
From the GestureTek website:
"Introducing The Cube - a compact, turnkey, 'plug and play' interactive display unit that brings the power of gesture control to a variety of display spaces. Project the interactive 80” diagonal display onto almost any floor, wall, table or counter for branding, advertising, entertainment and product promotion. The Cube will engage customers, turn heads and drive business results."
Brochure (pdf)
May 10, 2009
Future of Interaction: Microsoft Lab's Vision 2019 Montage; Future Healthcare Montage
Video Presentation from MIX09: Untold Stories of Touch, Gesture, & NUI
Joseph Fletcher, Design Manager, Microsoft Surface
"Learn about new patterns, behaviors, and design approaches for touch and gesture interfaces from a practitioners point of view. Learn early lessons from applied knowledge of touch applications, devices, and design methods."
LukeW's notes from Joseph Fletcher's presentation
LukeW's post: Design Principles at Microsoft
LukeW is the Senior Director of Product Ideation & Design at Yahoo! Inc.
Apr 25, 2009
How soon will we see interactive information visualization for multi-touch & gesture systems?
Why? Interactive duo and multi-touch interfaces are becoming more common, and now come in all sizes of screens, from the iPhone, the Surface, and CNN's multi-touch "Magic Wall". People of all ages play interactive games on the Wii, often on large flat-panel displays. In my opinion, the time is right for those developing applications for the InfoViz world to think about harnessing the power of multi-touch.







Below is a picture of the front page of the Visual Complexity website. If you go to the site, you can select a visualization, and then explore it more in detail, as each picture links to a web-page that provides background information about the visualization project, the artist or team behind the project, and links to the project's website.
I took a look at a variety of the examples posted on the Information Complexity website, and think many would be enhanced by a transformation to a multi-touch, gesture, bi-manual, or duo/multi user system. I'm interested in learning what others think about this. If you are working on a collaborative information visualization project, feel free to add a comment and post a link.
Here is a a nice quote from the website:
| "Functional visualizations are more than innovative statistical analyses and computational algorithms. They must make sense to the user and require a visual language system that uses colour, shape, line, hierarchy and composition to communicate clearly and appropriately, much like the alphabetic and character-based languages used worldwide between humans." Matt Woolman |
Update: I did a search for "multi-touch" on the Visual Complexity website and found a couple of interactive applications:
Reactable (I've posted about this system a few times!)
(Reactable website)




Dec 14, 2008
Links to videos of natural user interfaces - multi-touch & gesture
Jonathan Brill's Point& Do: "Your Guide to Natural User Interfaces"
More from Lm3labs: Ubiq'window & Reactor.cmc's touch screen shoppiong catalog, AUDI's touch-less interactive showroom screen, and the DNP Museum Lab
Nov 13, 2008
RENCI at Duke University: Multi-Touch Collaborative Wall and Table utilizing TouchLib; More about UNC-C's Viz lab...
RENCI is a multi-disciplinary collaboration between several universities in North Carolina, with centers located at the Europa Center, Duke University, N.C. State, UNC Chapel Hill, East Carolina University, UNC-Asheville, UNC-Charlotte, and the Health Sciences Library at UNC-Chapel Hill. Many of the centers focus on visualization and collaborative technologies, and have been involved in multi-touch "surface" computing.
The pictures below are from the RENCI center at Duke University:

Duke Multi-Touch Collaborative Wall
The multi-touch wall is 13 x 5 feet, and utilizes six high-definition projectors, resulting in a combined resolution of 5760-2160, and supports multiple users. According to information on the RENCI website, the design is scalable and applicable to non-flat surfaces. The wall system runs on Windows and Linux.
(Photo by Josh Coyle)
(Photo by Josh Coyle)
DI, or Direct Illumination is used for touch detection in both the wall and the table for detecting touch. A separate instance of Touchlib runs for each of the 8 cameras used to detect touch. A gesture engine interprets the information about touches on the screen as gesture events. Each camera is handled separately for image processing and blob tracking tasks.
Graphics from the RENCI Vis Group Multi-Touch Blog
Here is cool picture of the "Multi-touch Calibration Device", which uses a built-in TouchLib utility.
Additional information can be found on the RENCI Vis Group Multi-Touch Blog.
FYI
Touchlib is a multi-touch development kit that can be found on the NUI-Group website.
"Touchlib is a library for creating multi-touch interaction surfaces. It handles tracking blobs of infrared light, and sends your programs these multi-touch events, such as 'finger down', 'finger moved', and 'finger released'. It includes a configuration app and a few demos to get you started, and will interace with most types of webcams and video capture devices. It currently works only under Windows but efforts are being made to port it to other platforms."
If you are interested in creating your own multi-touch table, the NUI-Group website and forums are a great place to start.
Related:
If you follow my blog, you probably know that I've taken several graduate courses at UNC-Charlotte. Some of my professors and a classmate or two have been involved in some exciting visualization research over the past year. (If you are serious about multi-touch and other visually-based applications, it is worth taking some time to familiarize yourself with visualization and interaction research.)
News from the UNC-Charlotte Vis Center:
At the University of North Carolina at Charlotte, RENCI is a collaboration between the UNC Charlotte Urban Institute, the Center for Applied Geographic Information Science, and the Charlotte Visualization Center.
11/06/2008
Robert Kosara’s group wins two awards at IEEE VisWeek Caroline Ziemkiewicz and Robert Kosara won Honorable Mention (the second highest award) at the IEEE InfoVis Conference for their paper, “The Shaping of Information by Visual Metaphors”. Also, Alex Godwin, Kosara’s student, won Best Poster for his submission, “Visual Data Mining of Unevenly-Spaced Event Sequences”.

The Vis Center is pretty fascinating, as you can see by the group of visitors at an open house.
If you are just as fascinated by this stuff as the guys in the picture, here are links to some recent papers by UNC-Charlotte faculty affiliated with the Vis Center:
The Shaping of Information by Visual Metaphors (Caroline Ziemkiewicz and Robert Kosara)
Evaluating the Relationship Between User Interaction and Financial Visual Analysis (Don Hyun Jeong, Wenwen Dou, Felsia Stukes, William Ribarsky, Heather Richter Lipford, Remco Chang)
Visual Analytics for Complex Concepts Using a Human Cognition Model (Tera Marie Green, William Ribarsky, and Brian Fisher)






