SPARSH: DYI demo of an open-source multi-touch table and applications by NUI-group members

The following video is a demonstration of "Sparsh", an interactive multi-touch FTIR table built in eight weeks by a group of engineering students in India. Most of the information regarding the hardware and software you see running on this low-cost system can be found on the open-source NUI-group website, forums, and wiki.


Sparsh Multitouch Display from anirudh on Vimeo.

I especially like the multi-touch DJ application!

For more information, view the posts related to the NUI group on this blog.

Sparsh Website

Eye Tribe Eye Tracker Dev Kit, $99; Open Source ITU Gaze Tracker Grows Up!

The Eye Tribe Eye Tracker developer kit is available for pre-order for $99.00. The kit comes with an SDK with C++, C#, and Java, full source code included.  

I've been waiting for a while to see this happen! 

The Eye Tribe Eye Tracker is an outgrowth of the work of a group of researchers at the IT University of Copenhagen.  At the time, it was known as the open-source ITU Gaze Tracker. 
I came across it a few years ago in a NUI-Group forum, and later wrote a post about it when the 2.0 version was released. 

Although the Eye Tribe Tracker was originally developed to meet the needs of people with disabilities who could not access computers, it was found to have potential for a number of other uses that were not really possible before the spread of mobile technologies such as touch-screen tablets and smart phones. 

To get a better understanding of eye-gaze/tracking technology, take a look at the following videos and follow the related links.



Below is a demonstration of the gaze UI on an Android smartphone:


Here is another look at this technology running on a Windows 8 Tablet:





RELATED
The Eye Tribe (website)
Eye Tribe starts taking pre-orders for $99 Windows eye tracker
Senseye will let you control your mobile phone with your eyes
Martin Bryant, The Next Web, 12/2/11
Open-source Eye-tracking: The ITU Gaze Tracker 2.0 Beta Via Martin Tull, NUI-Group Member
Lynn Marentette, Interactive Multimedia Technology, 11/1/10
ITU GazeGroup
Gaze Tracker Development
GazeGroup Forum
Martin Tall


RELATED VIDEOS
Eye Tribe was formally known as Senseye. Below is an earlier video that shows how it worked with a web-cam on a mobile device:



Open-Source ITU Gaze Tracker

ITU Gaze Tracker from ITUcph on Vimeo.


Earlier Videos of the ITU Gaze Tracker:
Technical Demonstration 




Seeking Sustainable Innovation

UPDATE: What's New for Kinect? Fusion, real-time 3D digitizing, design considerations, and more.

The Evolution of Microsoft Kinect

I've been following the evolution of Microsoft's Kinect, and recently discovered a few interesting videos that show how far the system has come. According to Josh Blake, the founder of the OpenKinect community and author of the Deconstructing the NUI blog,  the Kinect for Windows SDK v1.7 will be released on Monday, March 18th, from http://www.kinectforwindows.com.  More details about this version can be found on Josh's blog as well as the official Kinect for Windows blog.


It is possible to create applications for desktop systems that work with the Kinect in interesting ways, as you'll see in the following videos. I think there is potential here for use in education/edutainment!

Below is a video of Toby Sharp, of Microsoft Research, Cambridge, demonstrating Kinect Fusion.  The software allows you to use a regular Kinect camera to reconstruct the world in 3D.



KinEtre: A Novel Way to Bring Computer Animation to Life
According to information from the YouTube description, "KinÊtre is a research project from Microsoft Research Cambridge that allows novice users to scan physical objects and bring them to life in seconds by using their own bodies to animate them. This system has a multitude of potential uses for interactive storytelling, physical gaming, or more immersive communications."




The following videos are quite long, so feel free to re-visit this post when you have time to relax and take it all in!

Kinect Design Considerations
This video covers Microsoft's Human Interface Guidelines, scenarios for interaction and use, and best practices for user interactions.  It also includes a preview of the next major version of the Kinect SDK. 


Kinect for Windows Programming Deep Dive
This video discusses how to build Windows Desktop apps and experiences with the Kinect, and also previews some future work.




RELATED
Kinect for Windows Developer Downloads
Kinect for Windows Blog
Deconstructing the NUI Blog (Josh Blake)
Microsoft Kinect Learns to Read Hand Gestures, Minority Report-Style Interface Now Possible
Celia Gorman, IEEE Spectrum, 3/13/13
Kinect hand recognition due soon, supports pinch-to-zoom and mouse click gestures.
Tom Warren, The Verge, 3/6/13
Microsoft's KinEtre Animates Household Objects
Samuel K. Moore, IEEE Spectrum, 8/8/12

Kinect Fusion Lets You Build 3-D Models of Anything Celia Gorman, IEEE Spectrum, 3/6/13
Description of Kinect sessions at Build 2012
Kinect for every developer!
Tom Kerhove, Kinecting for Windows, 2/15/13
Kinect in the Classroom
Kinect Education

Note: Although I recently received my developer kit for Leap Motion, another gesture-based interface, I haven't lost interest in following news for Kinect.

Christian Bannister's Interactive Multi-touch and Gesture-based Subcycle Project. Use your hands to shape sound, create, and manipulate music. Wow!!

I came across a link to Christian Bannister's Subcycle Labs website when I was taking a look at the Creative Applications Network website.


If you have an interest in music technology and innovative gesture/multitouch applications, you'll appreciate the details that are shared on the Subcycle website.  In the meantime, take the time to watch a few of Christian's videos.  Enjoy!


Blip Shaper Walkthrough

Blip Shaper Walkthrough from christian bannister on Vimeo.
"a) creating percussive patterns with monome b) shaping the individual sounds that make up the patterns with multitouch gestures c) recording touchscreen gestures as automation d) storing, duplicating and navigation patterns e) recording the resulting audio to a dynamic buffer f) manipulating the buffer with a multitouch cut-up approach g) visualizing everything with dual screens"
Subcycle Walkthrough

Subcycle Blip Shaper from christian bannister on Vimeo.

"a) creating percussive patterns with monome b) shaping the individual sounds that make up the patterns with multitouch gestures c) recording touchscreen gestures as automation d) storing, duplicating and navigation patterns e) recording the resulting audio to a dynamic buffer f) manipulating the buffer with a multitouch cut-up approach g) visualizing everything with dual screens"


The following information describing the Blip Shaper is from Christian's Subcycle website/blog:
"For the drum sounds I have Drumaxx running for synthesized sounds and Battery running for sampled sounds.  These are running in parallel so for each voice there is a separate patch running in each VST.  The Parameters are modified with the touchscreen independently but in all cases a single touch gesture on the X-Axis will cross fade between the sampled version of the sound and the synthesized version of the sound.  I love this because I have never seen this before and I can never decide which technique I like better.  The synthesized drums are more malleable and have more interesting parameters to play with but the sampled sounds seem more substantial.  I will post a detailed list of parameters and gestures in the future."   


Christian currently uses Max for Live (ableton), and codes with Processing/Java, using Eclipse for organization.

Autopilot - Subcycle
autopilot - subcycle labs from christian bannister on Vimeo.
"sound visualization, multi-touch interface, break beat performance engine, autopilot, spacialized sound, dsp, max/MSP, fm synthesis, sonic navigation, sound storm visualization, time machine, granular, interactive sound sculpture, joystick array, more at subcycle.org"

SOMEWHAT RELATED 
Community Core Vision
NUI Group

BTW,I'm wondering if Christian or other NUI group members would like to do something with some of my ideas that incorporate interactive music. To learn more, see my post, "It must be  Voronoi: Looking for ideas for my music+art+dance+tech dream...."
...


....

Interested in the OpenNI Initiative? OpenKinect? To learn more, read Josh Blake's Interview of Tamir Berliner of PrimeSense

Interview: Tamir Berliner of PrimeSense

Josh Blake, Deconstructing the NUI, 12/9/10

Josh Blake recently interviewed Tamir Berliner, one of the founders of PrimeSense.  If you haven't heard, Microsoft's Kinect was based on work by PrimeSense, and licensed their technology. PrimeSense provides consumer electronics with natural user interaction capabilities. The good news is that the company recently released open-sourced middleware for natural interaction and depth-camera drivers. It will be interesting to see how this will play in the near future!

In the interview, Tamir discussed a number of topics related to postWIMP technologies.  He also announced the newly created  OpenNI, "an industry-led, not-for-profit organization formed to certify compatibility and interoperability of Natural Interaction (NI) devices, applications, and middleware."   It is good to see this level of support for the cause!

Here is a quote from the interview that I especially liked:

"I believe that till today the devices we’ve been using, made us learn greatly lot about them before we could use them and gain their value. I’m pretty sure everyone who is reading this has got at least 3 remotes sitting on his living room table, and at least once a week needs to help someone use their computer/media center/phone/etc. It’s time for that to change and it’s up to us, the technologists to make this revolution happen, it’s time for the devices to take the step of understanding what we want and making sure we get that, even without asking if it’s a trivial task as opening a door when we approach, closing the lights when we leave the room, even making sure we have hot water to shower with when we return from work or wake up in the morning, depends on what we normally do." -Tamir


RELATED
Here are a couple of videos from the OpenNI website that demonstrate OpenNI-compliant applications:

OpenNI-compliant real time skelton tracking by PrimeSense


OpenNI-compliant real time SceneAnalyzer by PrimeSense

FYI: 

Josh Blake is the author of the Deconstructing the NUI blog. Over the past couple of years, he's explored natural user interfaces and interactions through his work on applications designed for Microsoft Surface and Win7 with Windows Presentation Foundation.

About a month ago, Josh organized OpenKinect, an on-line community to support collaboration among people interested in exploring ways to use Kinect with PCs and other devices.  An example of this effort is the open source code, libfreenect, which includes drivers and libraries for Windows, Linux, and OS X. 

The Natural User Interface Revolution
Josh Blake, 1/5/09


Kinect for Xbox 360: The inside story of Microsft's secret 'Project Natal'  (long, but worth reading) David Rowan, Wired UK, 10/29/10


People of libreenect

OpenNI User Guide (pdf)

Update on Josh Blake, newly designated Microsoft Surface MVP

Josh Blake is the Tech Lead of the InfoStrat Advance Technology Group in DC.  He has been creating multi-touch applications Microsoft's Surface multi-user table-tops for a while. Recently, his team built a suite of applications designed for use by young children at a museum.  Below is a video demonstration of some of this work. It really looks exciting!


Microsoft Surface and Magical Object Interaction

Josh Blake's blog is called Deconstructing the NUI- for those of you new to this blog, NUI stands for Natural User Interface (also known as Natural User Interaction).  See his post, Microsoft Surface and Magical Object Interaction, for more information!

RELATED
Here is a plug for Josh Blake's book, "Multitouch on Windows"

Book Ordering Information

FYI:  InfoStrat  is hiring  WPF experts as well as Microsoft CRM and Microsoft SharePoint experts.


Microsoft Surface MVPs
Dr. Neil Roodyn
Dennis Vroegop
Rick Barraza
Joshua Blake

How a multi-touch table is made: NUI Group members Aaron Bitler and Brady Simpson on HAK.5

If you want to know more about the inner workings of DYI multi-touch, you'll be interested in the video below.

Aaron Bitler and Brady Simpson learned how to build a multi-touch table through their participation in the NUI Group, and recently formed a company, 3M8 to distribute multi-touch solutions.  The 1/2 hour video provides an in-depth tour of multi-touch technology, in an an on-line broadcast on the HAK.5 website.


"Brady Simpson explains the different multi-touch methods. Aaron Bitler gives us a tour of the x19 multitouch tables construction. Brady explains the software used to power the x19 lcdmultitouch table, including the nuigroup CCV application. Aaron wraps up by explaining to us how he got involved in multitouch. He shares with us some sites where others can get started in their own homebrew projects, as well as his own multitouch company 3M8s at LCDMultiTouch.com."

UPDATE:  New website for Brady Simpson's company: http://www.lcdmt.com/

Josh Blake's' Nice Multi-touch and Natural User Interface Applications for Surface (Cross-Post )

Information from Josh's YouTube channel:
"This is a video of some of the cool multi-touch and Natural User Interface (NUI) applications I designed and developed for Surface and Windows 7." The InfoStrat.VE map control for WPF and Surface is available for free at http://virtualearthwpf.codeplex.com.
I especially like the moving ring-menu concept, as it facilitates smoother collaboration between people on an interactive table or surface, where flexible orientation control is important.

At 3:15, the demonstration of Josh's ink-shape recognition begins. This is a feature that would be great to incorporate in my applications for children with disabilities who have some fine-motor limitations.

Josh's Blog:  Deconstructing the NUI
Josh's Recent Post about post-WIMP concepts:
Metaphors and OCGM
Josh works at InfoStrat

Want to make some multi-touch? Try PyMT- Python Multitouch. Featured in Make. (via Sharath Patali)

Sharath Patali, a member of the NUI-Group, has been working with Python Multitouch, otherwise known as PyMT, to create multi-touch applications.  He shared a link to a recent post in Make, featuring PyMT.  Sharath is the author of the UI Addict blog, and is currently doing his internship at NUITEQ (Natural User Interface Technologies).

I've been told that the beauty of PyMT is that it makes it "easy" to create multi-touch prototype applications using very few lines of code, which is great for trying out different ideas in a short period of time.  It helps if you already know Python!


PyMT - A post-WIMP Multi-Touch UI Toolkit from Thomas Hansen on Vimeo.

"PyMT is a python module for developing multi-touch enabled media rich applications. Currently the aim is to allow for quick and easy interaction design and rapid prototype development. PyMT is written in Python, based on pyglet toolkit."


PyMT Programming Guide


PyMT Website

Note: 
Christopher, author of The Space Station blog, is a member of the NUI-Group, and is building his own multi-touch table running his PyMT-based applications. Christopher is a student in Koblenz, Germany, studying computational visualistics, known as information visualization in the US.

Multi-Touch Musical Instruments- Surface Editor: Post via the NUI-Group Forum

Multi-touch Everywhere Technology in Action

Below is the video from Future Instruments, a company formed through Geneva's Conservatory of Music and the University of Applied Sciences Western Switzerland. It is worth watching the entire video clip.

The following information was posted in the NUI-Group Forum:
"The research group at Geneva’s Conservatory of Music presents the Surface Editor and Surface Tracker software applications for turning ordinary surfaces into multi-touch musical instruments. The Surface Editor allows users to custom design interfaces by dragging and dropping components, such as buttons, sliders, keyboards, and many others, onto the interface. The surface can then be used to control digital audio workstations, virtual instruments, hardware synthesizers and samplers, VJ software, and other applications, via either the MIDI or OSC protocol. The Surface Tracker is a multi-touch tracking application created for tracking movements on a surface using high speed infrared OptiTrack cameras. These cameras have on-board image processing, which allows them to process 100 frames/sec while sparing the host computer’s CPU. The applications are available at http://www.surface-editor.com."

Here are more details from the Future Instruments website:

Surface Tracker

"The Surface Tracker is an open source application for low latency multi-touch finger tracking. It was designed initially for our Multi-Touch Everywhere system (MUTE), but it should work more generally as well to track fingers which have been illuminated using the laser light plane method. It only supports low latency OptiTrack cameras; these cameras have frame rates of 100 frames per second, but due to their onboard image processing, these cameras only consume 3-5% of your computer's CPU (depending on your processor)."

"The Surface Tracker is a standalone application, which sends Open Sound Control messages to any connected client application at a user defineable IP address and port. It implements the TUIO protocol, which was specially designed for transmitting multi-touch events on a table surface."

"The Surface Tracker is currently only available on Windows. This is due to the fact that the driver for OptiTrack camera's is currently only available on Windows. We are, however, currently having discussions with the makers of the OptiTrack camera about extending support of their cameras to OS-X."

You can download the Surface Tracker from the Future Instruments download page.

I wish this system was around when I was taking a computer music technology class!

RELATED
Greg Kellum's Website: Surface Editor

A Flexible Mapping Editor for Multi-touch Musical Instruments(pdf)
Greg Kellum, Alain Crevosier (9th International Conference for New Interfaces for Musical Expression - NIMES

Sneak Peek: Musical Instruments of the Future
Eliot Van Buskirk, Gadget Lab, Wired 2/25/09

Lonely Planet Travel App by Amnesia-Razorfish for the Surface

I came across this video on the Microsoft Surface blog about a demo created for Lonely Planet by Amnesia-Razorfish, from REMIX 2009 Australia this past June:


Lonely Planet proof-of-concept at REMIX Australia 2009

Following a link from the Surface blog, I found yet another video:



Design in the Round. Creating Compelling User Experiences for Surface

The video provides an overview of the history of human computer interaction and look towards the future of NUI (natural user interface). The Surface is viewed as only as a step towards NUI, which follows a person as they go about the day, interacting with technology via a variety of devices and settings.

"Designing for multi-touch, gesture-based and tangible experience like Microsoft Surface presents a new set of challenges. How do you design for a user interface that doesn't have a top? How to allow for multiple simultaneous users without them getting in each other's way?..."

From the NUITEQ (Natural User Interface) Gallery, via Harry van der Veen

Kids take to multi-touch interaction naturally!

The following photos are from Harry van der Veen's Multi-touch blog. (Harry was one of the founding members of the NUI-Group, and also is the CEO of NUITEQ-Natural User Interface)

The last two pictures are of the HP TouchSmart running NUI Suite Snowflake software, developed by the Natural User Interface Europe AB (NUITEQ) for think LCD, Plasma, and FT displays.

Ideum's High Resolution Multitouch Table for Museum Exhibits, running NUI's SnowFlake Software Package.

The Don Harrington Discovery Center in Amarillo, Texas, and the Vulcan Park and Museum in Birmingham, Alabama will both have large, high resolution multi-touch table exhibits provided by Ideum.

Below is a demonstration video from Ideum:



Ideum's Portfolio Site
Ideum's multi-touch, multi-user mashup, created with NUI Suite 1.0 Snowflake, a gesture recognition multi-touch software package.
Ideum's mt2 Table Information & Specs (pdf)

Ideum's mt2 Table was tested at a hands-on museum during the development process.

Usability, Accessibility, and User Experience in a Win7 Environment

Microsoft's newest operating system, Win7 (beta), allows for multi-touch interaction and application development. From what I can tell from the news from the Consumer Electronic Show (CES 2009), touch and gesture interaction holds quite a bit of interest among consumers and application developers alike.

Hopefully people will think carefully about user experience and usability factors when developing the applications we'll be using in the not too distant future!

Because of my background as a school psychologist, I’m interested how touch/gesture applications can address accessibility and universal usability issues.

I wasn't able to locate user experience or usability information for Win7. Here are a few links from the Microsoft website that I think are worth reading:

Touch Interaction Guidelines

How to Design a Great User Experience

Designing with Windows Presentation Foundation

Accessibility

According to the research from Microsoft, ”more than half of all computer users experience difficulties or impairments related to accessibility, and are likely to benefit from the use of accessible technology. Moreover, approaching software design with the flexibility and inclusiveness that are the hallmarks of accessibility often results in overall improved usability and customer satisfaction.”

Regarding touch/gesture interaction, know that “19% (24.4 million) of computer users have a mild dexterity difficulty or impairment, and 5% (6.8 million) of computer users have a severe dexterity difficulty or impairment.”

Christian Moore, from the NUI-Group, shared this nice graphic that depicts the various fields that intersect or converge with interface design, which I think is useful when thinking about how we will design, develop, and use emerging technologies:



Concepts such as accessibility and universal usability can fit into this design!

For more information regarding multi-touch, see my previous post, "For Techies and the Tech-Curious: Multi-touch/Gesture from the NUI Group"

MULTIMEDIA, MULTITOUCH, GESTURE, AND INTERACTION RESOURCES

This list is a work in progress.

VIDEO
Jeff Han's Seminal 2006 TED Talk; Multi-Touch Interaction Experiments
Perceptive Pixel, Demo Reel Jan 2007 (via the Google Earth Blog)
My YouTube Playlist: Cool Technology, Interactive Multimedia, and More!
(100+ video clips are on this playlist)
Did You Know?
School Matters - The Games Children Play


BLOG ROLLS
My Blogs
TechPsych
The World Is My Interface
Interactive Multimedia Technology

Sample Blog Posts
Interactive multimedia for social skills, understanding feelings, relaxation, and coping strategies
Engaged Learning Revisited: Four video clips for reflection...
Interactive Multimedia for Science and Math
Interactive Literacy Applications and On-line Resources
Digital Storytelling, Multimodal Writing, Multiliteracies
Resources for All: Interactive Multimedia Technology and Universal Design for Learning
Dan Saffer's Book, Designing Gestural Interfaces
Interactive Touch-Screen Technology, Participatory Design, and "Getting It"
Visual Culture
Interactive whiteboards increase student attention, engagement, and test scores!
How to use Firefox to access resources offline
Online Switch-Accessible Games for Children on the BBC Website (includes screenshots and direct links)
Mega List of Resources and References (Scholarly list, updated in 2006-07, will be updated again soon - readers are welcome to contribute to the reference list.)
A Timeline of Teens and Technology

Blogs & More: Design, UX, Interactive Media,  Multitouch & Gestures, Interactive Displays & Whiteboards, & Kiosks
Inventing Interactive
Innovative Interactivity
Ron George:  Modern
Touch User Interface
Taha Bintahir's Blog
Multi-touch Blogs Directory
Robert Lewis Fashion Buddha
Marcin Ignac
Flash Gordon- Interactive Mediagician
Putting People First (one of my favorite blogs)
Nolan Ramseyer  Peau Productions
Signal vs. Noise
Kevin Arthur: Touch Usability
NUI Group (Open-source group for natural user interface/interaction)
Harry van der Veen's Multi-Touch Blog
Seth Sandler: AudioTouch and More
Jonathan Brill's Multi-touch Maven
Joshua Blake's Deconstructing the NUI
Adam Kinney Continuum Explorer
Multitouch Barcelona
Xavier, Roger, Pol, Dani
Matt Le Grand
Stephano Baraldi: On the Tabletop
Fighterfish (Ru Zarin)
Gorkem Cetin
Multitouch (Richard Monson-Haefel)
Fiery Ferret Blog Brigger Maxwell
Sharath Patali 
Edward Tse: Future of Digital Interaction
Rishi Bedi
Laurence Muller Multigesture.net
Struct: Multi-Touch Interface Research
Paul D'Intino Orion Multitouch
Jon Rose   Demand Evolution
Multi-Touch South Africa
RENCI Vis Group Multi-Touch Blog
Small Surfaces
Kinetic Interface
Jim Hertz Sassexperience
Jon Hull  Touch Factors
TeacherLED (On-line resources for interactive whiteboards)
Microsoft Surface Community Blog
MEX Mobile User Experience
The Web Outside
Interactive Kiosk News
KnowledgeWeave
inspireUX
Strombergs Blog
Justin Ireland    
Alpay Kasal   Litstudios
Mathieu Virbel Txzone
Chris Szadowski
Thomas Hansen
Dimitri Diakopoulos

Blogs: Information Visualization, Visual Communication, Data Visualization, Visual Thinking
Meryl.net
175+ Data and Information Visualization Examples and Resources
History of Visual Communication
Eager Eyes (Robert Kosara's in-depth blog about information visualization and visual communication)
Flowing Data
Information Aesthetics

Blogs: Games, Serious Games & Interactive Educational / MuseumTechnology
Learning Ecosystems (Daniel S. Christian)
Bill Mackenty: Games in Education
Gavin McLean: Interactive media, music tech, ed. tech.
Future-Making Serious Games: (Eliane Alhadeff)
In Touch (Educational Technology Resources)
Lost Garden (Interactive games approach to application development)
The Fischbowl (Staff development for 21st century technology and education)
Storytelling Memories
Ideum Blog
Papervision 3D Blog


Blogs: Assistive Technology, Universal Design, Accessibility
Assistive Technology Blog
IGDA Game Accessibility Blog

WEB SITES

Conferences, past and future
IEEE Tabletops and Interactive Surfaces 2009 (November 23-25, Banff, Canada)
IEEE Tabletops and Interactive Surfaces 2008
Bootcamp: Build Your Own Multi-Touch Surface
Bootcamp Multi-Touch Surface Guidebook (pdf)
IEEE Tabletops and Interactive Surfaces 2007


Tracermedia Interactive
Sentient City

City of Sound   Blog post about the Sentient City Exhibition


Websites: Design+ Interaction+Usability Guidelines
Gerd Waloszek: Interaction Design Guide for Touch Screens
Bill Buxton: Multi-itouch Systems that I Have Known and Loved
Microsoft: Touch Interaction Design Concepts, Guidelines, and Documentation
HP TouchSmart Software Developer Guidelines 1.5
The Digital Interactive Guidelines Project
Dan Saffer: Interactive Gestures Wiki
O' Danny Boy (Dan Saffer)
Stacey Scott: Tabletop Design Considerations & Interaction Theory
Don Norman
Nielsen Norman Group
Adaptive Path
Experientia
thirteen23
UserFocus
Usability Professionals' Association
IXDA: Interaction Design Association
SIGGRAPH Interaction Tomorrow
Thinktiv "Business Visualized"
Frog Design
IDEO

Websites: Interactive Displays & Whiteboards, MultiTouch, and Gesture / Movement Interaction
Reaction Faction: New Media Artists Interaction Design
Natural User Interface: Multi-touch Solutions
Reactable Systems
Microsoft Multipoint
nTrig Hands-on Computing
lm3labs
NextWindow
TouchTable
Reactable Systems
SMARTTable
HP TouchSmart
Smart Technologies
Promethean
Microsoft Surface
Fingertapps
Canine Interactive
nSquared Surface Applications
Vectorform Multi-touch Surface
Interknowlogy Surface Applications
Identity Mine: UX & Surface Apps
Stimulant IO: Surface Apps
Perceptive Pixel (Jeff Han)
Desney Tan: Visualization and Interaction, Microsoft Research
Dance.Draw: Exquisite Interaction
Urban Screens
Media Facades Festival

Websites: Information Visualization, Visual Communication, Data Visualization, Visual Thinking
Visual Simulations (Interactive online activities)
Visual Culture Resources
Gapminder World
Jonathan Harris

Websites: Games, Serious Games & Interactive Educational Technology
EduSim3D: 3D Virtual Learning Worlds for the Interactive Whiteboard
Multimedia: From Wagner to Virtual Reality (Interactive website)
Media Literacy Clearinghouse
Professional Development To Go: SMARTBoard Lessons
DIGRA: Digital Games Research Association
Game Research: The art, business, and science of video games
Wiki Guidelines 1.0
CAST: "Teaching Every Student In the Digital Age". Interactive book and website.
Edutopia: Technology integration, engaged learning, project based learning, and more.
NCREL Bibliography: "Technology and Engaged Learning".
National Educational Technology Plan.
The Encyclopedia of Educational Technology
The Center for Teaching History with Technology
Music Tech Teacher Music Quizzes and Games

Websites: Assistive Technology, Universal Design, Accessibility
CAST: "Teaching Every Student In the Digital Age". Interactive book and website.
CAST: Universal Design for Learning
CITEd Research Center: Universal Design for Learning in a Digital Multimedia Environment
Learning with Computer Games and Simulations
Multimedia Instruction of Social Skills
Multi-User Virtual Environments for Education

Websites: Related/Misc
Citilab
David Merrill's Resources

INTERESTING AND USEFUL ARTICLES
DAN SAFFER'S LIST: Essential Interaction Design Essays and Articles (you can link directly to the articles from this list.)

Abraham Maslow, “A Theory of Human Motivation” (1943)
Vannevar Bush, “As We May Think” (1945)
Claude Shannon, “A Mathematical Theory of Communication” (pdf) (1948)
William Hick, “On the rate of gain of information” (1952)
Ray Hyman, “Stimulus information as a determinant of reaction time” (1953)
Paul Fitts, “The information capacity of the human motor system in controlling the amplitude of movement” (pdf) (1954)
George Miller, “The magical number seven, plus or minus two: Some limits on our capacity for processing information” (pdf) (1956)
Bill Verplank, et al, “Designing the Star User Interface” (1982)
Ben Shneiderman, “Direct Manipulation: A Step Beyond Programming Languages” (1983)
Don Norman, “Affordances and Design” (1988)
Thomas Erickson, “Working with Interface Metaphors” (pdf) (1990)
Mitch Kapor, “A Software Design Manifesto” (1990)
Mark Weiser, “The Computer for the 21st Century” (1991)
Jeff Raskin, “Intuitive Equals Familiar” (1994)
Mark Weiser and John Seely Brown, “Designing Calm Technology” (1995)
Lucy Suchman, “Making Work Visible” (pdf) (1995)
Philip Johnson-Laird, “Mental Models, Deductive Reasoning, and The Brain” (pdf) (1995)
Hugh Dubberly, “Managing Complex Design Projects” (pdf) (1995)
Lauralee Alben, “At the Heart of Interaction Design” (pdf) (1996)
Don Gentner and Jakob Nielsen, “The Anti-Mac User Interface” (1996)
Bonnie Nardi, “Activity Theory and Human-Computer Interaction” (pdf) (1996)
Jared Spool, “Bridging Conceptual Gaps” (1996)
Brian Eno, “The Revenge of the Intuitive” (1999)
Shawn Barnett, “Jeff Hawkins: The man who almost single-handedly revived the handheld computer industry” (2000)
Chris Pacione, “Making Meaning” (pdf) (2000)
Richard Buchanan, “Good Design in the Digital Age” (pdf) (2000)
Robert Reimann, “So You Want to Be an Interaction Designer” (2001)
Don Norman, “Emotion and Design: Attractive Things Work Better” (2002)
Stephan Wensveen, Kees Overbeeke, and Tom Djajadiningrat, “But How, Donald, Tell Us How?: On the creation of meaning in interaction design through feedforward and inherent feedback” (pdf) (2002)
Bruce Tognazzini, “First Principles of Interaction Design” (2003)
Bill Buxton, “Performance by Design: The Role of Design in Software Product Development” (pdf) (2003)
Alan Cooper, “The Origin of Personas” (2003)
Andrei Herasimchuk, “Please Make Me Think!: Are high-tech usability priorities backwards?”(2004)
Gary Rivlin, “The Tug of the Newfangled Slot Machines” (2004)
Barry Schwartz, “The Tyranny of Choice” (pdf) (2004)
Dan Hill, “Insanely Great, Or Just Good Enough?” (2004)
Jodi Forlizzi and Katja Battarbee, “Understanding Experience in Interactive Systems” (pdf) (2004)
Paul Dourish, “What We Talk About When We Talk About Context” (pdf) (2004)
Jared Spool, “What Makes a Design Seem Intuitive?” (2005)
Anne Galloway, “Seams and scars, Or Where to look when assessing collaborative work” (pdf) (2005)
Michael Bierut, “This is My Process” (2006)
Julian Bleecker, “Why Things Matter” (2006)
Eli Blevis, “Sustainable Interaction Design: invention & disposal, renewal & reuse” (pdf) (2007)
Don Norman, “Simplicity is Highly Overrated” (2007)
James Surowiecki, “Feature Presentation” (2007)
Adam Greenfield, “On the ground running: Lessons from experience design” (2007)
Bill Buxton, “The Long Nose of Innovation” (2008)
Aaron Powers, “What Robotics Can Learn from HCI” (2008)
Don Norman, “Technology First, Needs Last” (2009)
Jonas Löwgren, “Toward an Articulation of Interaction Esthetics” (pdf) (2009)
Dan Saffer, “Controls are Choices” (2009)
BJ Fogg, “Creating Persuasive Technologies: An Eight-Step Design Process” (pdf) (2009)
Om Malik, “User Experience Matters: What Entrepreneurs Can Learn From ‘Objectified’” (2010)
Paul Seys, “11 Principles of Interaction Design Explained” (2010)

This was written by Dan Saffer. Posted on Friday, September 24, 2010, at 11:27 am.

Jacob O Wobbrock, Meredith Ringel Morris, Andrew D. Wilson User-Defined Gestures for Surface Computing CHI 2009, April 4–9, 2009, Boston, Massachusetts, USA. 

Wayne Westerman, John G. Elias, Alan Hedge   Multi-touch: A New Tactile 2-D Gesture Interface for Human-Computer Interaction    Proceedings of the Human Factors and Ergonomics Society 45th Annual Meeting - 2001

Bill Buxton's List of Input Sources for HCI

Bill Gerba's Summer Reading List for Digital Signage and Kiosk Teams

Roy Want: Carry Small, Live Large

Roy Want: Mobile and Ubiquitous Computing

Touch Panel Newsletter from Veritas et Visus

Hands-on Computing: How Multi-Touch Screens Could Change the Way We Interact with Computers and Each Other (Scientific American)

7 Things you should know about Digital Storytelling (Educause pdf)

Marc Prensky: "Listen to the Natives". ASCD article.

Oblinger & Oblinger: "Educating the Net Generation"

Oblinger: "The Next Generation of Educational Engagement."

Reeves: "The Impact of Multimedia Technology in Schools."

Reg Davis: "The Use of Interactive Multimedia in the Promotion of Mental Health in the Schools."(pdf)

James Gee: "What would a state-of-the-art instructional videogame look like?" Sign-in required.

James Gee: "Good Video Games and Good Learning."

Kurt Squire: "Changing the Game: What Happens when Videogames Enter the Classroom? (sign-in required)

Sandford & Williamson, NESTA FutureLab Handbook: "Games and Learning."

Rizzo: A Virtual Reality for All Seasons: The Virtual Classroom.pdf"

Spatial Thinking, Education, and the Workforce

Visual Learning for the 21st Century

Can't Touch This (Article about Jeff Han and Phil Davidson and multi-touch large-screen displays)

"The ICT Impact Report: A Review of Studies of ICT Impact on Schools in Europe" (Anja Balanskat, Roger Blamire, Stella Kefala, European Schoolnet.)
Resource and Research List, "Interactive Whiteboards", from the National Clearinghouse for Educational Facilities
"Speaking of Electronic Whiteboards?" Education Weekly Large Display Research Overview from Microsoft Research
"How to Be Where Your Customer Wants To Be" (pdf) Tara Prakriya
Ramblings on having privacy in public spaces "Lakshmi"
"The Trouble With Computers" Technology Quarterly

Historic/Seminal Must-Reads
As We May Think Vannevar Bush (Atlantic Monthly, July 1945)
The Computer for the 21st Century (Mark Weiser, Scientific American, 1991)
Designing Calm Technology (Mark Weiser and John Seely Brown, 12/21/95)
(Online memorial for Mark Weiser, with links to his work.)

Other Oldies but Goodies:
Computer Science Challenges for the Next 10 Years (Mark Weiser, 1996)
The World Is Not A Desktop (Mark Weiser, 1993)
Learning, Working & Playing in the Digital Age (John Seely Brown, 2000)
Fresh Flash: New Design Ideas with Flash MX (Friends of Ed)

BOOKS

Books: Interactive Multimedia, Visual/Multimedia Learning, Games, Virtual Worlds, & VR 
Digital Storytelling in the Classroom: New Media Pathways to Literacy, Learning, and Creativity. Jason Ohler

What Video Games Have to Teach Us About Learning and Literacy (James Paul Gee) Convergence Culture: Where Old and New Media Collide - Revised (Henry Jenkins) 
Learning to Think Spatially: GIS as a support System in the K-12 Curriculum The Cambridge Guide to Multimedia Learning (Richard E. Mayer, Editor)
Multimedia: From Wagner to Virtual Reality (Randall Packer & Ken Jordan)
Virtual Reality Technology (Grigore C. Burdea & Philippe Coiffet)
Thinking Like Einstein: Returning To Our Visual Roots With The Emerging Revolution in Computer Information Visualization (Thomas G. West)
In the Mind's Eye: Visual Thinkers, Gifted People With Dyslexia and Other Learning Difficulties, Computer Images and the Ironies of Creativity (Thomas G. West)

Books: User Experience, Interaction Design, Etc.

Dr. Jan Borcher's (Annotated) Top Ten List of Books on Human-Computer Interaction:

Note: I have read many of these books and I agree that they should be required reading anyone who plans to design, develop, program, or implement anything related to technology and people.

1. Alan Dix, Janet Finlay, Gregory D. Abowd, and Russell Beale: Human-Computer Interaction, 3rd ed., Prentice Hall, 2004. Currently the best, most well-rounded book I know to teach introductory HCI if you need to limit yourself to a single title. Technical enough, good breadth, not too fuzzy for a CS curriculum, very current, with a web site that includes resources such as sample programs, slides, etc.
   
   
2. Ben Shneiderman and Catherine Plaisant: Designing The User Interface, 4th ed., Pearson Addison-Wesley, 2004. Best overall reference book for all areas of HCI, providing an introduction and great up-to-date pointers to most sub-fields of HCI research and practice, especially different interaction techniques. His Golden Rules of User Interface Design and sample questionnaires for user testing are very useful in an introductory class. Unfortunately, the companion web site costs money after an initial trial period.
   
   
3. Donald A. Norman, The Design Of Everyday Things, Basic Books, 2002. A classic text from 1988 with an updated introduction that, while some of the technologies described or envisioned seem somewhat outdated now, still provides the best introduction to the spirit of good human-centered design. A not too technical read with hilarious stories of badly designed everyday technology, it provides some very useful basic models for human cognition, such as the Seven Stages of Action. This book also introduced the fundamental concept of affordances to HCI. Changed my view of the world of technology around me, and is probably the best initial brainwash for engineering students to "get" user-centered design.
   
   
4. Jenny Preece, Yvonne Rogers, and Helen Sharp: Interaction Design, 2nd ed., Wiley, 2007. This title focuses more on the process of designing good user interfaces, and is less technical, but excellent and up-to-date in the area it addresses. The companion web site has slides, case studies, and other materials.
   
   
5. Bill Moggridge, Designing Interactions, MIT Press, 2008. A truly beautiful "coffee-table style" book on interaction design, also covering product and industrial design of digital technology (Moggridge is a founder of IDEO). It has wonderful short essays about seminal digial product designs, from Engelbart's mouse, to the Mac and Palm, to Google and other internet services, as well as articles on digital product design theory. My own Sweet Sports and Baroque Technology article was based on one of the theory articles. Special treat: video interviews and chapters are available for free, on a weekly rotation, at http://www.designinginteractions.com/.
   
   
6. Bill Buxton, Sketching User Experiences, Elsevier, 2007. Similar to Moggridge's book in style, this book focuses on the early stages of product design. It also includes very interesting stories of key interactive products, such as Apple's iPod. And of course it's written by one of the long-time key players in HCI. More at http://www.billbuxton.com/.
   
   
7. Terry Winograd (ed.): Bringing Design to Software, Addison-Wesley, 1996. An excellent and very well edited collection of contributions from key players in HCI, from Kapor's Software Design Manifesto to Rheinfrank's Design Languages. Its particular value also comes from the profiles that link chapters and give an insider's view of how some of the most seminal UI designs came to be, from the Xerox Star to VisiCalc and HyperCard. Terry has some information about his book at http://hci.stanford.edu/bds/, and I used it with great success when I had the fortunate opportunity to teach an introductory HCI class in his program at Stanford in 2002.
   
   
8. Brenda Laurel (ed.): The Art of Human-Computer Interaction, Addison-Wesley, 1990. While ancient by today's standards, this book is another carefully compiled and very coherent collection of highly relevant articles on HCI by some of the most influential people in the field. I particularly like the article by Scott Kim on interdisciplinary design, and Tom Erickson's chapter.
   
   
9. Apple Computer: The Apple Software Design Guidelines, latest edition 2005. OK, I'm a Mac head, but then many HCI people are because Apple has such an excellent sense of doing the right thing when it comes to user interface design. These guidelines have been around since the 90's, with several new editions since then, and especially Part I ("Application Design Fundamentals") contains excellent, system-independent, hands-on advice for anybody developing interactive software, especially desktop applications. And it's free! Apple's developer website has the latest version both online and as downloadable PDF. I often recommend this as a quick read for engineering types that just want the bare essentials to help avoid major UI design catastrophes.
   
   
10. Jef Raskin, The Humane Interface, Addison-Wesley, 2000. Similar to Norman's book above, but more recent and more technical, this is another good first read to start thinking about user interface design, written by the father of the original Apple Macintosh. Some of the ideas presented here are quite unusual, and that's intended. Some related materials, such as demos of his Zoomable User Interface and The Humane Environment are at http://www.jefraskin.com/.

"So that's my top 10 list. I may add some more in the future. But I figure it's more important to restrict myself to those books I think are really outstanding than bother you with additional titles that don't really have that special something.


For a good current PhD-level HCI reading list that is based more on papers and individual chapters than single books, see Terry Winograd's HCI reading list at Stanford University. " - Dr. Jan Borchers


Thoughts on Interaction Design (Joe Kolko)
Acting With Technology: Activity Theory and Interaction Design (Victor Kaptelinin and Bonnie A. Nardi)
The Design of Everyday Things (Donald A. Norman)
Designing Gestural Interfaces: Touch Screens and Interactive Devices (Dan Saffer)
Designing for Interaction: Creating Smart Applications and Clever Devices (Dan Saffer)
Everyware: The dawning age of ubiquitous computing (Adam Greenfield)
Sketching User Experiences: Getting the design right and the right design (Bill Buxton)
Designing Interactions (Bill Moggridge)
Leonardo's Laptop: Human Needs and the New Computer Technologies (Ben Shneiderman)

Books: Programming and Software Design
HeadFirst C#
HeadRush Ajax


PODCASTS
SmartBoard Lessons
PRESENTATION SLIDES
My slides
Slide Share: Games for Health 2008
Slide Share: Interactive Touch-Tables and Displays for Work and Leisure
WEB 2.0
Classroom 2.0
Visualization and Education Network


PHOTOS
Technology and My World (Flickr Set)
SurfaceBlog's Photostream
Internet of Surfaces: Photo Examples of Screens of All Sizes
QUOTES
Media Literacy Definitions and Quotes
TO SORT
Multi-Touch News
Stephen Randall: The Web Outside
Links: Usability design, interfaces, multi-touch, and more

OTHER BLOG ROLLS
KnowledgeWeave's Blog Roll
Andrea Resmini - Information Architect
Asomatic (Michael Arnold Mages)
Elegant Hack (Christina Wodtke)
Experientia (Nathan Shedroff)
Findability.org (Peter Morville)
InkBlurt (Andrew Hinton)
Joe Lamantia
Mauvy Russet (Richard Dalton)
Mike Madaio
Technology-Supported Human-World Interaction
Thinking and Making (Austin Govella)
UX Crank (Dan Willis)
World Wide Intertubes (John Ferrara)

TO SORT

Touch User Interface's Links!

Touch User Interface Overview

Due to the success of iPhone, touch user interface has been a hot issue of industrial and academic fields. Many people are newly joining this interesting topic, but it's quite difficult for them to find overview/tutorial resources for starters. Exactly speaking, those kinds of information exist but scattered over the Internet.

So, in this post, I collect and list resources that are helpful to gain overview of touch user interface.

1. Case history: Touching the future: Let’s warm up first. This magazine article briefs the past, present, and future of touch screen user interfaces.
2. Bill Buxton: I recommend his three articles before your touch UI study.
   
   
   
   
   
   
   • Human Input to Computer Systems: Theories, Techniques and Technology
   • A Directory of Sources for Input Technologies
   • Multi-Touch Systems that I Have Known and Loved
   
   
   
   
   
3. See also Ken Hinckley’s Input Technologies and Techniques
4. Wikipedia article about touch user interface: definitions are always important. Also, it is worth while to read articles on haptics, organic user interfaces, and tangible user interface that are closely related interaction methods of touch UI.
5. A New Era for High Precision Touchscreens in "Advances in human-computer interaction"Advances in Human-Computer Interaction”, but contains fundamentals of touch screen interfaces - from hardware to software and user interface. : This is a single chapter of very old book “
6. Wikipedia article on touch screens and Technology Review: Point of Sale and Self Service Kiosk Touch Screens: Touch screens are main hardware systems of touch user interfaces. Those two articles summarize various touch screen technologies.
7. A Survey of Mainstream and Emerging Touch Technologies : Geoff Walker is a touch screen expert who is currently a product marketing manager at NextWindow. This presentation illustrates almost all conventional and new touch screen technologies around the world.
8. Designing Gestural Interfaces: Touchscreens and Interactive Devices : The book deals with gestural interaction that encompass touch user interface and 3D gesture, and bare-hand interaction. It contains many real life and conceptual examples of gestural user interface. I bought it and love it.

* This post is partially motivated by an email question from a Ph. D. candidate student who wanted to have survey articles about multi touch technologies. - TouchUser Interface

People who post on the Putting People First blog:

People regularly featured on this blog

In alphabetical order:

A
Marko Ahtisaari
Ken Anderson

B
Nik Baerten
Genevieve Bell
Chris Bernard
Tim Berners-Lee
Ralf Beuker
Niti Bhan
Nina Boesch
Stefana Broadbent
Tyler Brûlé
Bill Buxton

C
Jan Chipchase
Hilary Cottam
Mihaly Csikszentmihalyi
Alistair Curtis

D
Uday Dandavate
Liz Danzico
Regine Debatty
Paul Dourish

E
Jyri Engeström
Richard Eisermann

G
Jesse James Garrett
Fabien Girardin
Bruno Giussani
Adam Greenfield

H
Laurent Haug

I
Mizuko Ito

J
Bob Jacobson
Matt Jones

K
Jonathan Kestenbaum
Anne Kirah
Dirk Knemeyer
Jon Kolko
Mike Kuniavsky

L
Loïc Lemeur
Victor Lombardi

M
Nico Macdonald
John Maeda
Ranjit Makkuni
Ezio Manzini
Roger Martin
Stefano Marzano
Simona Maschi
Bruce Mau
Grant McCracken
Jess McMullin
Peter Merholz
Crysta Metcalf
Bill Moggridge
Peter Morville
Ulla-Maaria Mutanen

N
Jakob Nielsen
Donald Norman
Nicolas Nova
Bruce Nussbaum

P
Steve Portigal

R
Carlo Ratti
Howard Rheingold
Louis Rosenfeld
Stephen Rustow

S
Dan Saffer
Nathan Shedroff
Jared Spool
Yaniv Steiner
Bruce Sterling

T
John Thackara

V
Marco van Hout
Rob van Kranenburg
Mark Vanderbeeken
Joannes Vandermeulen
Jeffrey Veen
Timo Veikkola
Michele Visciola
Eric von Hippel

W
Luke Wroblewski

Z
Paola Zini
Jan-Christoph Zoels