Interaction Design 2009: Video of Dan Saffer's Presentation

Dan Saffer - Attention Awareness for Interaction Designers 2009 from Interaction Design Association on Vimeo.

Dan Saffer is an interaction designer who recently spoke at the 2009 IXDA conference. His company is Kicker Studio.

FYI: Gestural Entertainment Center


RELATED:
What is Interaction Design?

Pervasive Checkers on Microsoft's Surface: The Gamepack Video

In early 2007, before we knew of the existence of Microsoft's multi-touch surface, I worked on a "Pervasive Checkers" project with Johnny Hopkins, a classmate in my Ubiquitous Computing class. I'd previously worked with XNA Game Studio Express in an AI for Games class, and thought that it would be cool to make a casual checkers game on a multi-touch table that could be played in gathering spots such as coffee houses and neighborhood cafes.

Below is a screen shot of what we created using Inspiration software - (in the application, you can click on an item and it expands to reveal additional information.)

Two years later, and the Pervasive Checkers idea is reality - but I wasn't involved in the process. Checkers is one of the games that is included in a game-pack created specifically for the Surface.

Take a look at the video:



From Surface Computer News:

"The Microsoft Surface Games Pack is a clear illustration of where the Natural User Interface of Windows 7 has the potential to take games. Windows is traditionally the number 1 gaming platform around the world. With the introduction of the NUI, allowing players to literally have titles at their fingertips via touch, Windows 7 can kick the gaming experience up a notch. Provided that developers rise up to the challenge."

Interactive Displays in Public Spaces

Daniel Michelis recently completed his Ph.D. dissertation on a topic that is dear to my heart. Information about his research can be found on his Interactive Displays in Public Spaces blog.

(Note: This was cross posted on the Technology-Supported Human World Interaction blog.)

Here are links to a few of his posts:

Interactive Displays: Perception, Awareness, and Interaction

Evaluating Interaction with Display Applications in Public Space

I especially like the diagrams Daniel uses to depict zones of interaction:

Figure 3: Four-phase Model
(Source: Daniel Michelis (2009), according to: Vogel and Balakrishnan, 2004)

(Author: Daniel Michelis, Institute for Media and Communications Management, University of St. Gallen)

Figure 1: Perception and Usage of Interactive Displays
(Source: Daniel Michelis (2009), according to: Brignull & Rogers, 2003)

Joel Eden's Informative Post: Designing for Multi-Touch, Multi-User and Gesture-Based Systems

Joel Eden is a User Experience Consultant at Infragistics- he recently wrote a detailed article/post in the Architecture & Design section of Dr. Dobbs Portal, "Designing for Multi-Touch, Multi-User and Gesture Based Systems". I thought I'd share the link, since I've been writing on the same topic.

In his article, Joel explains the differences between traditional WIMP (Window, Icon, Menue, Pointer) interaction and gesture, multi-touch, and multi-user systems. These systems are also known as Natural User Interfaces, or NUI. He recommends that "rather than trying to come up with new complicated ways to interact with digital objects, your first goal should be to try to leverage how people already interact with objects and each other when designing gesture based systems."

Joel goes on to outline UX (User Experience, IxD (Interaction Design), and HCI (Human-Computer Interaction) concepts that designers should consider when developing new systems, - Affordances, Engagement, Feedback, and "Don't Make Us Think", which he summarizes in the conclusion of his article.

I especially liked Joel's references:

Clark, Andy. Supersizing the Mind: Embodiment, Action, and Cognitive Extension

Few, Stephen. Information Dashboard Design: The Effective Visual Communication of Data

Gibson, John J. The Ecological Approach to Visual Perception

Krug, Steve. Don't Make Me Think: A Common Sense Approach to Web Usability, Second Edition

Norman, Don. The Design of Everyday Things

Norman, Don. Things That Make Us Smart: Defending Human Attributes In The Age Of The Machine

I would also add the following references:
Bill Buxton
Multi-touch Systems I have Known and Loved
(Regularly updated!)
Sketching User Experiences: Getting the Design Right and the Right Design

"Our lack of attention to place, time, function, and human considerations means these fancy new technologies fail to deliver their real potential to real people." - Bill Buxton

Dan Saffer
Designing for Interaction: Creating Smart Applications and Clever Devices
Designing Gestural Interfaces
SAP
Touchscreen Usability in Short
(Summary by Gerd Waloszek of the SAP Design Guild)
SAP Design Guild Resources (User-Centered Design, User Experience, Usability, UI Guidelines, Visual Design, Accessibility)
Kevin Arthur (Synaptics)
Touch Usability
Bruce "Tog" Tognazzini
Ask Tog: Interaction Design Solutions for the Real World
Inclusive Design, Part I
First Principles of Interaction Design
John M. Carroll
Human Computer Interaction (HCI) (History of HCI)
Bill Moggridge
Designing Interactions
Ben Shneiderman
Leonardo's Laptop: Human Needs and the New Computing Technologies
Edward Tufte
Visual Explanations
Beautiful Evidence
The Visual Display of Quantitative Information
Envisioning Information
Rudolf Arnheim (Gestalt)
Art and Visual Perception: A Psychology of the Creative Eye

Update: A great reading list on general HCI. Some of the authors were involved in the early days of touch, bi-manual, and multi-touch interaction.

Jan's Top Ten List of Books on Human-Computer Interaction


FYI: If you know much about Windows Presentation Foundation, you probably know that Josh Smith, WPF guru, also works at Infragistics

Tangible Embedded Interaction '09 Conference - Wish I could go!

I really wish I could attend the Tangible and Embedded Interaction '09 conference in the UK! All of the sessions look intriguing:

Session One: "Embedded Artifacts, Garments, and Environments"
Session Two: "New Perspectives and Theories on Tangibility"
Session Three: "Tangible and Embedded Interaction - in the Lab and in the Wild"
Session Four: "Tabletop Tangibles and Augmented Surfaces"
Session Five: "Enabling Technologies and Design Techniques"
Demo and Poster Session 1
Demo and Poster Session 2

Keynote Speakers
Tom Igoe (Physical Computing, NYU) (I'd love to take Igoe's Physical Computing class.)

Tom Igoe on Physical Computing at Ignite NYC


Durrel Bishop (Luckybite)

Durrell Bishop's interview in Bill Moggridge's book, Designing Interactions, Chapter 8: Multisensory and Multimedia

Invited Talk
Ayah Bdeir
Bdeir's littleBits Intro:

littleBits intro from ayah bdeir on Vimeo.

Tangible Embedded Interaction '09 Program Guide (pdf)

Interaction & User Experience Design and Adaptive Path's UX 2008

For more information, see my recent post & related video clipson the Tech Supported Human-World Interaction blog:

Internet of People, Things, Surfaces, and Interactions: Insights from UX 2008 & Adaptive Path

Touch Screen Interaction in Public Spaces: Room for Improvement, if "Every Surface is to be a Computer"

Via Technology Supported Human-World Interaction

With Win7's "multi-touch" capabilities, Bill Gate's statement that every surface will be a computer might come true, but from what I can see, we have a long way to go before we get it right!

One of the concerns I have about "touch" screen interaction in public spaces is that not much thought is given to the environmental factors surrounding the system. The system that is tested for the first time after installation might work just fine, but not when it is put to use under a variety of conditions or situations.

EXAMPLES (I've recycled a few of these from previous posts.)

Touch Screen Kiosk/Display in Upscale Shopping Area, St. Thomas(?)

I came across this system when I was walking around in St. Thomas with my husband. I wasn't sure what to make of it. There wasn't anything on the large display, and I couldn't tell from a short distance if the two smaller displays were active. I noticed that the display had an awning of sorts, so the fact that the display would be used in a sunny location had been considered. Unfortunately, the awning didn't solve this problem, as you can see from the pictures below:

CLOSER UP

As we approached the display, it was almost impossible to see what was on the screen. In fact, we weren't sure what the display was all about. It looked like something you'd see at an airport check-in, not in a lush tropical setting.

Below is a shot of my husband, trying to see what is on the screen. This photo shows exactly what the screen looked like at the time:

The next picture tells it all. You can see that there is some sort of alphabet on the right side of the screen, but the reflection of my husband is easier to see. We couldn't figure this one out.

The following clip demonstrates how environmental concerns, such as wind and "sideways" rain, can make an interactive display at a shopping center difficult to use. During this clip, you can hear me muttering about the difficulties I had interacting with the information on the touch-screen display. It is possible that the system wasn't working well because of the exposure to the elements.

WIND AND SIDEWAYS RAIN

You can see that the navigation system was confusing. At the end, the entire screen turned red, and I gave up.

The next video-clip is of a touch-screen map at the Cleveland Clinic hospital. I was with my elderly aunt and my sister, and we were trying to figure out how to get to a restaurant. We never found it.

Although the screen was easy to see, the application was... crazy!

In my opinion, an interactive wayfinding application should run on a larger display and support two or more people, since many hospital visitors often run in pairs or small groups.

Windows Vista User Experience Guidelines: All Windows programs should be touchable!

If you are a designer/developer who would like to develop applications that can support touch interaction, it will save you some time to study the Windows Vista User Experience "Touch" guidelines. You'll find information about the eight flick events, (gestures that correspond to keyboard shortcuts for navigation and editing), the 40 or so gestures recognized by Vista, tips for creating controls, GetMessageExtraInfo function, and more.


"As touch spreads from Tablet PCs to other types of computers, software program developers and designers will find it increasingly important to support touch as well. All Windows programs should have a great touch experience. Users should be able to perform your program's most important tasks efficiently using a finger."

Catchyoo's Interactive Touch-screen on the Panasonic Road Show

Lm3Labs/Catchyoo's application was featured on a giant Panasonic interactive touch-screen display. The ripple effect looked intriguing:


Cathyoo on Panasonic's national roadshow from Nicolas Loeillot on Vimeo.

Lm3Labs focuses on interactive applications that harness the power of computer vision. Although much of what I've seen has been retail and market-related, they are also involved with innovative interactive museum displays:
Lm3Labs: Interactive Museum Solutions featuring ubiq'window and Catchyoo

Wouldn't this sort of interactivity be fun in our schools? How about at the public library? It is a leap or two ahead of the interactive whiteboards.

Natural User Interface: Overview of multi-touch technology and application development by Harry van der Veen,- Business to Buttons

Harry van der Veen from Natural User Interface Europe AB, was one of the keynote speakers at the Business to Buttons: Designing for Effect conference, held in June 2008. In this presentation video, Harry discusses the past, present, and future of multi-touch technology, and reviews the importance of multi-touch over single touch displays. He also provides a good overview of gesture interaction, something that he researched when he was a student. This presentation includes several video examples of multi-touch applications in action.

The presentation is well worth the 30-minute view!

"Harry van der Veen is a Bachelor of Multimedia, derived from the Dutch education Communication, Multimedia and Design, focused on Interaction Design and Project Management. He is CEO, co-founder and co-owner of the Sweden based commercial company Natural User Interface Europe AB, which focuses on delivering standardized and customized multi-touch hardware / software solutions and services to the global market. In addition to that, he co-founded the NUIGroup community, which is the worlds largest online platform where a global network of people share their ideas and information in an open source community, focused on multi-touch hardware and software solutions."


NUIGroup Community


Harry van der Veen's blog

Natural User Interface Europe AB (Harry van der Veen's company)

NUIGroup Wiki: This wiki includes tutorials for developing multi-touch applications, building your own low-cost multi-touch table, and information about current projects that are in progress.

Related Information:

The Business to Buttons: Designing for Effect conference was held on June 12-12 in Malmo, Sweden, organized by Malmo University and inUse, a user experience consultancy. Partners in this conference included Adaptive Path, a product experience strategy and design company, Patrick W. Jordan, a design, marketing, and brand strategist, the cocktail, a user experience and interaction design studio, cooper, a product design company, and OresundIT, a non-profit network.

Don Norman, the author of books such as "Design of Everyday Things" and "The Design of Future Things", presented at this conference. Don Norman is one of the founding fathers of the Human-Computer Interaction and related fields, and is the co-founder of the Nielsen Norman Group, a consultant firm that helps company create human-centered products.

Videos of Don Norman's Presentations:
Emotional Design: Total User Experience
Cautious Cars and Cantankerous Kitchens

Other:
Business to Buttons 2008 Recorded Sessions
Business to Buttons 2008 Downloads

My posts about the work of NUI Group members:

Multi-Touch Plug-in for NASA World Wind?!

More Multitouch: NUI Group's Christopher Jette's multi-touch work featured in Engaget ; Croquet?

More Multi-Touch from members of the NUI group!

Multi-touch Crayon Physics from multitouch-barcelona, inspired by Crayon Physics by Kloonig Games

Cross Post: Seth Sandler's YouTube Video, "How to Make a Cheap Multi-touch Pad" goes viral

NUI-Group Member Bridger Maxwell Receives High School Science Fair Award for Multi-Touch Screen Project

Look, touch, listen, and play: Seth Sandler's interactive Audio Touch Table video; NUI Group and Google's Summer of Code

Link to post: Reflecting about Interaction-Updated

If you are interested in interaction and technology, I've updated my "Reflecting about Interaction" posts on my TSHWI (Technology-Supported Human-world Interaction) blog. I've embedded a couple of video clips that show how body movements interact with music applications. One is an interactive music wall, and the other is a staircase that senses people's movements and plays music and sounds as they move.

You might also be interested in my latest post about Johnny Chung Lee and his Wii hacks. "I wish I could be Johnny Chung Lee for a Day!"

Touch Screen Interaction and Usability in Public Spaces: Link

If you are interested in what is happening with interactive displays in public spaces, you can find a description of the technology used at the Ballantyne Village center in Charlotte, N.C., along with video clips and my critique, on the Technology Supported Human-World Interaction blog.

Its a great concept, but as you can see from the unedited video clips of my frustrating interactions with large touch screen displays, there is much more work to be done in this arena.

Interaction Tomorrow: SIGGRAPH Course Materials On-line

If you are interested in interaction design for the future, take a look at the course notes from Interaction Tomorrow, presented at SIGGRAPH 2007.

Here is the course abstract:

"This course provides a comprehensive overview to user interface technologies on the newly emerging interactive tabletops and large wall displays. The course will cover input devices, interface metaphors, modality of interaction, sensing technologies, applications, and future directions. Materials will be drawn from both commercial systems and research prototypes." The Interaction Tomorrow is a document that contains 172 pages of pictures and text.

The course was organized by Michael Haller, from the Upper Austria University of Applied Sciences, and Chia Shen, from the Mitsubishi Electric Research Laboratories. The lecturers included Gerald Morrison, from Smart Technologies, Bruce H. Thomas, from the University of Southern Australia, and Andy Wilson from Microsoft Research.

Bios

The course website http://www.interactiontomorrow.org/

This was cross-posted on the Technology Supported Human-World Interaction blog.

More touch-screen videos...

Here is an interesting video about TouchTable's $59,000.00 system:



ESRI's ArcGIS running on a TouchTable:



The application supports collaboration. People can interact with one another around a single table, with others at remote tables, and people using laptops and mobile devices out in the field.

Link: ESRI's GIS for K12 Schools

About: Programming for interactive multimedia applications- WPF, Silverlight, EduSim, NeuroVR..

Part I Musings: Learning about application development and programming at mid-life:

I thought I'd write on a more personal level this time.

I'm a school psychologist, so in 2003, my motivation for taking computers at mid-life stemmed from my desire to create engaging interactive multimedia games for learning, games that could be played on hand-held devices as well as on the interactive whiteboards that I noticed were inching into my schools.

It was difficult for me to figure out how to get from Point A to Point B.

I shouldn't have been shocked to learn that most introductory programming classes provide instruction, as well as endless lab assignments, that are geared for people who want to make business forms and manipulate business-related data, build e-commerce websites, or create relational databases for... banks!

I now can make a mortgage calculator forms that adjust for various scenarios and provide cute error messages, in beginning Visual Basic.Net, C#, and Java. I can create a database that will let users look up part numbers for all sorts of widgets, in all sorts of combinations, and ensure that client data can be easily accessed in a nice looking form.

Why should I learn all of the old stuff when there are so many new avenues to explore?

Over the past few years, I've been fortunate to take a variety of classes that were not readily available just 8-10 years ago:

Computer/Internet Multimedia. Computer Music Technology. Game Design/Development. AI for Games. Ubiquitous Computing. Web Development Tools. Virtual Reality for Education and Training. These courses have motivated me to learn more about programming. The traditional programming courses had the opposite effect.


PART II Keeping up

I recently attended a day-long code camp at Central Piedmont Community College to learn more about Windows Presentation Foundation (WPF) and Silverlight. WPF and Silverlight provide the foundation for applications that can run on Microsoft's Surface. WPF and Silverlight provide designers and developers with more efficient ways of developing visualization and interactive multimedia applications.

The architecture behind WPF and Silverlight seems to align more with the way people think and envision, which in my opinion, gives hope for those of us who have toyed with the idea of programming, but were frightened off the first time they opened up a traditional programming textbook.


Part III Visualization and Interactive Multimedia

For more information about WPF, Silverlight, design, etc, read Sam Batterman's (a Microsoft Evangalist) recent blog post: Some thoughts about WPF and Data Visualization

Here are some pictures and text that I lifted from Sam's blog that will give you a picture of what I'm talking about:

"Here's an application that one of our partners built in a few weeks - all WPF and actually, not complicated code...getting that heart rendered was probably less than 100 lines of code. This app is used in a hospital for documenting heart surgery procedures. You can draw and annotate the 3D surface, rotate the heart, etc."





Can you imagine how kids would love to manipulate something like this in a science class?! This would be great on an interactive whiteboard or display.

This focuses on user experience, much more so than applications in the past. For more information about designing for user experience, take a look at the link to Bill Buxton's book, "Sketching User Experiences."

Bill Buxton's webiste, "Multi-touch systems that I have known and loved" is a good resource for those of you who'd like to get a better picture of multi-touch systems and interaction.

(I'll write more about WPF and Silverlight after I get my laptop repaired and have a chance to experiment some more.)

Croquet and EduSim

Right now, I've been experimenting with Croquet, which uses Squeak, and EduSim, which is powered by Croquet, to put together some learning activities for students.The best part is that Croquet is open-source, and all of the EduSim applications are free.

Here are some pictures that link to short video clips from the Greenbush EduSim website:






Neuro VR

Another application that I'm working with is NeuroVR.

NeuroVR is a free virtual environment that was designed for use in clinical settings. If you don't have access to virtual-reality hardware, you can still use NeuroVR on a desktop or large-screen display. Available 3D environments include an office, a supermarket, a park, a classroom, a poolside setting, and a home.

Andrea Gaggioli, Ph.D., is the Chief Technical Officer of the this project. He's also behind the Positive Technology blog, which is a great resource. Andrea is a researcher working at the intersections of psychology, neuroscience, and emerging technologies.

NeuroVR allows the clinician (or educator) to easily insert pictures, objects, and videoclips into each virtual world. Doors open and close, and you can move items around in the environment. For example, fruit set on a table can be moved over to a counter.

In my opinion, NeuroVR has potential not only in the area of rehabilitation and therapy, but in special education as well, particularly for students who have multiple special needs, including severe autism. A variety of "how-to" videos are posted on DaevornLi's YouTube channel.


Here are some videos clips to give you a better picture of the application:

Interesting link to an NYU course's journal: Representation & Interaction Design

If you are searching for great information about multimedia interaction design, the Representation and Interaction Design Journal is a great resource. It was developed for students taking a course of the same name at NYU.

Posted topics include multimedia theories about information design, definitions, defining interactivity, information design and emotions, informaton design: language and semiotics, cognition/theoretical framework, media specific interaction design, and more.

Some of the "names" include Schneiderman & Plaisant, Betrancourt, Dan Saffer, and Shedroff.

If you are interested in these topics, I've listed a few books and on-line resources that might be helpful:

The Cambridge Handbook of Multimedia Learning (2005), Richard E. Mayer, Ed.
Designing for Interaction: Creating Smart Applications and Clever Devices (2006), Dan Saffer
Digital Multimedia Perception and Design (2006), Sherry Y. Chen & Gheorghita Ghinea
Teaching Every Student in the Digital Age (online)
Edutopia (online)
Multimedia: From Wagner to Virtual Reality (online)
Book Randall Parker & Ken Jordan

Visualizing Content on the Web: Great pictures, descriptions, and links on Max Kiesler's blog

"Content Visualizations: The Next Wave of Interaction Design" is the topic of Max Kiesler's recent blog post.

Searches conducted on the Web are primarily word-based, and the same is true for searches conducted on PC's. What results is a list of phrases, which are often difficult to sort and prioritize quickly.

The increased bandwidth available on the web now allows for content that contains a range of multimedia components, such as photos and video clips. Many on-line journal articles, blog posts, and of course, information visualization websites, contain important visual-oriented information that might be missed through traditional searches.

Kiesler discusses some solutions to this problem in his post. Take a look at it soon- it contains great visualizations, as well as links to additional information and resources.

Although Keisler does not state it directly, his post makes the case for the importance visual and multimedia literacy in our society.

Note: For those of you who follow my blog, you'll know that this sort of visual approach would be a great tool in education, especially when presented on interactive large-screen displays.

About: Interaction Design (off the desktop)

Interaction design is a relatively new field that combines concepts related to human-computer interaction (HCI), mobile, pervasive, and ubiquitous computing (ubicomp), interface design, service design, user-experience design, interactive media design, and more.

According to Dan Saffer, an interaction designer at Adaptive Path, and author of Interaction Design: Creating Smart Applications and Clever Devices, interaction design is "about people: how people connect with other people through the products and services they use." If you are interested in learning more about designing for interaction, Saffer's book is a great starting point.

Saffer has recently established a wikki about interactive gestures, a site for the "dissemination of gestural interface information such as found on the iPhone and Wii." This is an important resource for those of us who are interested in developing useful interactive applications for emerging technologies.

(Related information can also be found on this blog.)

About Interaction: Bill Buxton and Multi-Touch Systems

Based on data from SiteMeter, there are a number of my readers who are searching for information about touch screen interaction design and multi-touch systems. A great resource for anyone interested in these topics is Bill Buxton's website, "Multi-Touch Systems that I have Known and Loved".

Bill provides a good overview of the various flavors of multi-touch interaction and interfaces, along with a an annotated chronology of multi-touch and related technologies, starting before 1982. The chronology includes some interesting pictures.

Bill Buxton works for Microsoft Research. The following information is from his bio:

"Bill Buxton is a designer and a researcher concerned with human aspects of technology. His work reflects a particular interest in the use of technology to support creative activities such as design, film making and music. Buxton's research specialties include technologies, techniques and theories of input to computers, technology mediated human-human collaboration, and ubiquitous computing."