Feb 26, 2011

Why bother switching from GUI to NUI? - Asked and Answered by Josh Blake; My 2-cents; Stevie B’s Microsoft Research Video; Marco Silva’s NUI-HCI Presentation (and links)

In Chapter 1 of Natural User Interfaces in .NET,  Josh Blake asks and answers a question posed by many people who have been under the spell of keyboard input and GUI/ WIMP interaction: 

Why bother switching from GUI to NUI?  The answer?  Read Chapter 1 (pdf) of the book - the chapter is free.

Here are a few of my personal reasons:  
1.  I want to buy the next version of the iPad or something like it.
2.  I want to buy a new large-screen Internet HD TV.
3.  I want to buy a Kinect.
4.  I do NOT want to interact with my new TV with a Sony remote.  Too many tiny buttons!

5. I do NOT want to interact with my new TV with a keyboard,  because it reminds me of...work.

6.  Most importantly: 

I want to design apps for the people I care about, and others with similar needs:
    My mom.  
    My grandson.
Moms and dads with kids in tow.
People with special needs and/or health concerns, and the people who care and guide them.
Knowledge sharers and (life-long) learners....


"Smart" Interactive Display Research

View more presentations from Marco Silva

My YouTube Playlist:
"Natural user interfaces, gesture interaction, multi-touch, natural interaction, post WIMP examples and more... "
RELATED - and somewhat related   
Encyclopedia:  Human Computer Interaction, Interaction Design, User Experience, Information Architecture, Usability and More (Interaction-Design.org)

Josh Blake's Blog: Deconstructing the NUI    Book: Chapter 1 (pdf)  Free!
"Blake.NUI is a collection of helpful controls, utilities, and samples useful for multi-touch and NUI development with WPF, Surface, and Silverlight."
 (This is not an inclusive list.)

GUI to NUI Post-WIMP Manifesto:  TBA

A little off-topic: Slideshow of my cell phone camera effects experiments at Amelie's French Bakery, NoDa, Charlotte, NC.

I dropped my nephew off at the airport before the crack of dawn this morning, and took the opportunity to visit Amelie's French Bakery in the NoDa neighborhood of Charlotte (NC). The croissant was awesome and the coffee hit the spot. It occurred to me that I'd never checked out the camera effects features on my cellphone.

Below is a slideshow of my experiments - there are many pictures of the same "scene", with different effects. (The flash was on for some of the pictures, and off for others.)

Feb 24, 2011

Today I hooked up a Wii to the IWB in the school's therapy room. Next - a Kinect?! (Angels are welcome to help us expand the school's "games" programs for students with special needs)

Today, I hooked up our school's Wii to the newly-installed IWB in the therapy room for the first time, and used it during a session for for a social-skills activity with two students who have autism spectrum disorders.  The students helped each other to create their own "Mii" avatars.  I had them play the bowling game, and was impressed how this activity elicited social conversation between the students, with minimal effort on my part.

Most of students in the program at Wolfe are in grades 6-12. They have cognitive, language, and motor delays.  A good number of the students also have autism spectrum disorders. Some have multiple special needs.

We have found that when our students are provided with interactive activities displayed on the large screen of an IWB, they tend to increase their level of attention to their peers and also communicate more with one-another, as well as with the teachers and staff.

By using an IWB for games at school, we are extending the reach of how this technology can be used with students who have more complex special needs.   By providing a means  for our students to learn to play positive games, we help them develop important physical, social, and leisure skills that they can use outside the school setting with non-disabled peers and siblings.  My hunch is that the games will also help promote cognitive/problem solving skills, too.

The funds to purchase the Wii were donated to our school, but did not cover additional games or accessories. We'd like to purchase Rock Band and Wii Fit. We'd also like to expand our program and purchase an Xbox with a Kinect.   An iPad or two would be OK, too : )

If there are any angels out there who'd like to donate funds for our "games" program at Wolfe, please contact me through my Google profile. I will connect you with the appropriate person.

Game designer/developers/researchers/students:
If you are interested in volunteering your efforts to work on a basic cooperative, pro-social game for the Kinect, SMARTTable, or interactive whiteboard, please contact me.

NCSU D.H. Hill Library has a Perceptive Pixel Multitouch Display (updated)

NCSU 'sandbox' lets students touch the technology
Jay Price, Charlotte Observer, 2/22/11

"The Perceptive Pixel display wall, which was switched on this week, is the $100,000 centerpiece of the new Technology Sandbox at the D.H. Hill Library. The sandbox is a room of cutting-edge, interactive hardware that's aimed partly at familiarizing students with the latest high-tech hardware, particularly large displays and gesture-based computing. It was paid for with a federal grant from the Institute of Museum and Library Services that is administered by the State Library of North Carolina."

Credit: Corey Lowenstein - clowenst@newsobserver.com

Technology Sandbox introduces NCSU students to latest gadgets
WRAL Techwire, 8/25/10
New Learning Spaces for New Learning Styles (pdf)
Terry B. Hill, Mohan Ramaswamy,  NCSU


North Carolina State University Libraries
Perceptive Pixel

Vision-Based Hand-Gesture Applications: Video from Communications of the ACM

The latest edition of Communications of the ACM, via "snail mail", was the inspiration for this post:

Vision-Based Hand-Gesture Applications
Juan Pablo Wachs, Mathias Kolsch, Helman Stern and Yael Edan

"Body posture and finger pointing are a natural modality for human-machine interaction, but first the system must know what it's seeing."

More to come!

Feb 23, 2011

Interactive Timeline "Wordle" for WikiLeaks: Explore the Cable Street Database, an Example of Interactive Journalism

What can a cloud of words tell us about foreign policy?:
"An interactive analysis of all the 250,000 documents in the so-called Cable Street database" -Aftenposten

Behind the scenes of Aftenposten's Wikileaks visualization
Tracy Boyer, Innovative Interactivity (II), 2/23/11

Eirick Wallem Fossan: Multimedia journalist and Flash developer for Aftenposten.

Comment:  I usually don't blog about political topics, but I felt that this interactive visualization was an effective method of helping the public understand the information/data shared through the Wikileaks process.  

My last post focused on the large multi-touch wall installed at a new library at North Carolina State University.  The library was designed for collaborative activities.  An interactive "Wordle" displayed on a multi-touch wall for students to explore with peers would probably be worth using!    

Public libraries, outfitted with collaborative displays, would also be great places for the people to explore all sorts of visualized data.  

Feb 22, 2011

How Social Can News Get? SoCon11 Presentation by Lee Rainie, Pew Internet Project

If you'd like to learn more about social media,  take a look at the following presentation from the recent SoCon11 conference, "How social can news get?", by Lee Rainie,  Director of the Pew Internet Project:

"Lee Rainie, Director of the Pew Internet Project, discusses the Project’s latest findings at the SoCon11 conference. He goes through trends in social media use in the last five years of the Project’s data. He explores how the turn to pervasive, participatory, personal, and portable news changes the way news consumers and producers behave and think about the role of news in their lives." - Pew Internet and American Life Project

Quite a few topics were covered at the conference, which can be found on the SoCon11 Agenda site.

Urban Ministries of Duram's online serious game, SPENT: Can you make it through the month on $1,000.00? (via Tracy Boyer)

SPENT is an online serious game that is a result of a collaboration between the ad agency McKinney and Urban Ministries of Durham, N.C.    The game was launched this month to help increase community awareness about the needs of people who have been negatively impacted by the on-going economic crisis - people who have lost jobs, the under-employed, and the homeless.   

The factoids provided in the game are based on data (see link below).  One goal for presenting this information is to help players sift through the myths they might be carrying around in their heads about poverty and the realties that many people face every day.   

The game offers players the opportunity to take on different roles, make tough decisions, and see the results of these decisions. 

The game is a call to action.

McKinney launches Spent, an interactive game about poverty
Tracy Boyer, Innovative Interactivity, 2/21/11
Sources for the data/information provided in SPENT (pdf)
Urban Ministries of Durham
WhiteNoise | Lab (Sound Design for SPENT)
Emphathize Walt Barron, 5 Words (McKinney Blog)  2/8/11

Feb 21, 2011

Make It So: Interaction Design Lessons from Science Fiction - Video of MacWorld presentation by authors of upcoming book.

I'll admit it. I've been a SciFi fan and Trekkie since childhood. 

When I discovered that Nathan Shedroff and Cris Noessel were writing Make It So: Interaction Design Lessons from Science Fiction, I just had to take a break from writing reports to share the news.  

The video below is from the author's MacWorld presentation that covers some of the content of the book's first chapter:

Below is information about the book, from the publisher's website:

"Interaction and interface designers can learn practical lessons from the interfaces in Science Fiction films and television. Though lacking rigorous engagement with users, production designers are nonetheless allowed to develop influential "blue-sky" examples that are inspiring, humorous, prophetic, useful, and can be incorporated into "real" work to make online, mobile, and ubiquitous interfaces more interesting and more successful. This book will share lessons and examples culled from imaginative interfaces free from traditional constraints. In addition, the authors will outline their process of investigation and describe a toolkit for others to make similar explorations into other domains." 
book in progress by Nathan Shedroff & Chris Noessel. Publisher: Rosenfeld Media. Anticipated publication date: 2012
Nathan Shedroff is the chair of the new MBA in Design Strategy program at California College of the Arts.  Chris Noessel is an interaction design director at Cooper.

via Putting People First 

Transit: Digital Video Dance Storytelling on Large Displays at the LAX Airport, by Scott Snibbe and Francesca Penzani

Scott Sona Snibbe, an interactive artist, created a large-scale video installation on twenty-nine connected HD displays that curved around at the Tom Bradley International Terminal at the LAX airport in 2010.   The video tells a story of groups of people traveling through an airport who eventually begin dancing in interesting ways.  

Francesca Penzani was the choreographer for this project. Noah Cunningham was responsible for the cinematography, editing, and post-production.

The Snibbe Interactive website has lots of great photos and videos of the company's work.

Thanks to Daniel Chen for sharing this video!

Feb 20, 2011

Human-Computer Interaction (HCI) Is Changing the World: BLUR Conference, February 22-23, Omni Orlando Resort (Includes video)

Tuesday, February 22, 2011 at 8:00 AM - Wednesday, February 23, 2011 at 5:00 PM (ET) Omni Orlando Resort, 1500 Masters Blvd. 
 ChampionsGate, Florida 33896 Phone: (407) 390-6664
Blur Conference

(from the conference website)

"It’s easy to forget that the computer mouse is over 45 years old."

"What’s not as easy to forget is that we’re now collectively getting used to interacting with computers via means and interfaces that have moved way beyond the keyboard and the mouse — the iPhone and Wii being the most prominent examples."

"The truth is that we stand on the verge of a major revolution in the models of Human Computer Interaction (HCI). A revolution that will fly right past academic and into a world of retail, medical, gaming, military, public event, sporting, personal and marketing applications."

"From multi-touch to motion capture to spatial operating environments, over the next 10 years, everything we know about HCI will change."

"Blur is the only conference that is exploring the line of interaction between computers and humans in a substantive, real-world and hands-on way."

"At Blur, vendors, strategists, buyers and visionaries assemble to not only discuss the larger issues of HCI, but also to lay their hands on the latest in HCI technology. Blur is the only forum for a focused, hands-on exploration of the varied technologies evolving in the HCI."

"Come play, investigate, learn and apply at Blur — where we’re changing how you interact with computers forever." -Blur

BLUR Conference Agenda
(Note:   I added the links to conference participants and/or their organizations. Feel free to leave a comment if you know of any corrections or better links!)

Neuroergonomics: How an Understanding of the Brain is Changing the Practice of Human Factors Engineering - Dr. Kay Stanney, Design Interactive
When Computers Feel: Understanding Human Emotional Measurement  - Hans Lee, EmSense
A Quick Hit on Mobility and HCI - Juan Pons, Swype
Panel Discussion: Haptics- The Beginnings and Future of Touch  - Nimish Mehta
Why HCI will lead the biggest tech revolution yet - Andrew Tschesnok, Organic Motion
Location as a Primary Interface Input - Matt Galligan, SimpleGeo; Nick Brachet, Skyhook Wireless
Robotics, Gaming and The Future of Entertainment- Paul Berberian, Orbotix
Virtual Coaches in Healthcare: A Vision of the Future - Dan Siewiorek, Carnegie Mellon University
10 reasons to be happy about giving computers emotion sensing - Dr. Rosalind Picard, MIT
Commercializing HCI Technology - Dr. Paul Kedrosky, Ewing Marion Kauffman Foundation and Dr. Gerry Barnett

Breakout Sessions:
Human Instrumentation - James Park, FitBit; Ben Rubin, Zeo; Jason Jacobs, RunKeeper;
Steve Larsen, moderator

New Museum Experiences: Learning from Multitouch and Multiuser Installations - Jim Spadacinni, Ideum
Kinect Hacks - Jonathan C. Hall; Lonergan Harrington; Jim Spadacinni, Sean Kean, moderator
Interactive Ads and Consumer Experiences - Alessio Signorini, Immersive Labs; Jon Fox, Helios Interactive
Augmented Reality - Ready for Primetime? - Vikas Reddy, Occipital; Carlin Getliffe, OmniarEdwin Rivera, Credelis; Dan Rua, moderator
Building an Interface for Endangered Language Learners - Finn Thye and Kelson Adams, Univ of Colorado - Boulder
Alternative Interface Inputs - Gary Clayton, Nuance; Nick Langdale-Smith, Seeing Machines; RJ Auburn, Voxeo; Steve Larsen, moderator
Ewing Marion Kauffman Foundation "Idea Hack" - Commercializing HCI Technology: A Discussion - led by Paul Kedrosky
Building Natural User Interfaces - Thomas Peterson, SoftKinetic; Ohad Shvueli, Prime Sense; David Minnen, Oblong
3D Interactive Design for the Human Body - Albert Hwang
Panel: Will the Kinect Change the HCI Industry Forever? A Group Discussion
HCI in the 21st Century:  Technologies for Extending and Amplifying the Human Experience (pdf) - Dr. Charlie Hughes, UCF; Dr. David Pratt; Dr. Joseph LaViola;  moderated by Steve Fiore, UCF

Some Videos of HCI/Tech featured at Blur 2011 
360 Panorama occipitalhq

"Illuminous" Eric Gradman

"Standard Gravity" Eric Gradman, OpenKinect (libfreenect/python)

Organic Motion Markerless Motion Capture

Steve Fiore, University of Central Florida
Bob Allen, Disney R&D
Kay Stanney, Design Interactive
Capt. Dylan Schmorrow, USN

Feb 18, 2011

New Radiohead video, Lotus Flower: Will we see Thom Yorke's moves on Kinect's Dance Central or Just Dance 2 anytime soon?

When I saw the new Radiohead video, Lotus Flower, featuring a dancing Thom Yorke, I wondered how his moves might play out in a dance application for the Kinect or Wii.

Wouldn't it be fun to figure out a way to represent the dance "steps" in this video?  Just a thought.  

Radiohead Surprises With Early Release
Jon Pareles, New York Times 2/18/11
"Wii Just Dance 2 and Kinect Dance Central: UI and Usability Approaches"

About the Web of Things - Great Presentations & Links

"The Web of Things is a community of developers, researchers, and designers that explore the future of the physical Web. We explore how to leverage Web standards to interconnect all types of embedded devices (sensors, mobile phones, etc), in order to make them easier to use and to integrate in classic Web applications. We aim to build a future Web of devices that is truly open, flexible, and scalable, and we believe Web standards are the best way to do it."

Web of Things - Towards Open and Sharable Networks of Embedded Devices

View more presentations from vladounet.
Design of a Web-based Distributed Location-aware Infrastructure for Mobile Devices

View more presentations from vladounet
Web of Things Upcoming Event
What: 2nd International Workshop on the Web of Things (WoT 2011)
Where: Pervasive 2011 conference, San Francisco, USA
When: June 12, 2011
For who: Hackers, developers, web, and ubicomp/pervasive researchers.

Web of Things Projects

Feb 16, 2011

Supertouch Group: Use an iPhone or iPad to Send Balls into Kinect Dodgeball

See how they inter-"kinect"!

"The SuperTouch group mission is to generate wonder through simple, fun, entertaining and educational uses of interactive technology and HCI."


iPAD + Kinect + Processing

IPAD + Kinect + Processing = stellar foundation for interactions from SuperTouch Group on Vimeo.

Feb 13, 2011

Wii Just Dance2 and Kinect Dance Central: UI and Usability Approaches; Challenges for Developing Accessible Games (revised)

I love to dance- I studied dance through college, and off and on as an adult.   I have a DDR (Dance Dance Revolution) game-floor pad somewhere in my attic gathering dust.  I'm ready for new challenges.

I'm planning on buying a couple new dance games for the Wii and the Kinect. There is more to this story, given my interest off-the-desktop, post-WIMP HCI (human-computer interaction), interactive multimedia and games, and a career as a school psychologist dedicated to young people with disabilities, I'm excited to see where new technologies, interfaces, and interactions will take us.

So what do the wise men of usability have to say about new ways of interacting with games and other applications?

"Kinect has many great design elements that clearly show that the team (a) knows usability, (b) did user testing, and (c) had management support to prioritize usability improvements, even when they required extra development work." -Jakob Nielsen

Jakob Nielsen, one of the godfathers of usability,  shared a few words of wisdom about the Kinect in his 12/27/10 Alertbox post: Kinect Gestural UI: First Impressions.  Although he did not review Dance Central, he concludes that the game he reviewed, Kinect Adventures, was fun to play, despite usability problems.

If this is a topic that interests you, I recommend you read Neilsen's post, and also take a look at which are outlined in the post.  Also take a look at recent essay Neilsen co-authored with Don Norman, another godfather of usability: Gestural Interfaces: A Step Backwards In Usability

Why is this topic important to me?
I have been involved in the Games for Health and Game Accessibility movement for many years.  Lately I've been exploring the OpenKinect project with an aim to create ways of making movement-oriented games accessible for young people with more complex disabilities.  For example, there is a need to have dance and movement games modified for students (and adults!) who need wheelchairs or walkers.  There are students who have milder mobility challenges who love to dance, and the current games don't address their needs.  Some of my students have vision or hearing impairments, too.  They deserve a chance to play things designed for the Kinect.

"OpenKinect is an open community of people interested in making use of the amazing Xbox Kinect hardware with our PCs and other devices. We are working on free, open source libraries that will enable the Kinect to be used with Windows, Linux, and Mac."

Note:  I currently work as a school psychologist with students up to age 22. My main office is adjacent to a large OT and PT room at Wolfe, a program for students who have special needs.   We just had a large interactive whiteboard installed in the room that is begging for us to connect it with the school's Wii,  and soon (we hope), a Kinect.   If we are going to use dance games to help promote healthy activities among our special students, the games need to be accessible for students with cognitive, motor, and other limitations.

Although I can dance, I understand what the world is like through the eyes of many of the young people I work with who have motor coordination and sensory integration problems that interfere with their ability to  move and dance, let alone access fast-paced dance games on the Wii or Kinect. 

My initial plan is to look at what the new dance games might be like from the view of someone who doesn't know how to dance, and admits that they have "two left feet" - an perhaps, no sense of rhythm.  Where would I start?

Wii's Just Dance2 seems to offer some support for learning how to dance through the use of simple movement icons, in the form of outlined figures, that provide information about how to move with the dancer on the screen. As you can see from the video below, the gamer is provided with information about upcoming moves throughout the game.

I decided to take a look at Just Dance2's  MIKA "Big Girl" (You Are Beautiful) because some of the adolescent females I work with have weight concerns that interfere with their health. During the teen years, this can become a vicious cycle, resulting  in less movement, and less participation with peers in physical activities, such as playing dance games.  If a teen has depression as part of this mix, we know that exercise can help,  and a fun dance game might be a life-saver, in more ways than one.

The screen shots below show how the movement icons are used in the game:

I thought it would be useful to learn more about the story behind the making of JustDance2.
At 2:22, Alexia, the project's usability expert, makes her presence known. From what I can tell, she focused on aspects of the game that would make it more usable for non-dancers, including those with "two left feet", to play the game.  (I don't know if there was anyone consulted about accessibility concerns for the game.)

Kinect Dance Central
Dance Central uses a different approach when it comes to "teaching" people how to dance along through the game. It would be interesting to test out Dance Central and JustDance 2 with the same set of people to get a better feel for what works and what doesn't.  Below is a video that previews, in split-screen, the interaction that takes place in Dance Central:

Dance Central Full Motion Preview

In Dance Central, gamers are provided with information about the moves through icons that cycle up the right hand side of the screen.  The level of dance-coordination to keep up with the moves is challenging at times, even for people who are OK at dancing.  Players can select dances according to level of difficulty. 

Kinect Usability with Regular People

Steve Cable (CX Partmers) shared his team's look at usability issues related to the Kinect by testing several games, including Dance Central, with groups of people in his article, "Designing for XBox Kinect - a usability study".  The quote below is from the Steve's article:

"We’ve loved playing with the Kinect. There’s no doubt that the game play is lots of fun. In-game menus are a barrier to that fun. Kinect should allow players to move through menus quickly and compensate for inaccuracy.

We felt the Kinect would benefit from some standardised global controls – much like a controller uses the A button to select and the B button to move backwards. We also think it needs a more responsive pause gesture – one that doesn’t interfere with the user’s game play.

Most of our participants found the Dance Central menu to be more effective, more efficient and more satisfying to use. Here are our recommendations for designing a Kinect menu interface:
  1. Allow users to make selections through positive gestures, rather than timed positions
  2. Place options on a single axis to make them easier and quicker to select
  3. Allow users to control menus with the game pad if they prefer
  4. Use large easy to read text
  5. Don’t make users scroll through options unnecessarily – it takes too long
  6. Users will be distracted if used in a social setting – test your menus in a social context to see if they are prone to errors
  7. Avoid the cursor metaphor, it’s not what gamers are used to seeing in game menus, and makes it harder to implement alternative joypad controls"

Below are screen shots that provide examples of how the movement icons are displayed in Dance Central:

Just Dance 2 Review: Get your body moving. No, really. Give it a shot.
Kexa MacDonald, 10/19/10