When I saw the new Radiohead video, Lotus Flower, featuring a dancing Thom Yorke, I wondered how his moves might play out in a dance application for the Kinect or Wii.
Wouldn't it be fun to figure out a way to represent the dance "steps" in this video? Just a thought.
I love to dance- I studied dance through college, and off and on as an adult. I have a DDR (Dance Dance Revolution) game-floor pad somewhere in my attic gathering dust. I'm ready for new challenges.
I'm planning on buying a couple new dance games for the Wii and the Kinect. There is more to this story, given my interest off-the-desktop, post-WIMP HCI (human-computer interaction), interactive multimedia and games, and a career as a school psychologist dedicated to young people with disabilities, I'm excited to see where new technologies, interfaces, and interactions will take us.
So what do the wise men of usability have to say about new ways of interacting with games and other applications?
"Kinect has many great design elements that clearly show that the team (a) knows usability, (b) did user testing, and (c) had management support to prioritize usability improvements, even when they required extra development work." -Jakob Nielsen
Jakob Nielsen, one of the godfathers of usability, shared a few words of wisdom about the Kinect in his 12/27/10 Alertbox post: Kinect Gestural UI: First Impressions. Although he did not review Dance Central, he concludes that the game he reviewed, Kinect Adventures, was fun to play, despite usability problems.
If this is a topic that interests you, I recommend you read Neilsen's post, and also take a look at which are outlined in the post. Also take a look at recent essay Neilsen co-authored with Don Norman, another godfather of usability: Gestural Interfaces: A Step Backwards In Usability
Why is this topic important to me?
I have been involved in the Games for Health and Game Accessibility movement for many years. Lately I've been exploring the OpenKinect project with an aim to create ways of making movement-oriented games accessible for young people with more complex disabilities. For example, there is a need to have dance and movement games modified for students (and adults!) who need wheelchairs or walkers. There are students who have milder mobility challenges who love to dance, and the current games don't address their needs. Some of my students have vision or hearing impairments, too. They deserve a chance to play things designed for the Kinect.
"OpenKinect is an open community of people interested in making use of the amazing Xbox Kinect hardware with our PCs and other devices. We are working on free, open source libraries that will enable the Kinect to be used with Windows, Linux, and Mac."
Note: I currently work as a school psychologist with students up to age 22. My main office is adjacent to a large OT and PT room at Wolfe, a program for students who have special needs. We just had a large interactive whiteboard installed in the room that is begging for us to connect it with the school's Wii, and soon (we hope), a Kinect. If we are going to use dance games to help promote healthy activities among our special students, the games need to be accessible for students with cognitive, motor, and other limitations.
FIRST STEPS Although I can dance, I understand what the world is like through the eyes of many of the young people I work with who have motor coordination and sensory integration problems that interfere with their ability to move and dance, let alone access fast-paced dance games on the Wii or Kinect.
My initial plan is to look at what the new dance games might be like from the view of someone who doesn't know how to dance, and admits that they have "two left feet" - an perhaps, no sense of rhythm. Where would I start?
Wii's Just Dance2 seems to offer some support for learning how to dance through the use of simple movement icons, in the form of outlined figures, that provide information about how to move with the dancer on the screen. As you can see from the video below, the gamer is provided with information about upcoming moves throughout the game.
I decided to take a look at Just Dance2's MIKA "Big Girl" (You Are Beautiful) because some of the adolescent females I work with have weight concerns that interfere with their health. During the teen years, this can become a vicious cycle, resulting in less movement, and less participation with peers in physical activities, such as playing dance games. If a teen has depression as part of this mix, we know that exercise can help, and a fun dance game might be a life-saver, in more ways than one.
The screen shots below show how the movement icons are used in the game:
I thought it would be useful to learn more about the story behind the making of JustDance2. At 2:22, Alexia, the project's usability expert, makes her presence known. From what I can tell, she focused on aspects of the game that would make it more usable for non-dancers, including those with "two left feet", to play the game. (I don't know if there was anyone consulted about accessibility concerns for the game.)
Kinect Dance Central
Dance Central uses a different approach when it comes to "teaching" people how to dance along through the game. It would be interesting to test out Dance Central and JustDance 2 with the same set of people to get a better feel for what works and what doesn't. Below is a video that previews, in split-screen, the interaction that takes place in Dance Central:
Dance Central Full Motion Preview
In Dance Central, gamers are provided with information about the moves through icons that cycle up the right hand side of the screen. The level of dance-coordination to keep up with the moves is challenging at times, even for people who are OK at dancing. Players can select dances according to level of difficulty.
Kinect Usability with Regular People
Steve Cable (CX Partmers) shared his team's look at usability issues related to the Kinect by testing several games, including Dance Central, with groups of people in his article, "Designing for XBox Kinect - a usability study". The quote below is from the Steve's article:
"We’ve loved playing with the Kinect. There’s no doubt that the game play is lots of fun. In-game menus are a barrier to that fun. Kinect should allow players to move through menus quickly and compensate for inaccuracy.
We felt the Kinect would benefit from some standardised global controls – much like a controller uses the A button to select and the B button to move backwards. We also think it needs a more responsive pause gesture – one that doesn’t interfere with the user’s game play.
Most of our participants found the Dance Central menu to be more effective, more efficient and more satisfying to use. Here are our recommendations for designing a Kinect menu interface:
Allow users to make selections through positive gestures, rather than timed positions
Place options on a single axis to make them easier and quicker to select
Allow users to control menus with the game pad if they prefer
Use large easy to read text
Don’t make users scroll through options unnecessarily – it takes too long
Users will be distracted if used in a social setting – test your menus in a social context to see if they are prone to errors
Avoid the cursor metaphor, it’s not what gamers are used to seeing in game menus, and makes it harder to implement alternative joypad controls"
Below are screen shots that provide examples of how the movement icons are displayed in Dance Central:
During today's visit to Best Buy, I noticed that there were more display centers in various departments in the store, and many of the displays had useful and informative content. The Kinect demo was up and running, too.
Within the store, I noticed a strong emphasis on HDTVs with internet capabilities. I was hoping that the new Internet HDTVs would come with user-friendly touch-screen controllers, or at least an app for use on touch-screen smartphones, iPads, or other touch-screen tablets.
What I found was disappointing. On display were traditional-looking multi-button remote controllers, controllers that looked like PC keyboards, and of course, Sony's confusing multi-featured contribution to the Internet TV scene.
None of the controllers seemed to be easy-to-use, or capable of supporting web-navigation and other web interactions on HDTVs from a distance, especially when the goal is to watch movies and video from a recliner in a darkened family room.
Below is a partially annotated slideshow of pictures I took of some of the displays and other things I encountered at Best Buy. Enjoy!
Close Encounter with Kinect Bowling It wasn't easy trying to bowl and take video with a phone at the same time! (Please excuse the shaky video effects and the view of my fingers.)
Over the next 9 months or so, I'll be in the market for a computer, an iPad or something similar, a Kinect, and maybe a mid-size Internet-enabled HDTV. As a consequence, I turned in to a Best Buy on my way home from work Friday to see what I could find. I didn't want to spend much time at the store, since I'm the type of shopper who goes in, finds the desired item(s), and leaves.
During this trip to Best Buy, my shopping habits were broken, at least temporarily. I found myself wandering around, looking at the numerous displays of all sizes located about the store. The displays distracted me from my intended mission. Below is a slideshow of the pictures I snapped as I wandered about Best Buy:
As a result of the displays, I spend more time in the store than I'd planned. I left the store empty-handed. But that is OK, since I think that this strategy, in the long run, will have the power to entice me back to the store and leave knowing that I've made an informed purchase.
According to Gail Chiasson in her Daily DOOH post,"Best Buy's New In-Store Network", Best Buy established an internal advertising and editorial team in 2009, and officially launched a multichannel network called Best Buy On, which extends the in-store network of displays, to the online magazine, also called Best Buy On . Best Buy On focuses on different theme each month. This month's theme featured all of the cool things that were unveiled at the recent Consumer Electronics Show.
The bottom line with in-store DOOH is that customers are provided with with enhanced information about various products in each department, and this information is coordinated with the enhanced information they can access on-line. Put together, this might lead to better sales, and happier repeat customers in the long run.
As I went about Best Buy, I thought that it would be better if the in-store network of displays was less disjointed. I also was puzzled by the somewhat inconvenient placement of some of the displays. People who stand in the middle of an aisle to look at a screen block the view of the screen and the other items on display, and also block the paths of other shoppers!
Although I didn't spend a lot of time scrutinizing the content on all of the screens I encountered, what I did look at gave me the impression that there is need for improvement. Pretty multimedia content might be attractive to the eye, but if it doesn't meet the needs of customers in specific departments, it might not be effective.
Here is a personal scenario: My 80-year-old mother needs a new laptop and would also like to find an eReader that is suitable for someone her age. She needs a new printer and is thinking about getting a digital camera. She has some money to spend, but from what I can tell, she'd be a little confused by all of the flashing screens!
I'm not sure what my mother would make of the displays, especially those that run animated comments about products from Facebook fans!
(Note: I plan to go with my mom to Best Buy soon, and I'll share this experience on this blog in the future.)
Although there is room for improvement, the Best Buy On approach is a start. I would like to see more touch-enabled displays as part the in-store network, given the fact that Best Buy sells touch-screen gadgets, tablets, and all-in-one PCs. The large-screen touch-enabled displays could provide customers with an Amazon- like advisory feature. This is something that would be very helpful to my mom and other non-geeks.
Best Buy also needs to get the Kinect demo up and running!
Info from the DepthJS website: "Navigating the web is only one application of the framework we built - that is, we envision all sorts of applications that run in the browser, from games to specific utilities for specific sites. The great part is that now web developers who specialize in Javascript can work with the Kinect without having to learn any special languages or code. We believe this will allow a new set of interactions beyond what we first developed."
Comment: I went to Best Buy today and almost bought a Kinect. Unfortunately, the demo hadn't come in yet, so I decided to wait until I could give it a try. I'm curious to experiment with what it can do.
"Hi, Google. My name is Johnny" Johnny Chung Lee announced on his Procrastineering blog that he's accepted a position at Google as a "Rapid Evaluator". I'm not sure what he will be doing in this position, but his title is intriguing!
Here are some of my previous posts devoted to the work of Johnny Chung Lee:
Josh Blake recently interviewed Tamir Berliner, one of the founders of PrimeSense. If you haven't heard, Microsoft's Kinectwas based on work by PrimeSense, and licensed their technology. PrimeSense provides consumer electronics with natural user interaction capabilities. The good news is that the company recently released open-sourced middleware for natural interaction and depth-camera drivers. It will be interesting to see how this will play in the near future!
In the interview, Tamir discussed a number of topics related to postWIMP technologies. He also announced the newly created OpenNI, "an industry-led, not-for-profit organization formed to certify compatibility and interoperability of Natural Interaction (NI) devices, applications, and middleware." It is good to see this level of support for the cause!
Here is a quote from the interview that I especially liked:
"I believe that till today the devices we’ve been using, made us learn greatly lot about them before we could use them and gain their value. I’m pretty sure everyone who is reading this has got at least 3 remotes sitting on his living room table, and at least once a week needs to help someone use their computer/media center/phone/etc. It’s time for that to change and it’s up to us, the technologists to make this revolution happen, it’s time for the devices to take the step of understanding what we want and making sure we get that, even without asking if it’s a trivial task as opening a door when we approach, closing the lights when we leave the room, even making sure we have hot water to shower with when we return from work or wake up in the morning, depends on what we normally do." -Tamir
RELATED Here are a couple of videos from the OpenNI website that demonstrate OpenNI-compliant applications:
OpenNI-compliant real time skelton tracking by PrimeSense
OpenNI-compliant real time SceneAnalyzer by PrimeSense
FYI:
Josh Blake is the author of the Deconstructing the NUI blog. Over the past couple of years, he's explored natural user interfaces and interactions through his work on applications designed for Microsoft Surface and Win7 with Windows Presentation Foundation.
About a month ago, Josh organized OpenKinect, an on-line community to support collaboration among people interested in exploring ways to use Kinect with PCs and other devices. An example of this effort is the open source code, libfreenect, which includes drivers and libraries for Windows, Linux, and OS X.
I recently posted about the Therenect, a gesture-controlled digital theremin created for Microsoft's Kinect, created by Martin Kaltenbrenner - Therenect: Theremin for the Kinect! (via Martin Kaltenbrenner)It looks like Martin has been busy polishing up the application over the past few days, as you can see from the video below:
I follow the O'Reilly Radar blogs and came across a recent post about an information visualization created by blprnt two years ago using Processing. I think it would have great potential if it was re-purposed for use on the Kinect! In the article, Edd Dumbill discusses the advantages of using Processing to create data and information visualizations.
One example of the power of Processing is an information visualization, "Just Landed -36 Hours, created by Jer Thorp. Jer gathered tweets from Twitter that included the statement, "just landed", along with location information for each tweet, within a 36-hour period, to create the visualization.
36 Hours- Just Landed is a great 3D visualization of air travel on our planet. I especially lik the different views that the application provides. As soon as I watched the Just Landed video, I thought it would be great if it could be revamped for use on the Kinect! (Leave a comment if you know of anyone working on a project in this area.)
Information about the video from blprnt's Vimeo site:
"I was discussing H1N1 with a bioinformatics friend of mine last weekend, and we ended up talking about ways that epidemiologists model transmission of disease. I wondered how some of the information that is shared voluntarily on social networks might be used to build useful models of various kinds...I'm also interested in visualizing information that isn't implicitly shared - but instead is inferred or suggested...This piece looks for tweets containing the phrases 'just landed in...' or 'just arrived in...'. Locations from these tweets are located using MetaCarta's Location Finder API. The home location for the traveling users are scraped from their Twitter pages. The system then plots these voyages over time...I'm not entirely sure where this will end up going, but I am reasonably happy with the results so far. Built with Processing (processing.org) You can read more about this project on my blog - blog.blprnt.com"
As soon as Kinect was released by Microsoft, there was a flurry of app development. Evoluce and So Touch partnered to create a presentation application for the Kinect that could be used in work settings. Take a look!
Information about Air Presenter Plus, from the So touch's YouTube channel:
"So touch, the leading creative software company for new digital technologies, in partnership with Evoluce, the leading provider of advanced multi-touch screen technologies, present: So touch Air Presenter for Kinect. The world's first presentation software optimized for Kinect.
Turn your corporate presentations, welcome areas, trade show booths and point of sales into mind boggling experiences, controlling your presentation with multi-touch gestures leveraging So touch Air Presenter gestures software and Evoluce Kinect Windows 7 software.
Integrate your usual PDF, Power point, JPG and video materials into So touch multi-touch minority report's style interface and control it with gestures in the air.
So touch Air Presenter is delivered with a very graphic player, featuring a multi-touch zoom mode and an integrated video player as well as a very easy to use content manager.
So touch Air Presenter content, sourced locally or from the network, can be played on multiple screens at the same time. So touch Air Presenter content manager can deliver customize or generic content to each player.
So touch Air Presenter packaged with Evoluce Kinect Windows 7 software will be released soon. So touch Air Presenter is already available for TUIO based gestures devices. To know more and download a free trial version, visit http://www.so-touch.com/air-presenter"
Here are a couple of new natural user interface videos. The first video, by Evoluce, demonstrates gesture interaction/navigation in Windows 7 applications supported by Kinect. The second video, by Immersive Labs, shows multi-touch product browsing interaction on a large display.
Martin Kaltenbrunner's description of TuioKinect: "TuioKinect tracks simple hand gestures using the Kinect controller and sends control data based on the TUIO protocol. This allows the rapid creation of gesture enabled applications with any platform or environment that supports TUIO tuio.org/ You can download the application from: code.google.com/p/tuiokinect/ Music: Jabon Jabon by El Club de los Astronautas (Institut Fatima)"
I've played around with Tuio and OpenFrameworks, but it has been a while. I can't wait until I have time to dig into this with a Kinect. I think this has great potential for supporting learning and communication among students with special needs.
"The Therenect is a virtual Theremin for the Kinect controller. It defines two virtual antenna points, which allow controlling the pitch and volume of a simple oscillator. The distance to these points can be controlled by freely moving the hand in three dimensions or by reshaping the hand, which allows gestures that are quite similar to playing an actual Theremin."
"This musical instrument has been developed by Martin Kaltenbrunner at the Interface Culture Lab at the University of Art and Industrial Design in Linz, Austria. The software has been developed using the Open Frameworks and OpenKinect libraries."
I was watching a DVR'd episode of NCIS tonight with my husband and noticed Cisco Telepresence video conferencing system was a player in the story line, as well as a Cisco Cius touch-screen tablet. It seems that on TV, laptops, desktops, and old-fashioned cell phones are history. "Emerging" technologies are woven into the story lines of more television episodes, including CSI, NCIS, Bones, Grey's Anatomy, and others.
The original intention of this post was to discuss the concept of emerging technologies and product placement/embedded advertising on television programs and movies, and share a few interesting examples related to his topic. I quickly realized that there is much more to this story. Why? More people access video and interactive multimedia content when they are on-the-go using laptops, smart phones, iPads, and similar tablets. New televisions, such as Sony Internet TV, are internet-enabled, and many people already access the web content on their televisions through devices such as game consoles or Apple TV.
It is a marketer's dream.
Unfortunately, we might not be ways to "opt-out" of all of the indirect (and direct) advertising that will come our way as we access video and related content across multiple platforms. It won't be as easy as blocking pop-up ads or fast-forwarding the DVR!
Below are some examples of ways some emerging technologies are "placed" in television/film, grouped by company. In the "Apple" section, I've included video of Steve Jobs introducing iAds. Near the end of this post, I've included links that relate to ethical legal and ethical issues regarding product placement and embedded advertising.
Food for thought. I'm still digesting what I've found!
Apple iADMobile advertising that delivers interaction and emotion, 1 billion ad impressions a day, within your app. Apple's iAD isn't really product placement. It is about embedded ads in your mobile devices. "Who wants to get yanked out of their ad?"-Steve Jobs
"iAd is a breakthrough mobile advertising platform from Apple. With it, apps can feature rich media ads that combine the emotion of TV with the interactivity of the web. For developers, it means a new, easy-to-implement source of revenue. For advertisers, it creates a new media outlet that offers consumers highly targeted information." -Apple
As I searched for more information about product placement and embedded advertising, I came across a few posts/websites that suggests that in some circles, this is a hot/controversial topic:
Joseph Lewczak and Ann DiGiovanni, WLF Legal Backgrounder, 4/9/10
FIT Media FAQs (FIT= Fairness and Integrity in Telecommunications Media)
"FIT Media is a non-partisan coalition of health, media and child advocacy organizations and professionals supporting transparency and child protection in embedded TV advertising."
This is an interesting website - FIT Media covers topics such as "Advernews", "Embedded Propoganda", "Deceptive Advertising", and ways that embedded advertising might be harmful. Week ahead: FCC meeting, Do Not Track hearing Cecilia Kang, Washington Post 11/29/10
What is Apple's New Privacy Policy? "Amidst all the glitz of releasing a new mobile operating system and iPhone, Apple quietly updated their privacy policy. Why?"
Michael Kassner, Tech Republic, 6/28/10
FYI: If you have an iPhone running iOS 4 and wish to opt-out of iAD, you can do so at http://oo.apple.com
Xerox Patent Filing Make Product Placement Addressable The Media Buyer, 1/12/10 "The patent describes the system (via GoRumors) as having the ability to alter content within a program based on the viewer. For example, if a character on a show mentions Macy’s, that content could be shown to general viewers. But that small portion of the broadcast could be “marked,” and the content could be changed so that the character instead says the name of sporting goods store Modell’s. That portion of the broadcast would be served to viewers who are into sports. Similarly, if the storefront was shown during the program, general audiences would see the Macy’s store, while sports fans would see the Modell’s store."
