LM3LAB's Useful Map of Interactive Gesture-Based Technologies: Tracking fingers, bodies, faces, images, movement, motion, gestures - and more

Nicolas Loeillot, of LM3LABS, has been ahead of the natural user interaction/ interface game for many years as his company has expanded. He's done quite a bit of deep thinking about the work of his company, and has used this wisdom to create a nice concept map that describes how LM3LABS' solutions fit into the world of gesture-based control and interaction:




In my opinion, this chart would make a great template for mapping out other natural interaction applications and products!


Here is the description of the concepts outlined in the chart:


"If all of them belong to the “gesture control” world, the best segmentation is made from 4 categories:

• Finger tracking: precise finger tracking, it can be single touch or multi-touch (this latest not always being a plus). Finger tracking also encompasses hand tracking which comes, for LM3LABS products, as a gestures.
• Body tracking: using one’s body as a pointing device. Body tracking can be associated to “passive” interactivity (users are engaged without their decision to be) or “active” interactivity like 3D Feel where “players” use their body to interact with content.
• Face tracking: using user face as a pointing device. It can be mono user or multiple users. Face tracking is a “passive” interactivity tool for engaging user in an interactive relationship with digital content.
• Image Tracking: Augmented Reality (AR) lets users use images (flyers, real products, t-shirts, faces,…) to interact with digital content. AR can be markerless or marker-based. Markerless technology has advantages but marker-based AR is easier for users to understand. (Please note here that Markerless AR is made in close collaboration with AR leader Total Immersion)."  -LM3LABS

   If you are interested in this subject and want to view some good examples of off-the-desktop interfaces and interactions, take a look at the LM3LABS blog, as well as Nicolas Loeillot's Vimeo channel.  Also take a look at the sample of posts I've written about LM3LABS over the last few years - the links are at the end of this post.

I love LM3LABS' Interactive Balloon:

Interactive balloons from Nicolas Loeillot on Vimeo.


Interactive Balloons v lm3 labs v2 (SlideShare)

Background

I first discovered LM3LABS when I was taking a VR class and researching interactive, immersive large displays in 2005 or 2006.  Back then, there wasn't much information about this sort of technology.  A lot has changed since then!


I've learned quite a bit from watching LM3LABS (and others) grow, given my passion for postWIMP interactive technology and my commitment to blogging about this subject.   Nicolas has really worked hard in this arena.  As early as 2005, LM3LABS was working with Scala to provide "smart" interactive displays, and his company's applications have been supported by computer vision technologies for many years, allowing for gesture-based, or "touch-less" interaction, as demonstrated by the Catchyoo Interactive Table.  This application caught my eye back in early 2007, when I was working on projects for large interactive displays for my HCI and Ubicomp classes, and was thinking about creating a table-top application.


My hunch is that LM3LABS has set the foundation for further growth in the future, given the lessons they've learned by taking risks with postWIMP technologies over the past few years!


Previous Blog Posts Related to LM3LABS:
Interactive Retail Book (Celebrating history of Christian Dior from 1948-2010 (video)
Ubiq Motion Sensor Display at Future Ready Singapore (video)
Interactive Virtual DJ on a Transparent Pane, by LM3LABS and Brief Ad
LM3LABS' Catchyoo Interactive Koi Pond: Release of ubiq'window 2.6 Development Kit and Reader
A Few Things from LM3LABS
LM3LABS, Nicolas Leoillot, and Multi-touch
More from LM3LABS: Ubiq'window and Reactor.cmc's touch screen shopping catalog, Audi's touch-less showroom screen, and the DNP Museum Lab.

About LM3LABS

"Founded in 2003 by a team of passionate researchers, engineers, designers, and marketers from various international backgrounds, focused on fast transformation of innovation into unique products, LM3LABS is a recognized pioneer in computer vision-based interactivity solutions. Keeping a strong customer focus, LM3LABS' team of unique people pioneers new directions, explores new concepts, new technologies and new interactions.  Engaging, playful and magic, LM3LABS' products and solutions are always scalable and reliable"

info@lm3labs.com

Note to readers:

Over the past couple of years there has been an explosion of postWIMP technologies and applications, and with this pace, it has been difficult for me to keep abreast of it all. There is quite a bit I miss, given my full time job and daily life!

I welcome information about postWIMP interactive technologies and applications from my readers.  Due to time constraints, not interest, I am not always able to post about a topic as soon as I'd like.  That is OK, as my intention is not to be the first blogger to spread the latest tech news.  I like to dig in deep when I can and make connections between innovative, interesting technologies and the people and ideas behind them. 

Philipp Geist: Blending the Physical with the Digital; Google TV/Leanback, Vimeo's new Couch Mode, oh...and ViewSonic's 3D (glasses-less) pocket camcorder...

I'm thinking about getting one of the new "internet ready" TVs.  I have a serious reason to do this. I'm working on some interactive video projects, and a couple of my projects are geared for teens and young adults who have autism.*   My hunch is that many of my students would like to watch- and interact with-content optimized for Google TV and Vimeo's Couch Mode.  The content is designed to look good on larger high-resolution flat-screen displays, and I'm sure it would be great on my school's newer SMARTBoards.  I need to learn more about  developing applications for this purpose.

(Currently I use my HP 22-inch TouchSmart PC to view web-based video content, and to evaluate websites that provide "touchable" and interactive content that might work well on interactive whiteboards.)

At any rate,  I've been looking for great videos that have the potential for use at work with older students who have autism. I'm also looking for effective ways that the students can use to interact with multimedia and video content. This is important, since the students have minimal verbal communication skills, have limited reading ability- if they can read at all.  They learn about their world through visual means, and are capable of learning much more - but not through traditional means.

Since our school is focusing on globalization and learning about the cultures of other countries, I've been on the lookout for some interesting videos that might appeal to our students.  

Today I came across a great find- Philipp Geist.  Who is Philipp Geist?  According to his bio, Philipp works internationally as a light and multi-media artist in the mediums of video, performance, photography and painting. Some of his work focuses on architecture, history, and cultural heritage.  A good example of his work is the installation he created for a festival in Thailand in 2009:

Bangkok / Thailand 4D Light & Sound Installation by Philipp Geist (23Min) from Philipp Geist | Videogeist on Vimeo.

"The one-hour show is the central part of the celebrations and will be seen by thousands of visitors.  It interprets artistically the king's life and his work dedicated to public welfare. The art installation combines images of the kings and his social projects in the past and present with 3D animations of Thai natural and cultural heritage and abstract painterly passages." (from the Vimeo site)

Phillip Geist's Showreel

Philipp Geist Showreel 2009 from Philipp Geist | Videogeist on Vimeo.

HIGH-RES MULTIMEDIA WEB CONTENT ON LARGE PANEL HD TV!

This might boost holiday gift sales and in turn, give a little jolt to the economy. To do my duty for my country, I will continue to research Internet TV as I narrow down my selection for my new Internet-ready TV.... Below is some Information about Google TV, Google Leanback, and Vimeo's Couch mode that I've recently gathered to share with my IMT followers:

GOOGLE TV:  "The web is now a channel"

"With Google Chrome and Adobe Flash Player 10.1, Google TV lets you access everything on the web. Watch your favorite web videos, view photos, play games, check fantasy scores, chat with friends, and do everything else you're accustomed to doing online. Plus, the world's best websites are now being perfected for television -- check out our Spotlight gallery for examples."  "The worlds' favorite websites are being tweaked and perfected for the television." -Google TV

I'm not too excited about the design of the application that transforms your Android phone or iPhone into a remote control.  I hate most remote controls.  According to Google TV, multiple phones can control the same TV, and you can use your voice to search, which seems like it would be a good thing...  I wonder if they tested this out with real families, not just families of Google TV techies .

GOOGLE LEANBACK Video (Integrated into GoogleTV)

Google Leanback 
When I visited the Leanback website, I encountered the following screen with a suggestion that I type in what I was looking for.  I typed in "lynnvm", the name of my YouTube channel.  Apparently Google provides you with a randomly generated featured video that appears in the background that has nothing to do with what you are looking for.  


In this screen shot, my YouTube channel offerings are in the foreground. "Maleficent Halloween Tutorial" is what played in the background: 

VIMEO INTRODUCES COUCH MODE: Vimeo's version of Google's Leanback is Couch Mode.  It is optimized for use on Google TV, so that makes things less complicated in the world of videoviewingland.   According to Ryan Hefner's article on the Vimeo staff blog, "Couch Mode is a special new section of Vimeo that allows you to watch collections of videos (such as Staff Picks, your inbox, your videos, etc.) completely uninterrupted like a TV channel."
Couch Mode works on computers, but since it relies on HTML5 and CSS3, without Flash, it only works with Chrome and Safari browsers.  For more information, see the video below:



RELATED

VideoGeist: Media installation/mapping, visual art concepts, visual art

Philipp's Blog

VIMEO Couch Mode

Google TV

Google TV Web Site Optimization Guide

Google TV is coming to 10,000 lucky developers

"A few of our favorites include Net-A-Porter, which lets you watch runway videos and shop for high fashion; Meegenius, a place where you can read and customize children’s books; TuneIn, a personal radio for your TV; and The Onion which always gives us a good laugh." - Google TV Blog

MeeGenius If you are a teacher, parent, kid, or lover of children's books, visit this interactive website ASAP. It is optimized for Google TV and works nicely on touch-enabled screens and devices.

As I was wrapping up this post, I came across information about Viewsonic's new 3D, glassesless pocket camcorder.  I'll update information about this new gadget when I have a chance to learn more about it. !

Comment: The idea of developing interactive multimedia apps in 3D intrigues me. At this point, the technology is too new for an "armchair technologist" like me to pursue with my incredibly busy work obligations.  I don't have the money to buy a 3D video camera.  But I might try this out, if it is true that it only costs $238.00!

Viewsonic introduces 3Dv5 3D pocket camcorder, no glasses required
Darren Murph, Engadget, 10/20/10
Film Videos In 3D for Under $250 With Viewsonic's 3DV5
HotHardware, 10/28/10


* About me:  
I presently work full time as a school psychologist at a high school and at a program for students with more severe disabilities, including autism.  The students I work with have made amazing gains through the use of interactive multimedia applications, and also have responded well to video presented on the large IWB screens.   


I went back to school to take computer courses, initially so I could make interactive multimedia applications and games. I continue to blog about interactive multimedia,  emerging/ new technologies, and topics related to post-WIMP HCI/UX/ID/IA.  Although my "spare time" is limited,  I try to keep up my technical skills whenever I can by working on projects that can support the students I work with.