MIT MediaLab's DepthJS: Now your web page can interact with the Microsoft Kinect using Javascript (Link to code, more)

MIT Media Lab's DepthJS website

Info from the DepthJS website:
"Navigating the web is only one application of the framework we built - that is, we envision all sorts of applications that run in the browser, from games to specific utilities for specific sites. The great part is that now web developers who specialize in Javascript can work with the Kinect without having to learn any special languages or code. We believe this will allow a new set of interactions beyond what we first developed."


DepthJS is open source under the AGPL license. Code: https://github.com/doug/depthjs

RELATED
Gestures that your TV Will Understand (Features information about PrimeSense)
Tom Simonite, MIT Technology Review, 1/21/11


Hackers Take the Kinect to New Levels
Timothy Carmody, MIT Technology Review 12/2/10


Microsoft Kinect: How the device can respond to your voice and gestures
Erica Naone, MIT Technology Review, January/February 2011






Comment:
I went to Best Buy today and almost bought a Kinect.  Unfortunately, the demo hadn't come in yet, so I decided to wait until I could give it a try.  I'm curious to experiment with what it can do. 

Gesture "multitouch" 12 x 7 interactive video wall provides tours of I/O Data Centers' facilities

I came across this demonstration of I/O DataCenter's 12 by 7 foot interactive video wall that makes playing around with views of data center modules...interesting! The display is a gesture-based "multi-touch" system. (I'll update this post when I get more information.)



Here is the description from the Datacenter YouTube channel:


"Instead of hauling a 40-foot long modular data center to a trade show, i/o Data Centers is taking a high-tech approach to customer tours of their i/o Anywhere modular data center. The i/o team has created a 12-foot by 7-foot touchscreen video wall to provide interactive tours of the company's facilities. Selecting a "hot spot" pops up a virtual data center, complete with cross sections and product info, following the concept of the touch screens in the sci-fi movie "Minority Report.""


FYI: I/O Data Centers has an application that runs on the Surface.

UPCOMING:
Stay tuned for my upcoming posts! 


News about LM3LABS (Previous post)
Interactive Surveillance CCeline Latulipe (technologist) Annabel Manning (artist)

Interactive Video

OpenKinect, NUI, NI

Games, serious games

3D, including glassless

Large interactive displays in public spaces

Convergence 2011
Telepresence Updates
RENCI, Charlotte Viz Lab

Gesture Vocabulary from N-Trig: "N-act Hands-on"

N-Trig is a company founded in 1999 that provides pen and multi-touch solutions that integrate into LCDs and other devices, and provides opportunities for independent software vendors (ISVs) and original equipment manufacturers (OEMs) to create new interactive and hands-on computing experiences, according to the company's profile. The latest news about N-Trig's interactive capabilities was outlined in a recent article by DanaWollman, in Laptop:

N-Trig's Four-Fingered, Two-Handed Multi-touch Promises a World of New Gestures

I found the following video from N-Trig on YouTube, released on 5/11/10, that shows the new gesture set that is supported by N-Trig:

The N-act Gesture Set (depicted in the video below)
N-act3SideSweep for browsing, use fingers together for browsing
N-act2+1 - select from a displayed menu
N-act3Tap- displays open windows in a 3D carousel
N-act3Hold-rotates the 3D carousel
N-act2Scroll- scroll through a document
N-act2Tap-minimizes the open window, displays the desktop
N-act1Touch- select an item on the screen
N-act4Tap-displays customized, relevant list of web page icons; selected text/item is pasted into the chosen app.
N-act4Zoom-magnifies a movable selected area of the screen
N-act4Select-selects an area and opens a context sensitive menu

avitalntrig
Here is the promotional information from the YouTube video:
"This video demonstrates the N-trig N-act Gesture Vocabulary, a set of true multi-touch gestures for two plus one, three- and four-fingers, enabling users to perform an action directly on the screen, and providing a rich set of hand movements that enhance the overall user experience, enabling a whole new approach to how we interact with our computing devices, for a true Hands-on computing experience."


RELATED

N-Trig's Four-Fingered, Two-Handed Multi-touch Promises a World of New Gestures

Dana Wollman, 5/1/10, Laptop

www.n-trig.com
N-trig DuoSense Technology
The Future is Now:  Creating and Developing a Touch-Enabled World (pdf)
N-trig N-act Hands-On Gesture Vocabulary (N-Trig website)
Better Multi-Touch Displays Coming 
Mike Miller, Forward Thinking Blog, PC Mag (3/3/10)
DuoSense: Creating a Multi-touch Enabled World (November 2009)

The attracTable is Coming Soon: Sony will launch a high-definition touch and gesture- interactive tabletop, using Actracsys's technology!

Sony will be introducing a full high-definition interactive table, a result of a collaboration with the Swiss company Atracsys.





(At about 2:14 in the video below, there is a demonstration of an application that recognizes facial features and expressions, which are used to control and manipulate images on the screen.)
Images from the Sony Stand at Vision 2009


Here is an "overview" video that shows a number of uses for the Attractable:

Here is a version of the atracTable, using a tangible user interface to create music:

Here is the "Nespresso" table, which provides people with information about the type of coffee that you are drinking. It makes more sense as demonstrated in the video.
Atracsys @ Baselworld 2010


beMerlin:  Interactive gesture-based application for retail:

Applications for touch, multi-touch, and gesture interaction: What's Marching to Market?

Every week or so, I hear about a new multi-touch enabled laptop.Today, I came across a review on Gizmodo about the Lenovo ThinkPad T400, which is a 14.1 capacitive touchscreen laptop that can handle four touches at once:

Lenovo ThinkPad T400s with Multitouch Review: Finger Flicking Fun


The Multitouch ThinkPad in Action

Lenovo ThinkPad T400s with Multitouch from Gizmodo on Vimeo.

SCREENSHOT
 
"The screen is pure beauty. Like a capacitive touchscreen phone, light taps on the display are responsive and you don't have to think about positioning your finger in a certain way to open apps or rearrange windows." - Joanna Stern, Gizmodo, 9/15/09


OTHER EXAMPLES
 
THE FUJITSU LIFEBOOK T5010

Add the Fujitsu LifeBook t5010 to the Windows 7 Multitouch Laptop List
Gizmodo, Joanna Stern, 9/15/09



DELL LATITUDE XT2
 

 












TOUCH DEMO- HP TOUCHSMART TX2 & WINDOWS 7


AIR HOCKEY FOR WINDOWS 7: IDENTITY MINE


FINGERTAPPS





Want to create something multi-touch?

It looks like there will be a growing need for Windows 7 application developers.  Here's some code examples using Windows Presentation Foundation and the Windows 7 Multitouch API

Windows 7 Multitouch Application Development, Part I (The Code Project: Kunal Chowdhury)


Other Developer Resources

Touchlib Multi-touch Development Kit  (NUI GROUP)

MSDN Windows Touch: Developer Resources

nTrig's Multi-Touch Download for Windows 7

NUITech's Snowflake Suite for NextWindow

TouchKit Multi-touch Development Kit


GestureWorks Flash Multitouch SDK

3M Multi-touch Developer Kit


RELATED
Touch-less Interaction from lm3labs

Create the future you want. with ubiq'window from Nicolas Loeillot on Vimeo.

More about gesture interaction and multi-touch can be found on this blog- do a search!

The Interdisciplinary World of Dance and Interactive Technology:

After I finished my post "What happens when engineers and musicians get together? They get Calvin Harris "Ready for the Weekend" with the Humanthesizer and Bare Ink", I thought I'd catch up on the world of dance and technology.

I took a look at an on-line community, Dance-Tech.NET which focuses on the "interdisciplinary explorations on the performance of motion". I was happy to find that two people I know are members of this vibrant group of people:

Celine Latulipe

Dr. Celine Latulipe is a Human-Computer Interaction researcher at the University of North Carolina-Charlotte. She is the lead in the Dance.Draw project, a collaboration between the Software and Information Systems department, the Department of Dance, and a digital artist.

“A Mischief of Mus Musculus”

"Dance.Draw: Exquisite Interaction"

As Celine puts it, "You are more than your eyes and one hand. Why should you have to be less than you are when confronted with a digital device? Exquisite interaction is a collection of research projects...that aim to enrich your expressive creativity in the digital realm by allowing you and your collaborators to use more of your body in that digital interaction"

Visualization code was designed by Mike Wirth, using Processing, to create the art displayed on the Art from Dance page of the Dance.Draw project.

Doug Fox

Doug Fox created and maintains the Great Dance and Kinetic Interface blogs. When he was 42, he decided to study dance - modern, ballet, and jazz. He is interested in the intersection of dance and technology, and more recently became involved in the study of animation related to dance and movement.

The following is from Doug's "About" page - the videos and links are worth exploring:

"As a starting point, I'd like to encourage readers to visit the Movement Is at the Heart of Scientific and Technological Change background page. Here you will find 16 videos (plus links to more videos) that show in very concrete terms how new computer interfaces and digital devices being used in a range of fields are, in essence, body-centric and movement-centric..."

Here is a sample of Doug's topics:

Dance Theater Workshop's Twitter Community Choreography
Dance Vlogging, Will this Video Genre Increase in Popularity?
Shoot Dance Videos with the new iPhone 3GS
Prodigy Warrior's Dance Combines Stop-Motion Animation and Puppetry
Choreographing Gesture Controls for Interactive Devices (be sure to read the comments and link to Arizona State University: School of Arts, Media and Engineering)

Doug Fox writes about topics that are truly interdisciplinary:




Dance Your Ph.D. Contest - A Wonderful Merging of Dance and Science
This contest was funded by the American Association for the Advancement of Science. The objective of the contest? "Using no words or images, interpret your Ph.D. thesis in dance form".

Contest winners were paired with choreographers, who created a new dance work based on a peer-reviewed article in a scientific journal. (The Science Dance Match-Up Challenge)

I just had to post the videos here - you can read detailed descriptions about each video on YouTube:

The role of Vitamin D in beta-cell function


Cerebral activation patterns induced by inflection of regular and irregular verbs with positron emission tomography: A comparison between single subject and group analysis


A Molecular Dance in the Blood, Observed


Popular Choice: Physics Tango "Single Molecule Measurements of Protelomerase TelK-DNA Complexes."


For a look at the 2008 "Dance Your Ph.D." videos, and the the videos of the 2009 contest entries, visit the 2009 AAS/Science Dance Contest web page.

The four videos generated by the professional choreographers can be accessed on Vimeo.

Links to the scientific articles, the bios of the choreographers and scientists, and videos of the choreographer's renditions of the scientist's work can be found on the Science/AAAS website:

The Gonzo Scientist: The Science Dance Match-Up Challenge
John Bohannon, Science, 4/17/09

SOMEWHAT RELATED
Usually I reserve this section for links and information from external sources, but this time, I thought I'd share a few of my opinions that are somewhat related to this topic.

Why do I think interdisciplinary pursuits are important?

In my opinion, to move forward, the arts and other disciplines need to embrace the interdisciplinary way of thinking. There is much that is mixing and converging as I type these words. There is less emphasis for young people to pick one little corner of a field of study and make it their life's obsession. I have always had an "interdisciplinary" approach to life, ever since I can remember. I attribute this to my parents, who nurtured me as a musical, dancing, artistic child to pursue my talents at a young age, and when faced with choosing a college major, to go for a double major.

My interdisciplinary nature has fueled my journey into the world of technology, and my early background in the arts probably explains why I'm excited about interactive multimedia, extending into the realms of emmersive games, multi-touch and gesture interaction, and technology-supported interaction that takes place in larger public spaces.

Note:

As many of my readers know, I work as a school psychologist in my "day job". I DO miss the time when I was working part-time and taking graduate classes at UNC-Charlotte, but when the economy went downhill, It was necessary for me to return to work full-time.

The upcoming academic year will be busy! With the recent budget cuts to school districts in the in the state of North Carolina, I will have another school added to my schedule. I'm excited that it is a high school for technology and the arts, and that the school has a strong dance program!

I've posted quite a bit recently, since I have plenty to share. Soon I will only have time to post about 3 times a week.

What happens when engineers and musicians get together? They get Calvin Harris "Ready for the Weekend" with the Humanthesizer and Bare Ink

So what happens when engineers and musicians get a chance to mix it up?

The Humanthesizer project is the result of a collaboration between musician Calvin Harris, and masters students from the Royal College of Art Industrial Design Engineering program. What makes this project innovative is that a special conductive ink, called Bare Ink, was used to support the creation of the music.

From the Bare Conductive website:

"Bare is a conductive ink that is applied directly onto the skin allowing the creation of custom electronic circuitry. This innovative material allows users to interact with electronics through gesture, movement, and touch. Bare can be applied with a brush, stamp or spray and is non-toxic and temporary. Application areas include dance, music, computer interfaces, communication and medical devices. Bare is an intuitive and non-invasive technology which will allow users to bridge the gap between electronics and the body."

(Please overlook the bikini-clad women in the videos below, because the innovative technology supporting the Humanthesizer is worthy of the watch. Parents be forewarned!)

The Making of the Humanthezizer


Close to the finished version:



Information from YouTube regarding the Humanthesizer project:

"Calvin performs a version of his new single Ready For The Weekend on a unique human synthesiser. The instrument employs 15 bikini clad models painted with Bare Conductive, a new skin safe ink which conducts electricity. When the performers touch the connection completes a circuit, triggering a sound....The instrument consists of 34 pads on the floor which have been painted with the conductive ink and connected to a computer via some clever custom electronics. The performers stand on the pads, and touch hands to complete a circuit and trigger a sound. Different combinations of pads trigger the different sounds needed to play the track....Bare Conductive was developed by RCA students Bibi Nelson, Becky Pilditch, Isabel Lizardi and Matt Johnson. The custom electronics and software for the project were created by Matt Johnson, employing two Arduinos and the graphical audio programming tool Max MSP. The performers, floor pads, Arduinos and Max MSP combine to create a giant MIDI controller which is used to create music which is sequenced and quantized with Ableton Live." (Special thanks to: RCA / Bare Conductive - Bibi Nelson, Becky Pilditch, Isabel Lizardi, Matt Johnson - http://www.bareconductive.com)

(The Ready for the Weekend music video is available on YouTube. Parents: the music video has dancing girls throughout, in one-piece bathing suits.)

Thanks to Celine Latulipe for the link!

Windows Vista User Experience Guidelines: All Windows programs should be touchable!

If you are a designer/developer who would like to develop applications that can support touch interaction, it will save you some time to study the Windows Vista User Experience "Touch" guidelines. You'll find information about the eight flick events, (gestures that correspond to keyboard shortcuts for navigation and editing), the 40 or so gestures recognized by Vista, tips for creating controls, GetMessageExtraInfo function, and more.


"As touch spreads from Tablet PCs to other types of computers, software program developers and designers will find it increasingly important to support touch as well. All Windows programs should have a great touch experience. Users should be able to perform your program's most important tasks efficiently using a finger."

Natural User Interface: Overview of multi-touch technology and application development by Harry van der Veen,- Business to Buttons

Harry van der Veen from Natural User Interface Europe AB, was one of the keynote speakers at the Business to Buttons: Designing for Effect conference, held in June 2008. In this presentation video, Harry discusses the past, present, and future of multi-touch technology, and reviews the importance of multi-touch over single touch displays. He also provides a good overview of gesture interaction, something that he researched when he was a student. This presentation includes several video examples of multi-touch applications in action.

The presentation is well worth the 30-minute view!

"Harry van der Veen is a Bachelor of Multimedia, derived from the Dutch education Communication, Multimedia and Design, focused on Interaction Design and Project Management. He is CEO, co-founder and co-owner of the Sweden based commercial company Natural User Interface Europe AB, which focuses on delivering standardized and customized multi-touch hardware / software solutions and services to the global market. In addition to that, he co-founded the NUIGroup community, which is the worlds largest online platform where a global network of people share their ideas and information in an open source community, focused on multi-touch hardware and software solutions."


NUIGroup Community


Harry van der Veen's blog

Natural User Interface Europe AB (Harry van der Veen's company)

NUIGroup Wiki: This wiki includes tutorials for developing multi-touch applications, building your own low-cost multi-touch table, and information about current projects that are in progress.

Related Information:

The Business to Buttons: Designing for Effect conference was held on June 12-12 in Malmo, Sweden, organized by Malmo University and inUse, a user experience consultancy. Partners in this conference included Adaptive Path, a product experience strategy and design company, Patrick W. Jordan, a design, marketing, and brand strategist, the cocktail, a user experience and interaction design studio, cooper, a product design company, and OresundIT, a non-profit network.

Don Norman, the author of books such as "Design of Everyday Things" and "The Design of Future Things", presented at this conference. Don Norman is one of the founding fathers of the Human-Computer Interaction and related fields, and is the co-founder of the Nielsen Norman Group, a consultant firm that helps company create human-centered products.

Videos of Don Norman's Presentations:
Emotional Design: Total User Experience
Cautious Cars and Cantankerous Kitchens

Other:
Business to Buttons 2008 Recorded Sessions
Business to Buttons 2008 Downloads

My posts about the work of NUI Group members:

Multi-Touch Plug-in for NASA World Wind?!

More Multitouch: NUI Group's Christopher Jette's multi-touch work featured in Engaget ; Croquet?

More Multi-Touch from members of the NUI group!

Multi-touch Crayon Physics from multitouch-barcelona, inspired by Crayon Physics by Kloonig Games

Cross Post: Seth Sandler's YouTube Video, "How to Make a Cheap Multi-touch Pad" goes viral

NUI-Group Member Bridger Maxwell Receives High School Science Fair Award for Multi-Touch Screen Project

Look, touch, listen, and play: Seth Sandler's interactive Audio Touch Table video; NUI Group and Google's Summer of Code