-Stephane Chatty, Benjamin Tissoires; Video by G. Tabart
Information from the YouTube video: "This video, shot by ENAC's Interactive Computing Laboratory, demonstrates how one can use out-of-the-box Linux applications with multitouch displays, using the lab's kernel drivers and modified X.org evdev driver. See http://lii-enac.fr/en/projects/shareit/xorg.html for technical details. This work was done during project ShareIT, carried out with Thales Avionics, IntuiLab and Stantum, and sponsored by Aerospace Valley. The ShareIT project explores the use of multitouch interaction for future commercial aircraft cockpits."
The multi-touch software shown in the video was created by IntuiLab. The Slate PC multi-touch hardware was built by Stantum. Later in the video, the software is shown running on a MERL DiamondTouch
Globacore Globacore used four 72" Blue Ocean Low Gain Screens 16:9 to create their twenty-foot multi-touch wall, which I think was also used for the "holographic" screen in the first video. The tracker and base flash API from the NUI-group were used in this system. I wasn't sure if I'd heard about Blue Ocean screens. Much to my surprise, the company behind Blue Ocean is Nippura, located in Japan. Nippura has another office in Charlotte, N.C. For years, Nippura's core business has been focused on creating "the world's largest acrylic panels, tunnels, cylinders and spheres for the professional aquarium industry", but is also focused on screen systems as well.
US Nippura/Blue Ocean Projection Screens Blue Ocean screens were use in the second videos of this post. Below is some information the screens from the company's website: "Blue Ocean® Projection Screens are the first ever to cast the screen into a hi-tolerance viewing plane centered between two ultra-clear panels of cell-cast acrylic, the same type of material used in hi-optic fighter jet canopies and the world’s largest aquariums. Blue Ocean® Projection Screens also offer something never before found in screens: self-rigidity. The thickness of each screen can be fabricated so that it maintains its form without external frames or tension systems. Frameless high definition projected images without borders amplify the infinite depth viewing experience. Because of its durability, outdoor applications of entertainment around the garden and pool are also possible. Normal wear and tear scratching has no long-term effect on the screens’ performance, and a ten year old screen, simply re-polished, will literally look and perform exactly as it did the day it left the factory." Super High Definition Stereo Image using 4K Projection Systems (pdf) Tetsuro Ogi, Hiroaki Daigo, So Sato, Yoshisuke Tateyama, Yasuaki Nishida, International Conference on Artificial Reality and Telexistence 2008
NUITEQ NUITEQ offers single and duo-touch plugins for Snowflake Suite that enables the software to run on NEXIO and NextWindow screens, including NextWindow's screens that are integrated in the HP TouchSmart and Dell Inspiron all-in-one PC's and others. SnowFlake Suite is also compatable with 3M Touch Systems, N-Trig, and Lumino. You can view the SnowFlake Suite application in action on NUITEQ's YouTube channel. "NUITEQ's software product Snowflake Suite, which received a Stevie Awards finalist recognition for Best New Product or Service of the Year 2009 - Media & Entertainment, is available to end clients, software developers, system integrators, VAR's and OEM's. Snowflake operates on a wide variety of hardware technology platforms including rear camera based solutions, IR overlay systems, bending wave, surface acoustic wave (SAW), projected and surface capacitive, resistive and other touch technologies for small hand held mobile devices and larger scale interactive displays." Recent news: NUITEQ Assists LTU Skelleftea at SACO event with new educational multi-touch desk
NextWindow and All-In-One PC's: NextWindow, the company that made the large single-touch display I used for a couple of class projects nearly three years ago, has really grown over the past few years. NextWindow provides optical touch screens for the HP TouchSmart PC, and also for the Dell Inspiron Studio One All-In-One PC, and the Dell Sx2210T desktop monitor. The next Dell with a NextWindow touch screen will be the Inspirion One 19, an all-in-one PC. You can find NextWindow touch screens in the Medion X9613, the NEC ValueStar W, and the Sony L Series. (NextWindow has Windows 7 certification for touch screens)
STANTUM and GUNZE USA Stantum has partnered with Gunze USA, a manufacturer of resistive touch-panels, to co-design and deliver multi-touch products. "Gunze USA will provide these custom multi-touch modules to the North American market for use in industrial, automotive, defense, aerospace, medical, kiosk/point-of-sales, and other professional applications. Unlike capacitive multi-touch, Stantum’s technology is particularly well suited to these professional markets, thanks to its:
immunity to EMI issues, via easy coexistence with Wi-Fi or Bluetooth antennas around the screen;
stylus input and high precision, enabling handwriting recognition; and
ability to work flawlessly with gloves."
According to the press release, Stantum's multi-touch technologies provide advantages such as handling 10 or more touches at a time, finger and stylus input, finger-pressure consumption, and native Windows 7 support, with no need for a driver. Developers can order the "beta" version of the Slate PC, based on the Dell Inspiron Mini netbook platform, from the Stantum website.
GESTURETEK The video below is of an interactive 3D application created for the Gondwana Das Praehistorium in Schiffweler, Germany, and provides an simulation of what it might be like to be a flying ancient pterosaur of giant dragonfly. Two gamers can play at the same time. If you want to fly, you flap your arms, and navigate by leaning. You crouch to dive, and increase your speed by flapping your arms even faster: GestureTek's 3D Interactive Flying and Virtual Game Experience for Gonwana Museum
This company always has something going on that looks fun and interactive. To get an idea this company's interactive work, take a look at the information on the GestureTek website links below. GestureTek has a variety product videos on the website.
SNIBBE INTERACTIVE Snibbe Interactive was founded by Scott Snibbe to develop social immersive media as a communication medium that incorporates the language of cinema and engages participation and interaction with others (and technology) through unencumbered body movements.
Scott Snibbe's Social Immersive Media presentation at Stanford's Seminar on People, Computers and Design, 5/2009
'Scott Snibbe of Sona Research discusses a distinct form of augmented reality focused on social interaction: social immersive media. He discusses design principles and interaction techniques to create strong emotional responses and social engagement through visceral interaction. He also describes approaches to clearly communicate cultural and scientific ideas through the medium, and how to promote specific distinct social behavior in uses." -YouTube description
RELATED Globacore "Globacore was founded by Ben Unsworth and Jeff Williams out of a passion for technology, art, and business. Globacore began as a web hosting and development company supporting the the technical needs of sports apparel startup VC Ultimate and it's associated companies."
"Globacore Inc. was officially incorporated in April 2003. In 2007 Ben and Jeff were joined by programmer Kyle Davis and graphic designer (and trail biker) Matt Venhuizen. In 2008, the Flexmaster Lukasz Ruminski joined us and continues to be a huge member of our team.Update: We'd like to welcome our newest team member: Developer Dave Rapin!"
Globacore Interactive Technologies 67 Mowat Ave. Suite 431 Toronto, ON M6K 3E3 Phone: + 1 (416) 925-2082 Tollfree: + 1 (888) 309-7658 Fax: + 1 (647) 477-3270 General Email:info@globacore.com
Natural User Interface Asia Pte Ltd(Asia office)
12 Little Road
#06-01 Lian Cheong Industrial Building
Singapore 536986
Phone: +65 6748 5245
Fax: +65 6748 7961
FINGERTAPPS Contact: James Bell-Booth
james[at]fingertapps.com
+64 21 226 6399 www.fingertapps.com/
GUNZE USA Product Overview Contact:Gunze Electronics USA 2113 Wells Branch Parkway, #5400 Austin, Texas 78728 Tel: 512-990-3400 Fax: 512-252-1181
LM3LABS Website: http://lm3labs.com/ENG/index_eng.html info@lm3labs.com Europe: Antipolis Innovation Campus 300, route des Cretes 06560 Sophia-Antipolis France Hong Kong, China +852-8198.8406 Australia +61(02)8011.3342 USA: +1(646)461.6146 TGarden Square 409 3-5-26 Funabori Edogawa-ku 134-0091 Tokyo, Japan +81(0)3.3877.0779
Outdoor Media Company (China) Products Contact: Kuo Lin (?) 42x46 Multi-touch LCD Display
HIWAD TOUCH - Beijing Yhjn Science and Technology Co. LTD
Interactive Sandbox "An interactive sand box or sand table for use in tactical training includes an automated control system and image generator so that infinitely variable visual images may be projected onto the contoured surface of the sand for training and preparation for action in the military field and disaster control, industrial planning, environmental protection, and similar situation."
Interactive Holographic Touch Screen
Beijing Yihejianing Science and Technology Co., Ltd
Address: Room 621, Jinyuan Modern shopping Mall, NO.1 Yuanda Road, Haidian District, Beijing, China
TE:8610-88892897，88892895
FAX:8610-88892867-801
Email: hxrong2001@hotmail.com http://www.cnhci.cn
A study in January of 2005 by Nokia indicated that there was a demand for interactive mobile multimedia services. That study was conducted about five years ago, before anyone had heard of the iPhone! The technology to support interactive mobile multimedia has come a long way since then, and many of the new applications support multi-touch, or at least duo touch interactivity.
I'm very much interested in figuring out how to design web-based interactive content (and apps) that can be optimized for touch (and multi-touch/gesture) screens of various sizes, from SmartPhones/iPhones to the large interactive whiteboards that are now in a multitude of classrooms.
From my experience as a school psychologist, I know that there are many teens who have graduated from traditional cell phone to the next level. If they don't have an iPhone, they have a smart phone. I don't have the statistics on this, but my personal observations tell me that there are teens who come from families who are from lower economic status who are somehow able to own 3G smartphones.
What a great opportunity to provide casual interactive multimedia educational games to support student learning! The games and activities could be assigned as homework from time to time, and with the appropriate LMS (Learning Management System), the teacher would have instant access to student progress. In addition, the students would be provided with immediate feedback about their "work", which we know is an important factor in learning.
It is difficult to figure out the best path to forge, since nearly every week someone announces a new platform, technology, and programming approach!
At any rate, here are a few interesting things related to this topic that I'd like to share. Many of these concepts are in the experimental phase, but are worth some attention.
SciLor's Comments: I have manged it to get Multitouch running in an vb.net app :)
There are still some bugs, which have to be resolved:
-Stop auto alignment!
-Identify the "Touches"
Google I/O 2009: Mastering the Android Media Framework
AT&T Interactive Mobile Website
Satellite-Terrestrial Network Delivering Mobile Video with Interactive Services - ICO mim
Fantasy HCI! My wish is to have my own lab so I can create and test out various interactive applications that run on screens of all sizes, and play with new interactive gadgets and displays. I'd also like to provide mobile lab services so I can go out and see how emerging technologies play out in real-life situations and settings during the design & development process as well as after-market. I'd like to focus on social-collaborative & cognitive aspects of emerging technologies. Because of my background in school psychology, I'd work towards ensuring that new applications, technologies, and systems follow the guidelines of Universal Design for Learning as well as Universal Usability. I have some ideas about the transdisciplinary characteristics I'd like to see for members of the lab's Dream Team, but I'm saving that for another post. Now I just need to win the lottery so I can hire my team and run with the ball. Team Charlotte, N.C., anyone? FYI: The HCI link is to a blog that corresponds to the Theory and Research in Human Computer Interaction class at Rensselaer Polytechnic Institute. For more information about HCI, visit the Human-Computer Interaction Resources website.
The video is is of the Stantum Slate PC, via Netbooked's YouTube channel...The system in the video is running on a modded Dell Mini 10, and doesn't require calibration. Notice how the system easily handles a variety of interaction- fingers, thumbs, pinch, rotation, multiple finger swipes, brush strokes, fingernail action, stylus, and more.
At this time, the Stantum Slate is available for developers only.
Sharath Patali, a member of the NUI-Group, has been working with Python Multitouch, otherwise known as PyMT, to create multi-touch applications. He shared a link to a recent post in Make, featuring PyMT. Sharath is the author of the UI Addict blog, and is currently doing his internship at NUITEQ (Natural User Interface Technologies).
I've been told that the beauty of PyMT is that it makes it "easy" to create multi-touch prototype applications using very few lines of code, which is great for trying out different ideas in a short period of time. It helps if you already know Python!
"PyMT is a python module for developing multi-touch enabled media rich applications. Currently the aim is to allow for quick and easy interaction design and rapid prototype development. PyMT is written in Python, based on pyglet toolkit."
Note:
Christopher, author of The Space Station blog, is a member of the NUI-Group, and is building his own multi-touch table running his PyMT-based applications. Christopher is a student in Koblenz, Germany, studying computational visualistics, known as information visualization in the US.
Medtronic is a medical technologies company that has been around for many years. Some of their projects include the Activa deep brain stimulation therapy used to treat Parkinson's disease and other conditions, pain management therapies, and more.
SOMEWHAT RELATED
The following is a video featuring Elastic Creative's work with Colardo Creative for Immersion, a company that creates systems and technology that integrates touch feedback into products, including medical simulators used in training physicians for minimally invasive surgical techniques.
Photo of MuchoWall, an 80" multimodal sensitive wall from Tangible Displays (Jimmy Hertz, Founder)
Jimmy Hertz is a member of the NUI-Group and has been involved in various activities to spread the world about natural-user interfaces and interaction.
Be sure to watch the entire video. It is almost like watching a dance performance when you watch artist interact with the display.
For techies and the tech-curious who like technologies that support collaboration and multi-touch interaction, this is great news!
Florian Echtler announced the first stable releas of libTISCH, a multi-touch development framwork, which can be found on Sourceforge. TISCH stands for Tangible Interaction Surfaces for Collaboration between Humans. libTISCH, a C++ software framework, is included in this project. It provides a means for creating GUIs based on multi-touch and/or tangible input devices.
Here is how it works:
Here is information from libTISCH announcement:
Highlights of this release are, among others, the following features:
- ready-to-use multitouch widgets based on OpenGL
- reconfigurable, hardware-independent gesture recognition engine
- support for widely used (move, scale, rotate..), pre-defined gestures
as well as custom-defined gestures
- hardware drivers for FTIR, DI, Wiimote, DiamondTouch..
- TUIO converters: source and sink
- cross-platform: Linux, MacOS X, Windows (32 and 64 bit)
- cross-language: C++ with bindings for C#, Java, Python
libTISCH has a lot to offer for the multitouch developer. For example, the textured widgets enable rapid development of applications for many
kinds of multi-touch or tangible interfaces. The separate gesture
recognition engine allows the translation of a wide range of highly
configurable gestures into pre-defined or custom events which are then
acted on by the widgets. While the lower layers of libTISCH provide
functionality similar to tbeta, touche etc. (you can interface existing
TUIO-based software with libTISCH in both directions), it goes far
beyond.
I'm not sure what I think about ubiquitous Flash 10.1. and Adobe's Open Screen project. I like the idea of anything that is seamlessly cross-platform, but I shudder to think that this might let out a wave unwanted or annoying "push" advertising on on-the-go screens of all sizes. I'm assuming web developers, along with TV ad producers, will be jumping on this train without fully thinking about how their applications and designs will play out in the off-the-desktop, digital-out-of home world.
I decided to take a look, drill down through the hype, and share a few links related to this topic.
Adobe Pushes for a Flash-ier Mobile Web Rob Pegararo, Faster Forward, Washington Post (10/5/09) "Are you anxious to bring Flash to the mobile Web, even if it means being subjected to some over-eager Web coder's song-and-dance routine? Or would you rather do without it on the go, even if that means having to switch to a "real" computer to use some Web sites' features?"
Hopefully the "over-eager web coders" will heed the MEX Manifesto:
MEX: Mobile User Experience 2009 Manifesto (pdf) "The Manifesto sets out our beliefs as to how user-centred design principles can enhance the experience of multi-platform digital services."
A framework for user journeys in a multi-platform world: Marek Poawlowski, founder of MEX
"User experiences are evolving into increasingly complex sets of interactions between multiple devices. In this video presentation, Marek Pawlowski of the MEX Mobile User Experience strategy forum, shows how a framework can be used to map user journeys through the multi-platform environment."
"Unencumbered by wires, information is flowing into every corner of our world at an ever increasing rate and through an ever increasing range of digital platforms. The single greatest challenge facing digital industries is understanding how this explosion of data will be woven into the fabric of consumers' lives."-- Marek Pawlowski, founder of MEX.
"The Open Screen Project is an industry-wide initiative, led by Adobeand backed by other industry leaders who all share one clear vision: Enable consumers to engage with rich Internet experiences seamlessly across any device, anywhere. Partners in the Open Screen Project are working together to provide a consistent runtime environment for open web browsing and standalone applications — taking advantage of Adobe® Flash® Player and, in the future, Adobe® AIR®. This consistent runtime environment will remove barriers to publishing content and applications across desktops, mobile phones, televisions, and other consumer electronics." Learn more
Reinventing Storytelling in the Digital Age Across Platforms, Across Screens
NAB 2009 presentation by Shantanu Narayen of Adobe and A.D. Albers, of Disney Interactive Media Group, fromNAB 2009 Adobe and NVIDIA Deliver Rich Web Experiences on Netbooks and Mobile Devices Reuters (10/5/009) "At Adobe MAX, Adobe's worldwide developer conference, Adobe Systems Incorporated and NVIDIA Corporation..announced that both companies are bringing uncompromised browsing of rich Web content to netbooks, smarphones and smartbooks built with NVIDIA GPUs. The companies have been working closely together as part of the Open Screen Project to optimize and dramatically improve performance of Flash Player 10.1 by taking advantage of GPU video and graphics acceleration on a wide range of mobile Internet devices. NVIDIA customers embracing Flash Player 10.1 for their new devices include HP, Lenovo, Samsung, Acer, Asus and more..."
Paramount Digital Entertainment Launches Interactive Thriller on MySpaceTracy Sedlow, InteractiveTV Today (10/28/09) "The company says that it can deliver the show's multiple interactive elements to viewers across devices using Adobe Flash Player and Adobe AIR, "because of efforts by the Open Screen Project, an industry-wide initiative led by Adobe and supported by PDE and close to 50 other industry leaders, to enable people to engage with rich Internet experiences across any device, anywhere." -
NUI-Group member Christian Bannister, a musician, designer, and developer behindSubcycle Labs has been experimenting with music and multi-touch technology and interactions. Here's a demonstration of what he's come up with so far:
multi-touch the storm - interactive sound visuals - subcycle labs from christian bannister on Vimeo. . "The big picture goal of this project is to bridge the gap between sound visualization and musical instrument. With multi-touch interaction it is possible to manipulate multiple characteristics of a sound—directly and visually. Right now a lot of electronic music involves staring at the back of the performers laptop. This is a shame because in many cases a lot of really interesting things are happening on the computer that the audience is completely unaware of. This project hopes to create a common visual language and experience for the electronic musician and the audience by enhancing the perception of sound and music on both sides...These sketches are built with Processing and Max/MSP networked with OSC on a single computer..." -Christian Bannister
JazzMutant Lemur The multi-touch tech company now known as Stantum evolved from JazzMutant, Jazzmutant was founded in 2002 by Guilluame Largillier and Pascal Joguet, and joined by Julien Olivier in 2003. The original focus was to create a multi-touch controller for music applications, and the Lemur was born. It is now in version 2, with features such as a gesture object that provides three ways for people to interact with sound, extended scripting abilities, and remote control of your computer's mouse cursor or keyboard. Stantum recently developed a next-gen multi-touch screen system for use in mobile devices such as smart phones and netbooks. (I'll discuss this further in a future post.)
Mapping Ableton Live to Jazz Mutant's multi-touch Lemur for sound design:
Additional tutorials can be found the Jazzmutant YouTube channel.
Ableton Live, the software used in the above video, will include Max. Max/MSP is now known as MAX 5: "In use for over twenty years by performers, composers, artists, scientists, teachers, and students, Max is the way to make your computer do things that reflect your individual ideas and dreams. Version 5 is a dramatic transformation of Max that begins a new chapter in the program's history."
I came a couple of interesting links to a couple of articles from the Putting People First blog. The links are articles written by Microsoft Research principal scientist, Bill Buxton. If you've never heard of Bill Buxton, he's the guy that was doing multi-touch research way back in the 1980's.
The Mad Dash Toward Touch Technology
Bill Buxton, Business Week, 10/21/09
"True innovators need to know as much about when, why, and how not to use trendy technology as when to use i."
The Long Nose of Innovation
Bill Buxton, Business Week, 1/2/08 "The bulk of innovation is low-amplitude and takes place over a long period. Companies should focus on refining existing technologies as much as on creation."
I came across Bill Buxton's Multi-Touch website in early 2007 when I was taking HCI and Ubicomp. I was searching for information about large touch-screen displays and applications for a couple of class projects. The website was the answer to my graduate student prayers. On the site, you'll find a fantastic overview of the history of "multi-touch", including gesture recognition and related surface technologies.
The website has interesting links. If you have the time, take a look at Buxton's main websitehttp://www.billbuxton.com/. You'll find loads of interesting links. I especially like the links to his Business Week articles.
