Ubiq'window, by LM3LABS, is a gesture-based system that is used for interactive show windows, interactive in-store marketing, museum installations, and more.
The slides provide details of the Ubiq'window's system specifications, including a gesture recognition set. The slides also higlight "Airstrike", a system that allows for free-air, touchless interaction.
The following video is a demonstration of "Sparsh", an interactive multi-touch FTIR table built in eight weeks by a group of engineering students in India. Most of the information regarding the hardware and software you see running on this low-cost system can be found on the open-source NUI-group website, forums, and wiki.
I'm in the process of creating an interactive timeline, and as I revisited my links and bookmarks, I came across a link to a video of Jonathan Harris discussing his ideas regarding digital storytelling, overlapping threads, and multiple perspectives.
Jonathan explores real-life stories and celebrates the interconnections between events, ideas, feelings, and people. Linear narrative and linear time lines do not do justice to the richness and complexities of human experience. "Combining elements of computer science, anthropology, visual art and storytelling, Jonathan Harris designs systems to explore and explain the human world."
"Jonathan Harris is redefining the idea of what it means to tell a story. Take a ride through an arctic whale hunt and plunge headfirst into the feelings Harris finds running rampant in cyberspace as he describes what he calls “storytelling platforms.” "
Below are links to two story-telling platforms described in the presentations. The Whale Hunt is organized so that the user can explore the story through a variety of perspectives and interfaces, and at different points in time.
The first screenshot shows how the user can select one of the "cast" members to see how the story unfolds from that person's point of view.
"Every few minutes, the system searches the world's newly posted blog entries for occurrences of the phrases "I feel" and "I am feeling". When it finds such a phrase, it records the full sentence, up to the period, and identifies the "feeling" expressed in that sentence (e.g. sad, happy, depressed, etc.). Because blogs are structured in largely standard ways, the age, gender, and geographical location of the author can often be extracted and saved along with the sentence, as can the local weather conditions at the time the sentence was written. All of this information is saved...The result is a database of several million human feelings, increasing by 15,000 - 20,000 new feelings per day. Using a series of playful interfaces, the feelings can be searched and sorted across a number of demographic slices, offering responses to specific questions..."
If you have some time on your hands, explore Jonathan's Universe project:
This photo depicts the nine stages of the Universe environment.
"Universe is a system that supports the exploration of personal mythology, allowing each of us to find our own constellations, based on our own interests and curiosities. Everyone's path through Universe is different, just as everyone's path through life is different. Using the metaphor of an interactive night sky, Universe presents an immersive environment for navigating the world's contemporary mythology, as found online in global news and information from Daylife. Universe opens with a color-shifting aurora borealis, at the center of which is a moon, and through which thousands of stars slowly move. Each star has a specific counterpart in the physical world — a news story, a quote, an image, a person, a company, a team, a place — and moving the cursor across the star field causes different stars to connect, forming constellations. Any constellation can be selected, making it the center of the universe, and sending everything else into its orbit."
Universe was created using Processing, which is an open-source software that is used by people from all sorts of disciplines to create interesting interactive information visualizations and more.The data used in Universe is fromDaylife. For more information about Daylife, visit the Daylife Labs.
Sidenote: Jonathan Harris collaborated with Sep Kamvar on the We Feel Fine project. Sep Kamvar teaches classes like "Social Software" and "Computational Methods in Data Mining" at Stanford University. Sep is part of the Stanford Human Computer Interaction (HCI) Group.
Stanford's HCI group's weekly seminars highlight a variety of interesting speakers. Current and previous talks are available via Stanford OnLine. You can link to current presentations and videos from the Human Computer Interaction Seminar website. If you are curious, past presentations/abstracts can be accessed on-line alphabetically or by date, going as far back as 1990.
Dan Saffer, author of Designing Gestural Interfaces, presented "Tap is the New Click" on January 23rd, 2009 at Stanford. You can access the video directly.
The picture below was posted on the Shakeout blog, and shows children gathered around Microsoft's Surface at the recent BETT 2009 educational technology conference in the UK. Read "Interesting tidbits from BETT 2009" for more information.
I'm preparing myself to explore the multi-touch potential of my HP TouchSmart PC with the beta version of Win7, Microsoft's newest operating system. While I was searching for information, I came across this cute video of a baby interacting with the touch-screen. The dad in the video is software developer Kurt Brockett.
I have lots of ideas for touch screen interaction applications for kids of all ages. Please leave a comment if you have a TouchSmart and working with Win7, or plan to do so in the future.
This video showcases the work of Natural User Interface-AB, using NUI Suite 1.0 Snowflake and Flash.
Here is the plug from the company's website: "Natural User Interface (NUI) is a Swedish innovative emerging technology company specializing in commercially available advanced multi-touch software, hardware and service solutions. NUI's solutions can convert an ordinary surface into an interactive, appealing and intelligent display that creates a stunning user experience."
With Win7's "multi-touch" capabilities, Bill Gate's statement that every surface will be a computer might come true, but from what I can see, we have a long way to go before we get it right!
One of the concerns I have about "touch" screen interaction in public spaces is that not much thought is given to the environmental factors surrounding the system. The system that is tested for the first time after installation might work just fine, but not when it is put to use under a variety of conditions or situations.
EXAMPLES(I've recycled a few of these from previous posts.)
Touch Screen Kiosk/Display in Upscale Shopping Area, St. Thomas(?)
I came across this system when I was walking around in St. Thomas with my husband. I wasn't sure what to make of it. There wasn't anything on the large display, and I couldn't tell from a short distance if the two smaller displays were active.I noticed that the display had an awning of sorts, so the fact that the display would be used in a sunny location had been considered. Unfortunately, the awning didn't solve this problem, as you can see from the pictures below:
CLOSER UP
As we approached the display, it was almost impossible to see what was on the screen. In fact, we weren't sure what the display was all about. It looked like something you'd see at an airport check-in, not in a lush tropical setting.
Below is a shot of my husband, trying to see what is on the screen. This photo shows exactly what the screen looked like at the time:
The next picture tells it all. You can see that there is some sort of alphabet on the right side of the screen, but the reflection of my husband is easier to see. We couldn't figure this one out.
The following clip demonstrates how environmental concerns, such as wind and "sideways" rain, can make an interactive display at a shopping center difficult to use. During this clip, you can hear me muttering about the difficulties I had interacting with the information on the touch-screen display. It is possible that the system wasn't working well because of the exposure to the elements.
WIND AND SIDEWAYS RAIN
You can see that the navigation system was confusing. At the end, the entire screen turned red, and I gave up.
The next video-clip is of a touch-screen map at the Cleveland Clinic hospital. I was with my elderly aunt and my sister, and we were trying to figure out how to get to a restaurant. We never found it.
Although the screen was easy to see, the application was... crazy!
In my opinion, an interactive wayfinding application should run on a larger display and support two or more people, since many hospital visitors often run in pairs or small groups.
BETT is the annual ICT & educational technology conference held in the UK. The UK has the highest number of classrooms in the world with interactive whiteboards, which has been an interesting transformation to follow over the past few years.
I've posted several video clips from BETT '09, which was held this month (January), along with some other resources. Tabletop computing applications for education were demonstrated by Microsoft Surface and Smart Technologies. Take a look!
BETT 2009 Video Overview
Microsoft Surface at BETT 2009
More Surface for Education: User Interface and Paint
Physics and Social Studies
Orientation and Images
Science: Medical and Health Care
SMARTTable at BETT 09 (Note: The two clips below look as if they were taken with a cell phone video camera. I'll post higher-quality videos if I find them.)
SMART Technologies PR video
I think there is a need for more application development in this area!
RELATED The following two clips are from the visitor's point of view, overwhelmed by it all...
Kloonigame's Crayon Physics Deluxe is a fun, interactive physics game that works well on a touch screen. It is not as complex as Algodoo, as it doesn't require the player to interact with lots of settings.
The above video is of the commercial version of the game. It is a labor of love by Petri Purho, of Helsinki, Finland. You can find Petri's games on the Kloonigame's website, including the free version. Crayon Physics won an award at the 2008 Independent Games Festival.
Algoryx, a software company, has announced an educational version of Phun Physics, called "Algodoo". It is on display Classmate PC at BETT 2009 in London this week (January 14-17).
"Algodoo is a 2D-simulation environment for creating interactive scenes in a playful, cartoony manner, making use of the physics that we use to explain our real world. Algodoo is designed to encourage young people’s own creativity, ability and motivation to construct knowledge. The synergy of science and art makes Algodoo as educational as it is entertaining. Algodoo applies a constructionistic learning paradigm - learning by designing, constructing and exploring physical systems."
"Algodoo has been optimized for the Intel-powered convertible classmate PC and makes innovative use of many of its features including the webcam for creating physical models from live photos, the touch screen and digital pen input for drawing objects, and the accelerometer for virtual experiments and very direct interaction with the physical simulation. We are quite amazed by the performance, features and look and feel of this little notebook. Learn more about it here: http://www.classmatepc.com/"
The HP TouchSmart Notes application can be used in a variety of interesting ways. I have an HP TouchSmart PC, and I like how easy it is to slide the notes about the screen using this multi-modal application. It allows the user to take capture pictures, video, or audio to create notes, as well as traditional "stickies" that allow for typing or finger-writing input. You can even draw or write over photos. It is an application that has many accessibility features. It also supports many of the principles of Universal Design for Learning (UDL).
The various notes can be put together in a variety of ways, and allow for video or photo storytelling and interesting ways of leaving messages- even a song or two. I did some hunting and found the more information about some non-verbal students with autism who are learning verbal skills through this technology at Hope Technology School in Palo Alto, California. Video of student using the HP TouchSmart Notes application at Hope Technology School:
Here are a few "How Two" videos that demonstrate the TouchSmart Notes features and interactions:
Microsoft's newest operating system, Win7 (beta), allows for multi-touch interaction and application development. From what I can tell from the news from the Consumer Electronic Show (CES 2009), touch and gesture interaction holds quite a bit of interest among consumers and application developers alike.
Hopefully people will think carefully about user experience and usability factors when developing the applications we'll be using in the not too distant future!
Because of my background as a school psychologist, I’m interested how touch/gesture applications can address accessibility and universal usability issues.
I wasn't able to locate user experience or usability information for Win7. Here are a few links from the Microsoft website that I think are worth reading:
Regarding touch/gesture interaction, know that “19% (24.4 million) of computer users have a mild dexterity difficulty or impairment, and 5% (6.8 million) of computer users have a severe dexterity difficulty or impairment.”
Christian Moore, from the NUI-Group, shared this nice graphic that depicts the various fields that intersect or converge with interface design, which I think is useful when thinking about how we will design, develop, and use emerging technologies:
Concepts such as accessibility and universal usability can fit into this design!
I received couple of interesting links about interactive multimedia applications from Anthony Uhrick, of NextWindow, who is attending CES (Consumer Electronics Show). (NextWindow is the company who produces large touch-screen displays that have duo and multi-touch capabilities.)
Kevin Kennedy and his team at InterKnowlogy partnered with Zygote 3D Human Anatomy and Intermountain Health Care to develop a health care application developed in Windows Presentation Foundation to run on Microsoft's multi-touch Surface computing table.
The application supports collaboration between health care professionals and could also support collaboration between patients and doctors as well. Aspects of the application could be useful for patient education.
I really liked the part that demonstrates how you can zoom deeply into the 3D heart and look at things from various angles.
The above video demos an application that might be useful for teaching history with an interesting timeline interface dial.
Visit InterKnowlogy for more videos and information about what they are doing with Windows Presentation & Silverlight as partners with Microsoft's Surface team.
If you are interested in more 3D anatomy, visit Zygote's 3D Human Anatomy site and 3D Science. If you are an educator, you'll see that 3D interaction has potential for creating more engaging science and health education lessons!
Another interesting link is to TouchTV Networks, which has partnered with companies such as Vectorform, who are also working with multi-touch applications using Windows Presentation Foundation.
Video from TouchTV Networks of CES 2009 Demo:
Vectorform's Virtual Drum Kit application:
Vectorform's Surface at School - demonstrates how this can be used in a classroom:
There are plenty of people reporting and blogging about CES 2009, and I won't have time to digest it all for a while. I've embeded the CNET overview video below, which includes a demonstration of some interesting interactive technologies and applications.
If you want the latest scoop, your first stop should be CNET's CES 2009 website. It is almost as good as being there, and you won't have to deal with the crowds.
If you are a new visitor to this blog and interested in interactive multimedia, you'll need to know more about the NUI-Group. Natural User Interface or ~ NUI Group is an interactive media group researching and creating open source machine sensing techniques to benefit artistic and educational applications. (For related information, please read my recent post, Usability, Accessibility, and User Experience in a Win7 Environment.)
Seth Sandler, of the NUI-Group, sent out a great email with links and resources for people who are interested in multi-touch/gesture interaction, hardware, and/or software development. The list of NUI-Group members who have completed projects is listed below, with links to project websites as well as related threads on the NUI-Group forum.
(The information can be found on the NUI-Group Wiki, which boasts a nice icon based front-page)
Thanks, Seth, for organizing this wealth of information!
Joyce Santos was kind enough to post the lyrics to the rap, which I've posted below. The rap is was inspired by Harvard professor Greg Mankiw's ten principles of economics. These principles can be downloaded from the following link: Chapter 1: Ten Principles of Economics (pdf) Principles of Economics, Fourth Edition N.Greg Mankiw Lyrics to Demand, Supply Rap, by Rhythm, Rhyme, Results Via Joyce Santos
This is a great video of Nicholas Rubin's Physical Computing project, in which he built a volumetric interface and projection surface. Rubin is a graduate student at NYU's Interactive Telecommunications program.
The following quote from the video was posted by Doug Fox, of Great Dance/Kinetic Interface: "In building this object I used as my theoretical foundation a practice called "glowsticking" that exists predominantly in rave and underground electronic music culture. Glowstickers, at least the talented ones, have cultivated the unique ability to twirl their glowsticks in mid-air and actually generate continuous, dynamic, 3D imagery in space. This is then observed by other party-goers whom might be near them. It usually happens spontaneously on the dance floor as opposed to being strictly a performance activity."
Doug Fox, of the Kinetic Interface blog (part of Great Dance), shared IKEA's new promotional site that allows for tons of multimedia interaction. The website features a series of rooms to highlight various products.
Visit Come into the Closet! IKEA Interactive Multimedia Game Unlike most static, or even flash-based websites, the user has control over many dimensions of experience:
Simply watch the dancers and listen to the music, navigating from room to room
Control the dancers by tapping on your computer's keyboard
Control the dancers by clapping, singing, or making other noises
Combine tapping your keys while watching and listening
Here is a video preview of some of the dance interaction in the site:
Duncan, of the Inspiration Room, also posted credit information regarding the IKEA website:
The following credits were posted by Duncan on the Inspiration Room Daily:
"The site was developed at Forsman Bodenfors, Sweden by web director Mathias Appelblad, copywriter Fredrik Jansson, art director Anders Eklind, designers Anton Robsarve and Christoffer Persson, agency producers Peter Gaudiano, Charlotte Most, Asa Jansson, account director Philip Mascher."
"An earlier version of the site was developed in 2006, winning a Yellow Pencil at the DandDs, and can be viewed online at Forsman Bodenfors."
"Web development was done at Kokokaka, Gothenburg."
"Filming was shot by director Amir Chamdin via Chamdin & Stöhr, Stockholm, with director of photography Gosta Reiland."
Note: Other pictures from the Phenom site gave a message, "Sorry, Our Photos are Copyrighted", so this is the only photo I was able to obtain. You can see more photos on Phenom's online photo-gallery.
I might have found the ultimate HCI - ubicomp research tool!The Phenom Watch-Phone. It might also come in handy when I'm conducting observations "across settings" in my job as a school psychologist. (Maybe Phenom will give me one for free to test out for a while...I'd be happy to develop some apps for it if it works out for me.)
I was delighted with some of the user-friendly marketing features on the Phenom website. I didn't have to dig and get lost and dig some more to find what I needed. The above videoclip is featured on the home page of the site, which gives gives a good info-tease about the advantages of the watch. The FAQ section is fairly extensive and easy to navigate.
When you explore to the "Gadget Freaks" page, you are provided with an audio presentation, with musical accompaniment,as you view pictures and prices. (You can turn the web page audio if you don't want to listen to the blurb, or turn it on and listen to it again, if you missed something.)
When you click on the picture of the "SpecialOPS Black" version of the watchphone, you are taken to another page where you can inspect different features more closely as you move your mouse around the photo. (Since I have an HP TouchSmart PC, I just moved my finger around the photo- a great effect!)
Here is the description of the SpecialOps phone, taken from the Phenom website:
"The ultimate watch phone for those who like to live on the edge. The SpecialOps is a fully functional GSM cell phone that has a touch screen and an external key pad. The SpecialOps has an MP3&MP4, built-in microphone and speakers, digital and video camera, MicroSD slot and built-in Bluetooth. You can even take notes with your convenient and compact stylus or record your thoughts on the run. See full list of features for more details."
More info from the Phenom website:
Features -External keypad -LCD: 1.3 inch TFT260k Pixels -Touch Screen -Language: English -Ring tone: 64 Polyphonic, Supported Formats: Mp3, MIDI, Wave -Music Format: MP4, Full Screen -Camera: 130 Pixels -T-Flash Supported -Built-in Bluetooth -Picture Format: JPG, GIF
Basic Functions -Notebook: 250 Groups -SMS and MMS Messaging -User-defined on-off switch -Game: Picture Mosaic -Other Function: MP3, MP4, Built-in Speakerphone, Group Messaging MMS, Call Barring
I meant to post about this a while ago- here is a group of people playing a relaxing version of "Stairway to Heaven" on their iPhones:
"How To" Video:
Picture via Brothersoft Editor's Blog: "- Blow into your microphone to generate music - Hold down combinations of the holes to change your pitch - Tilt your phone from front to back to change the vibrato depth - Tilt your phone from side to side to change the vibrato rate"
You can connect with other Ocarina players around the world via your iPhone. Video of Smule's Zephyr application:
Take some time and explore the other applications developed by Smule.
From the New York Times, photographed by Yana Paskova
"The Accenture Interactive Network recently installed a large interactive screen at Chicago's O'Hare International Airport. The plan is ultimately to create a network of interactive, wall-sized screens that deliver a variety of information via touch screen to thousands of users."
Video and Photos from TacTable
The following video shows some applications developed by TacTable. As you can see, many of the applications look like they'd be useful in public spaces, including airports and museums.
Below are some pictures from the TacTable website:
Where Did Language Come From? Liberty Science Center, N.J.
Accenture Welcome Wall, London, England Sprint Studio Digital Lounge Table Related: TacTable Contacts: USA: Henry Kaufman henry@tactable.com
"Clients include Sprint, Accenture, Cirque du Soleil, New York Museum of Modern Art (MoMA), Liberty Science Center, Detroit Institute of Art, Georgia Aquarium, London's Millenium Dome, Boston Museum of Science, Chicago Museum of Science and Industry, Chicago Mercantile Exchange, Warner Brothers, Discovery, Disney, British Telecom Retail, and many others."
