Focused on interactive multimedia and emerging technologies to enhance the lives of people as they collaborate, create, learn, work, and play.
Jul 30, 2008
Microsoft's Multi-Touch Sphere "Photo Globe"
I just had to post about this, even though this news has been rapidly circulating around the blogSPHERE.
The sphere could support an interactive travel planning/travel memory application.
Imagine if you were on a cruise ship, and uploaded your photos to the globe, and voila, they'd show up on the sphere in your mother's living room! Geo-tagged, cross-referenced, synched with your 2.0 apps, and linked to your vacation video-clips you previously uploaded to YouTube.
Jul 18, 2008
Natural User Interface: Overview of multi-touch technology and application development by Harry van der Veen,- Business to Buttons

Harry van der Veen from Natural User Interface Europe AB, was one of the keynote speakers at the Business to Buttons: Designing for Effect conference, held in June 2008. In this presentation video, Harry discusses the past, present, and future of multi-touch technology, and reviews the importance of multi-touch over single touch displays. He also provides a good overview of gesture interaction, something that he researched when he was a student. This presentation includes several video examples of multi-touch applications in action.
The presentation is well worth the 30-minute view!
"Harry van der Veen is a Bachelor of Multimedia, derived from the Dutch education Communication, Multimedia and Design, focused on Interaction Design and Project Management. He is CEO, co-founder and co-owner of the Sweden based commercial company Natural User Interface Europe AB, which focuses on delivering standardized and customized multi-touch hardware / software solutions and services to the global market. In addition to that, he co-founded the NUIGroup community, which is the worlds largest online platform where a global network of people share their ideas and information in an open source community, focused on multi-touch hardware and software solutions."
NUIGroup Community

Harry van der Veen's blog
Natural User Interface Europe AB (Harry van der Veen's company)
NUIGroup Wiki: This wiki includes tutorials for developing multi-touch applications, building your own low-cost multi-touch table, and information about current projects that are in progress.
Related Information:
The Business to Buttons: Designing for Effect conference was held on June 12-12 in Malmo, Sweden, organized by Malmo University and inUse, a user experience consultancy. Partners in this conference included Adaptive Path, a product experience strategy and design company, Patrick W. Jordan, a design, marketing, and brand strategist, the cocktail, a user experience and interaction design studio, cooper, a product design company, and OresundIT, a non-profit network.
Don Norman, the author of books such as "Design of Everyday Things" and "The Design of Future Things", presented at this conference. Don Norman is one of the founding fathers of the Human-Computer Interaction and related fields, and is the co-founder of the Nielsen Norman Group, a consultant firm that helps company create human-centered products.
Videos of Don Norman's Presentations:
Emotional Design: Total User Experience
Cautious Cars and Cantankerous Kitchens
Other:
Business to Buttons 2008 Recorded Sessions
Business to Buttons 2008 Downloads
My posts about the work of NUI Group members:
Multi-Touch Plug-in for NASA World Wind?!
More Multitouch: NUI Group's Christopher Jette's multi-touch work featured in Engaget ; Croquet?
Multi-touch Crayon Physics from multitouch-barcelona, inspired by Crayon Physics by Kloonig Games
Cross Post: Seth Sandler's YouTube Video, "How to Make a Cheap Multi-touch Pad" goes viral
NUI-Group Member Bridger Maxwell Receives High School Science Fair Award for Multi-Touch Screen Project
Look, touch, listen, and play: Seth Sandler's interactive Audio Touch Table video; NUI Group and Google's Summer of Code![[nuiab.jpg]](http://2.bp.blogspot.com/_m-4BFRu70T0/SDM4qqFGMtI/AAAAAAAAAUM/czAZW-KlQHI/s1600/nuiab.jpg)
Jul 15, 2008
More Multi-touch: Dell's Latitude XT tablet now comes with a multi-touch upgrade!
Dell Propels Laptop Innovation with First Multi-touch Convertable Tablet, Larger Solid State Drive
A word of caution about the following video demonstrating the mutli-touch capabilities of Dell's Lattitude XT. The guys presenting aren't related to Jeff Hans. You won't see the "Wow" factor that creative multi-touch applications can provide.
Photo from NotebookReview.com:

"The new capabilities include:
"The Latitude XT is designed with the future in mind with the capability to support multi-touch. As the technology matures, the system can serve as a premiere development platform for applications that can take advantage of multi-touch." “By integrating our DuoSense technology into the Latitude XT, Dell creates the ultimate machine destined to lead the tablet PC market,” said Amihai Ben-David, CEO of N-trig. “Customers using the new Latitude XT will find that the zero pressure touch combined with the accurate stylus changes the usability of tablet PCs. Once they try it, they won’t want to go back.” The multi-touch gesture upgrade should be available by Tuesday, July 15th, 2008, at http://support.dell.com.
|
V12's Dual-Touch-screen Canova: Multi-touch could support applications for universal design, education, and assistive technology.
Canova Generation 1#

(Photo via Gizmodo; depicts the first version of the dual LCD laptop.)
According to information from LaptopMag, Conova was..The first-generation model....was envisioned to transform from a sketch pad, to a writing pad with an electronic pen, to a newspaper."

From the Gizmodo AU website:
"An interview with Valerio Cometti, the founder and managing director of V12 Design, confirmed that the upcoming version would support multitouch input and that a microphone would be built into the design for voice commands."
With multi-touch and audio input, this device would be a godsend for young people with learning difficulties and others who require assistive technology. It would also be very useful for people in various fields of work. I assume that the Canova will come with speakers, a DVD player, a video camera, Blue-tooth and Wi-Fi capabilities.
I was excited about the new dual-touch screen laptops in development for the One Laptop Per Child project. I'm even more excited about the Canova. Hopefully, the company is devoting attention to usability studies with a wide range of people, using a variety of interactive demo applications.
I wonder if I can get my hands on a second-generation prototype of Canova to test out my touch-screen application "experiments". I'm bursting with ideas to try out, including gaming applications.
I want one now!
The second version will be developed by a US manufacturer. If anyone knows more about this leave a comment!
Related:
V12 Designs' Dual-Touchscreen Notebook Coming Within Two Years
V12 DEsign Dual-Touchscreen Notebook will make ebook readers drool
V12 Design Delivering Dual Touchscreen Laptop Within Two Years
Jun 19, 2008
Hands-On Computing: InfiniTouch's Innovation
If you are interested in touch interaction, it is worth exploring!
Jun 15, 2008
DigiBoard Multi-Touch Mixed Reality Game; Ideas for future design of a flexible, adjustable multi-touch surface..
This video is a demo of a multi-touch table, called the DigiBoard, running a game application called "Tower Defense". This multi-modal game takes input from physical tokens, touch, and provides auditory feedback.
According to the credits on the video-clip, the creators of this application were Andreas Hesel, Birna Run Olafsdottir, Dann Sandgreen, and Osk Hilmarsdottir. When I can track down more information about this group, I'll post it!
For more information about interactive display technologies, see Gizmo Watch's Top 15 Interactive Display Technologies. The list is full of photos and video clips of a variety of systems. It was created last year, so it might need some updating.
I am still looking for some examples of good multi-touch table games for use in education. I've created a few prototypes that are not fully fleshed out over the past year, but I haven't had the time or money to build the type of adjustable table that I want.
I'd like my table surface to adjust horizontally, vertically, and angles in-between, like a drafting table. I don't want my table to rely on a projector, since I don't want unnecessary occlusion. A rear-projection system would be heavy and bulky. My design is light and sleek, and it is mobile.
The advantage of an adjustable design is that it allows for flexibility in use, within a single environment, and across a variety of settings. It could work well in schools, libraries, museums, convention centers, exhibit halls, hospitals, visitor centers, airports, public transportation stations, board rooms, data centers, malls, supermarkets, community centers, coffee houses...just about anywhere.
I've written about this topic a few times. For more information, see "Emerging Interactive Technologies, Emerging Interactions, and Emerging Integrated Form Factors"
If you know of anyone who is creating an adjustable, multi-functional touch surface, please let me know.
I am playing around with ideas for a suite of multi-modal, adaptive game for use with young people who have suffered traumatic brain injuries, and others who have special needs. A stationary surface would not provide the flexibility my applications will require.
Jun 13, 2008
Fingertapps and NextWindow's Multi-Touch Firmware on the New HP Touchsmart PC,
Fingertapps is working on an application that utilizes NextWindow's multi-touch firmware, running on the second-generation HP Touchsmart PC. Fingertapps will be demonstrating their work in the NextWindow booth at Infocomm 2008 in Las Vegas, June 14-20.
Jun 6, 2008
Emerging Interactive Technologies, Emerging Interactions, and Emerging Integrated Form Factors
After browsing around Richard Bank's blogs, I found that he works for Microsoft Research. Like me, he maintains more than one blog, and he blogs about similar topics.
Here is a view into his world:
rb.trends: tracking future technology experience
rb.work: technology, design and research
rb.log: family, photography, architecture etc.
One of Richard's recent posts brought me back to my musings about interactive surfaces:
Via Richard Banks and Gizmodo: A Touch Screen Poker Table from Hard GamingIf it is possible to program an application to handle the metrics that support poker, then it is possible to program a multi-user application for work or education, and have this application integrated into any type display, according to need.
The form factor above would be useful for team meetings or collaborative project planning, especially during group decision-making or policy planning activities that involve data analysis or information visualization.
Could this concept be modified to fit the form of a kidney table for use in education? Think about it. We could combine the best features of surface computing with the best features of multi-user video games.

Rationale? Video-game applications can handle the metrics of multi-user interaction, which would be ideal for tracking student progress and interaction. Touch interaction would enable young children with an easy way to access the application- or all of the information that will be available on the multi-touch web of the future, without learning first to type.
(Touch and gesture interaction could help us move towards the implementation of the Internet of Things)
Could this concept be transformed for use by two people collaborating on an electronic drafting table?

If you are a NUI Group member, think about this!
Note: I came up with the phrase, "multi-touch web of the future" as I was typing this post. I guess I didn't invent it - I looked it up and found Andrew Foote's post, "The Multi-Touch Web"
I'm in.
Somewhat Related.......
More for the tech-curious:
Interactive data visualization in Second Life
Idle together "Technology blog oriented towards good design and impressive web applications"
Apple Developing Full Multi-Touch Macs - Apple Insider
More for the tech-savvy:
Multi-touch gestures in the Factor UI (Apple)- by Pestov. Includes code.
NSResponder Modifications: Swipe, Rotate, and Magnify, by Elliott Harris, a member of the NUI Group, I think.
Web Development Guidelines for iPhone (lots of good info, with video clips and code)
Web 3D Consortium
Unity 3D kit for the
Google's Android
Multi-touch Web Consortium (in my dreams)
Gesture Web Consortium (in my dreams)
New Interactive Technology: Tag Galaxy, Windows 7, PaperVision 3D, Microsoft Surface, Touch Screens, and Blog Posts Revisited


Steven Wood's Tag Galaxy
Via Papervision3D, Richard Banks, and Flowing Data
Vacation pictures on an interactive photo-globe - April 2007 The first day I learned about Microsoft Surface will be etched in my memory forever...
It is not because I'm a 100% Microsoft fan. It is because I'd been thinking about touch-screen interaction since my first encounter with an interactive whiteboard several years ago. At the time, I was involved with group counseling with middle school students, seated around a table.
What would happen if we took the whiteboard put it on a table?
As noted below, the demo projects worked best on a NextWindow Human Touch. Although the large display was only "single-touch", it provided excellent resolution and touch response. Since then, NextWindow has come out with a multi-touch display, which I've had a chance to see. It is not a table, but it has possibilities.
What would happen if we took a NextWindow multi-touch display and mounted it onto an adjustible drafting board? It could be used as a vertical display, a "surface" table, and a drafting board. The best thing is that this could support quite a bit of collaborative work between two or more people, as well as learning, creative-artistic, and gaming activities.
New things are on the horizon. Windows 7, will replace Vista in the future. Bill Gates has pronounced that "every surface will be a computer". Touch screen surfaces are finding homes on tablet PC's, cell phones, and the latest OLPC "$100.00" laptops for children.
I think there will be more possibilities for using touch-screen technology for education in the very near future.
Here is my post about Microsoft's Surface from last year, with some updates:
Microsoft Surface multi touch screen table - I wish I had one for my projects last semester!
Microsoft Research recently unveiled Microsoft Surface: http://microsoft.com/surface. This multi - touch table can be used for a variety of applications, as outlined in the video from CNET and YouTube below:
I'd like to work on applications for use on a touch-table to support students with special needs, especially those who have autism spectrum disorders.
.....Last semester, I worked on prototype applications for use on a touch-screen surface -here is a photo. We used a NextWindow Human Touch large-screen display, which provided great screen resolution and touch-response.

This application was part of a travel-planning prototype developed for a course in Human Computer Interaction. The application was demonstrated on a NextWindow Human Touch large screen display. Would it work on the iPhone?
Update: Examples from some of my other posts:
Below is another demo video-clip of a globe created in GoogleEarth using photo-overlays, with links to video clips that pop up on the screen. You can spin and rotate the globe at any size, and zoom into the pictures. The above photo and the video clip show the application on a NextWindow Human Touch large-screen display.
This application would be great on a touch-table or touch-table set up on a drafting board. Although it was designed for a travel-planning application, it would work well in educational settings in subjects such as geography.
Poetry Picture Share

This was my first attempt at a "poetry picture share" application. It was designed for eventual use on a multi-touch table. It was developed using JavaScript and Ajax. It could be accessed remotely so people in different places can move things around on the screen. The video shows how the application works on a large interactive touch-screen display.
I am planning on adapting this application for use with students with special needs, such as those who have autism or other communication disorders. (Note: I've used it with several students, with success. It still is a work in progress!)
Update:
Link to Papervision 3D: Press the picture to enter a 3-D interactive underwater world...
If you have tried PaperVision 3D or Tag Galaxy on an interactive whiteboard or touch screen display, please leave a comment and share your impressions.
May 28, 2008
Windows 7 Demo on a Dell laptop: More Multi Touch
Video: Multi-Touch in Windows 7
Via SoapBox, Greenbush Labsand CNET
If you are new to this blog and would like to learn more about multi-touch interaction and technology, enter a keyword in the search box.
Also take a look at the Technology Supported Human-World Interaction blog.
For multi-touch DYI, check out the NUI Group!
May 19, 2008
More Multi-Touch from members of the NUI group!
Here is a new video of a multi-touch creation by some of the members of the NUI group. Although this is a proof-of-concept example, it is fun to see how it is played out, using the little iPhone-like touch-pad widgets as a navigation tool for the large screen.
Read the "Multi-touch Goodness" article in Gizmodo of an interview with Christian Moore about this demo and his Lux open-source framework. (Christian is a colleague of Harry van der Veen, both members of the NUI group.)
Here is an excerpt from the interview:
"JD: Why Flash?
CM: Because it's fast to prototype in. However, the software is broken into several segments. One C++ application that tracks hands that talks to Flash... WPF... or another C++ app... and basically everything you can imagine. You can enable multitouch in any environment, like Cocoa."
High-resolution screen shots and additional information can be found on the nuiman website.
For my tech-minded readers:
I'm pretty sure that the C++ application that track hands and fingers in the video demo uses Touchlib, a library for creating multi-touch interaction. Touchlib can work with TUIO, a protocol for tabletop tangible user interfaces. Applications such as Flash and Processing support TUIO. For more information about TUIO, read "TUIO: A Protocol for Table-Top Tangible User Interfaces".
(Information from the NUI group website mentions that OpenCV, or Open Computer Vision Library, found on SourceForge, can support blog detection and tracking.)
The people behind TUIO are from the Reactable project, of the Music Technology Group at Pompeu Fabra University in Barcelona:

Here is my plug for the NUI group, once again!
"The NUI group, or Natural User Interface Group, is an interactive media group researching and creating open source machine sensing techniques to benefit artistic and educational applications.
We offer a collaborative environment for developers that are interested in learning and sharing new HCI (Human Computer Interaction) methods and concepts. This may include topics such as: voice/handwriting/gesture recognition, touch computing, computer vision, and information visualization."
FYI
I came across the Harry van der Veen of the NUI group in early 2007 when I was working on touch-screen projects for my HCI and Ubicomp classes, and I'm inspired by all of the creativity I've seen coming from this group.
If you'd like to see more demos, visit the Natural User Interface website, a commercial out-growth of Harry and his colleague's work, where you can view a reel that includes a few touch-screen games. I love the vision statement on this site:
"Technology should enable us to interact with computers, in the same way we interact with the real world; in a way which is natural to us, namely through gestures, expressions, movements, and manipulations. Our vision is to change the way people interact with computers."
May 14, 2008
Multi-touch Crayon Physics
Watch how you can draw simple shapes that can instantly turn into a game!
Multitouch Crayon Physics from multitouch-barcelona on Vimeo.
For a better version of this video, see http://www.vimeo.com/980528 For more information, see the RXSurface blog post. On May 18, Multi touch crayon physics will be offered as an alphabeta opensource! The people behind RXSurface are members of the Natural User Interface (NUI) group:
"Natural User Interface or ~ NUI Group is an interactive media group researching and creating open source machine sensing techniques to benefit artistic and educational applications."
May 12, 2008
Seth Sandler's "How to Make a Cheap Multitouch Pad" YouTube video is going viral...
Thanks Seth, for sharing this vision with the world!
Seth's AudioTouch Blog: "An Interactive Multi-user, Multi-touch Musical Table and More"
Hint for high school teachers: This sort of thing would be a great project for an after-school technology club!
Mar 17, 2008
Look, touch, listen, and play: Seth Sandler's Interactive Audio Touch Table Video; NUI Group and Google's Summer of Code
Seth is a member of the NUI group (Natural User Interface). He is finishing a Bachelors degree in Interdisciplinary Computing and the Arts, with an emphasis on Music, at the University of California, San Diego. His research and development work centers around multi-touch, multi-user musical interfaces.
Here is an update about the NUI group:
"Natural User Interface or ~ NUI Group is an interactive media group researching and creating open source machine sensing techniques to benefit artistic and educational applications."
"We offer a collaborative environment for developers that are interested in learning and sharing new HCI (Human Computer Interaction) methods and concepts. This may include topics such as: voice/handwriting/gesture recognition, touch computing, computer vision, and information visualization."
The NUI group has been selected for mentoring organization for Google's Summer of Code, for those of you who are interested in working on open-source code for multi-touch systems. The student application process begins Monday, March 24th, 2008, and ends Monday, March 31st, 2008.
NUI group's project ideas page outlines the requirements for the application, which includes a 7500 word project proposal. The project page has a long list of ideas to spark some thinking for potential Summer of Code applicants.
For those of you who aren't into coding, I encourage you to take a look at the NUI Groups project ideas page just to get an idea of th interesting ideas that are being explored. The page has a list of links to other good resources.
Share the word with anyone who might be interested in the NUI Group's projects for the Summer of Code. We need to get more people interested in STEM careers, and the project ideas outlined by the NUI group look enticing.
Mar 3, 2008
FireFly Game for Microsoft Surface: Demo by Carbonated Games; A look at a Multi-touch from Next Window
Watch the video clip to see more about Firefly game that was developed by Carbonated Games to take advantage of Microsoft's multi-touch enabled Surface table computer system. The game is in demo form only.For more information, see the article from Gaming Today.
For those of you who need multi-touch before you can get your hands on the Surface, you might be interested in NextWindow's multi-touch display, recently demonstrated at an expo:
Here are links to touch-screen project prototypes I worked on for HCI and Ubicomp classes I took in 2007:
poetry picture share photo-globe explorer
Oct 28, 2007
Link to video: Microsoft Research UK's multi-touch research team
Direct link to the wmv file"Inside MultiTouch: Team, Demo, Lab Tour"
Take a look at a post on the MSDN Channel 9 website about an interview of members of Microsoft Research UK's multi-touch team. The researchers on the video are Shahram Izadi, Alex Butler, and Steve Hodges. The video contains some interesting demos. This team's approach to multi-touch is different than approach taken by the Microsoft Surface team. This video is well worth the 30 minutes!
This technology would be great for interactive educational games and 3-D applications. Microsoft has plans for this technology to be used in the home and for shared applications.
Oct 18, 2007
Microsoft UK: "ThinSight" Multi-touch Technology on a Laptop
(Photos from New Scientist)Tom Simonite's article in a recentNewScientist article reviews a prototype for a touch-screen lap-top using "ThinSight", developed by Microsoft Research. Other companies are researching similar systems.
Multi-touch screens and surfaces support the work of people who increasingly rely on visualization or interactive multimedia to do their jobs. In addition, 3D multi-touch applications created for laptops and tablet PC's might be good for education, training simulations, and serious games.
Oct 16, 2007
Visualizing Content on the Web: Great pictures, descriptions, and links on Max Kiesler's blog
Searches conducted on the Web are primarily word-based, and the same is true for searches conducted on PC's. What results is a list of phrases, which are often difficult to sort and prioritize quickly.
The increased bandwidth available on the web now allows for content that contains a range of multimedia components, such as photos and video clips. Many on-line journal articles, blog posts, and of course, information visualization websites, contain important visual-oriented information that might be missed through traditional searches.
Kiesler discusses some solutions to this problem in his post. Take a look at it soon- it contains great visualizations, as well as links to additional information and resources.
Although Keisler does not state it directly, his post makes the case for the importance visual and multimedia literacy in our society.
Note: For those of you who follow my blog, you'll know that this sort of visual approach would be a great tool in education, especially when presented on interactive large-screen displays.
Aug 6, 2007
About Interaction: Bill Buxton and Multi-Touch Systems
Bill provides a good overview of the various flavors of multi-touch interaction and interfaces, along with a an annotated chronology of multi-touch and related technologies, starting before 1982. The chronology includes some interesting pictures.
Bill Buxton works for Microsoft Research. The following information is from his bio:
"Bill Buxton is a designer and a researcher concerned with human aspects of technology. His work reflects a particular interest in the use of technology to support creative activities such as design, film making and music. Buxton's research specialties include technologies, techniques and theories of input to computers, technology mediated human-human collaboration, and ubiquitous computing."
May 31, 2007
Microsoft Surface multi touch screen table - I wish I had one for my projects last semester!
This multi - touch table can be used for a variety of applications, as outlined in the video from CNET and YouTube below:
I'd like to work on applications for use on a touch-table to support students with special needs, especially those who have autism spectrum disorders.
Last semester, I worked on prototype applications for use on a touch-screen surface -here is a photo. We used a NextWindow Human Touch large-screen display, which provided great screen resolution and touch-response.

This application was part of a travel-planning prototype developed for a course in Human-Computer Interaction. The application was demonstrated on a NextWindow Human Touch large screen display. Would it work on the iPhone?
Update: Examples from some of my other posts:

Here is another demo videoclip of a globe created in GoogleEarth using photo-overlays, with links to video clips uploaded to YouTube and embedded in individual posts on a blog. The above photo and the video clip show the application on a NextWindow Human Touch large-screen display.
This application would be great on a touch-table or touch-table set up on a drafting board. Although it was designed for a travel-planning application, it would work well in educational settings in subjects such as geography.
Poetry Picture Share

This was my first attempt at a "poetry picture share" application. It was designed for use on a multi-touch table and can be accessed remotely so people in different places can move things around on the screen. The video shows how the application works on a NextWindow Human Touch interactive large-screen display.
I am planning on adapting this application for use with students with special needs, such as those who have autism or other communication disorders.

