Link to "Internet of Surfaces" Photo Examples

Take a look!
INTERNET OF SURFACES: Photo Examples of Screens of All Sizes

More pictures will be added.

Natural User Interface: Overview of multi-touch technology and application development by Harry van der Veen,- Business to Buttons

Harry van der Veen from Natural User Interface Europe AB, was one of the keynote speakers at the Business to Buttons: Designing for Effect conference, held in June 2008. In this presentation video, Harry discusses the past, present, and future of multi-touch technology, and reviews the importance of multi-touch over single touch displays. He also provides a good overview of gesture interaction, something that he researched when he was a student. This presentation includes several video examples of multi-touch applications in action.

The presentation is well worth the 30-minute view!

"Harry van der Veen is a Bachelor of Multimedia, derived from the Dutch education Communication, Multimedia and Design, focused on Interaction Design and Project Management. He is CEO, co-founder and co-owner of the Sweden based commercial company Natural User Interface Europe AB, which focuses on delivering standardized and customized multi-touch hardware / software solutions and services to the global market. In addition to that, he co-founded the NUIGroup community, which is the worlds largest online platform where a global network of people share their ideas and information in an open source community, focused on multi-touch hardware and software solutions."


NUIGroup Community


Harry van der Veen's blog

Natural User Interface Europe AB (Harry van der Veen's company)

NUIGroup Wiki: This wiki includes tutorials for developing multi-touch applications, building your own low-cost multi-touch table, and information about current projects that are in progress.

Related Information:

The Business to Buttons: Designing for Effect conference was held on June 12-12 in Malmo, Sweden, organized by Malmo University and inUse, a user experience consultancy. Partners in this conference included Adaptive Path, a product experience strategy and design company, Patrick W. Jordan, a design, marketing, and brand strategist, the cocktail, a user experience and interaction design studio, cooper, a product design company, and OresundIT, a non-profit network.

Don Norman, the author of books such as "Design of Everyday Things" and "The Design of Future Things", presented at this conference. Don Norman is one of the founding fathers of the Human-Computer Interaction and related fields, and is the co-founder of the Nielsen Norman Group, a consultant firm that helps company create human-centered products.

Videos of Don Norman's Presentations:
Emotional Design: Total User Experience
Cautious Cars and Cantankerous Kitchens

Other:
Business to Buttons 2008 Recorded Sessions
Business to Buttons 2008 Downloads

My posts about the work of NUI Group members:

Multi-Touch Plug-in for NASA World Wind?!

More Multitouch: NUI Group's Christopher Jette's multi-touch work featured in Engaget ; Croquet?

More Multi-Touch from members of the NUI group!

Multi-touch Crayon Physics from multitouch-barcelona, inspired by Crayon Physics by Kloonig Games

Cross Post: Seth Sandler's YouTube Video, "How to Make a Cheap Multi-touch Pad" goes viral

NUI-Group Member Bridger Maxwell Receives High School Science Fair Award for Multi-Touch Screen Project

Look, touch, listen, and play: Seth Sandler's interactive Audio Touch Table video; NUI Group and Google's Summer of Code

Emerging Interactive Technologies, Emerging Interactions, and Emerging Integrated Form Factors

Every so often I come across a blog that has been around for a while, but is a new discovery to me. If you are interested in interactive multimedia and emerging technology, and you haven't heard of Richard Banks, you will probably be pleased with his vast array of interesting posts!

After browsing around Richard Bank's blogs, I found that he works for Microsoft Research. Like me, he maintains more than one blog, and he blogs about similar topics.

Here is a view into his world:

rb.trends: tracking future technology experience
rb.work: technology, design and research
rb.log: family, photography, architecture etc.

One of Richard's recent posts brought me back to my musings about interactive surfaces:

Via Richard Banks and Gizmodo: A Touch Screen Poker Table from Hard Gaming

If it is possible to program an application to handle the metrics that support poker, then it is possible to program a multi-user application for work or education, and have this application integrated into any type display, according to need.

The form factor above would be useful for team meetings or collaborative project planning, especially during group decision-making or policy planning activities that involve data analysis or information visualization.

Could this concept be modified to fit the form of a kidney table for use in education? Think about it. We could combine the best features of surface computing with the best features of multi-user video games.

Rationale? Video-game applications can handle the metrics of multi-user interaction, which would be ideal for tracking student progress and interaction. Touch interaction would enable young children with an easy way to access the application- or all of the information that will be available on the multi-touch web of the future, without learning first to type.

(Touch and gesture interaction could help us move towards the implementation of the Internet of Things)

Could this concept be transformed for use by two people collaborating on an electronic drafting table?




If you are a NUI Group member, think about this!


Note: I came up with the phrase, "multi-touch web of the future" as I was typing this post. I guess I didn't invent it - I looked it up and found Andrew Foote's post, "The Multi-Touch Web"

I'm in.


Somewhat Related.......

More for the tech-curious:

Interactive data visualization in Second Life

| View | Upload your own

Idle together "Technology blog oriented towards good design and impressive web applications"

Apple Developing Full Multi-Touch Macs - Apple Insider

More for the tech-savvy:

Multi-touch gestures in the Factor UI (Apple)- by Pestov. Includes code.

NSResponder Modifications: Swipe, Rotate, and Magnify, by Elliott Harris, a member of the NUI Group, I think.

Web Development Guidelines for iPhone (lots of good info, with video clips and code)

Web 3D Consortium

Unity 3D kit for the

Google's Android

Multi-touch Web Consortium (in my dreams)

Gesture Web Consortium (in my dreams)

New Interactive Technology: Tag Galaxy, Windows 7, PaperVision 3D, Microsoft Surface, Touch Screens, and Blog Posts Revisited

Steven Wood's Tag Galaxy
Via Papervision3D, Richard Banks, and Flowing Data

Tag Galaxy is an application that uses Flickr and Papervision 3D. If you type in a tag, the tag is represented as the sun in the galaxy, and related tags are represented as planets. When you rotate and browse through the galaxy in 3D space, you can view the pictures according to the tag. Steven Wood created Tag Galaxy for his thesis project.

Separated at Birth?
Vacation pictures on an interactive photo-globe - April 2007

Just over a year ago, I finished some class projects that were designed for touch screens or tables. At the time, I had no idea that Microsoft was working on the Surface. I was determined that I'd make my own table if I couldn't get my hands on one, and even joined the NUI Group to learn more about the open-source DYI approach to satisfying my fascination with this interactive technology.


The first day I learned about Microsoft Surface will be etched in my memory forever...


It is not because I'm a 100% Microsoft fan. It is because I'd been thinking about touch-screen interaction since my first encounter with an interactive whiteboard several years ago. At the time, I was involved with group counseling with middle school students, seated around a table.

What would happen if we took the whiteboard put it on a table?

As noted below, the demo projects worked best on a NextWindow Human Touch. Although the large display was only "single-touch", it provided excellent resolution and touch response. Since then, NextWindow has come out with a multi-touch display, which I've had a chance to see. It is not a table, but it has possibilities.

What would happen if we took a NextWindow multi-touch display and mounted it onto an adjustible drafting board? It could be used as a vertical display, a "surface" table, and a drafting board. The best thing is that this could support quite a bit of collaborative work between two or more people, as well as learning, creative-artistic, and gaming activities.

New things are on the horizon. Windows 7, will replace Vista in the future. Bill Gates has pronounced that "every surface will be a computer". Touch screen surfaces are finding homes on tablet PC's, cell phones, and the latest OLPC "$100.00" laptops for children.

I think there will be more possibilities for using touch-screen technology for education in the very near future.

Here is my post about Microsoft's Surface from last year, with some updates:

Microsoft Surface multi touch screen table - I wish I had one for my projects last semester!

Microsoft Research recently unveiled Microsoft Surface: http://microsoft.com/surface. This multi - touch table can be used for a variety of applications, as outlined in the video from CNET and YouTube below:



I'd like to work on applications for use on a touch-table to support students with special needs, especially those who have autism spectrum disorders.

.....Last semester, I worked on prototype applications for use on a touch-screen surface -here is a photo. We used a NextWindow Human Touch large-screen display, which provided great screen resolution and touch-response.



This application was part of a travel-planning prototype developed for a course in Human Computer Interaction. The application was demonstrated on a NextWindow Human Touch large screen display. Would it work on the iPhone?







Update: Examples from some of my other posts:

Below is another demo video-clip of a globe created in GoogleEarth using photo-overlays, with links to video clips that pop up on the screen. You can spin and rotate the globe at any size, and zoom into the pictures. The above photo and the video clip show the application on a NextWindow Human Touch large-screen display.

This application would be great on a touch-table or touch-table set up on a drafting board. Although it was designed for a travel-planning application, it would work well in educational settings in subjects such as geography.


Poetry Picture Share

This was my first attempt at a "poetry picture share" application. It was designed for eventual use on a multi-touch table. It was developed using JavaScript and Ajax. It could be accessed remotely so people in different places can move things around on the screen. The video shows how the application works on a large interactive touch-screen display.

I am planning on adapting this application for use with students with special needs, such as those who have autism or other communication disorders. (Note: I've used it with several students, with success. It still is a work in progress!)



Update:

Link to Papervision 3D: Press the picture to enter a 3-D interactive underwater world...

If you have tried PaperVision 3D or Tag Galaxy on an interactive whiteboard or touch screen display, please leave a comment and share your impressions.

Top 15 Posts: Interactive Multimedia Technology

As of 5/31/08
Interactive multimedia for social skills, understanding feelings, relaxation and coping strategies
Engaged Learning Revisited: Four videoclips for reflection....
UMAGINE Creative -Digital Story Telling for Interactive Whiteboard or Touch Screen- free demo available
Interactive Multimedia Technology: May 2007
Dance.Draw Project : Exquisite Interaction - Collaboration between Software Information Systems -HCI- and Dance Department
Microsoft Surface multi touch screen table - I wish I had one for my projects last semester!
Programming for interactive multimedia applications: WPF, Silverlight, EduSim, NeuroVR - Musings: Learning about application development and programming at mid-life
Interactive Information Visualization: Digg Labs Website
iPod Touch Possibilities... I want one!
NASA's promotion of MMO games to support STEM learning; Space Station Sim game; EASe games for children
Interactive Multimedia Technology for Science and Math
More Multi-touch: Microsoft Windows 7 Demo on a Dell Laptop
More multi-touch from members of the NUI group!
My Nokia n800 Internet Tablet!
NUI Group Member Bridger Maxwell Receives High School Science Fair Awared for Multi-touch Screen Project

Windows 7 Demo on a Dell laptop: More Multi Touch

Video: Multi-Touch in Windows 7

Via SoapBox, Greenbush Labsand CNET

If you are new to this blog and would like to learn more about multi-touch interaction and technology, enter a keyword in the search box.

Also take a look at the Technology Supported Human-World Interaction blog.

For multi-touch DYI, check out the NUI Group!

NUI-Group Member Bridger Maxwell Receives High School Science Fair Award for Multi-Touch Screen Project

Yet another post about a NUI group member... Bridger Maxwell, a high school student at the Utah County Academy of Sciences, submitted his multi-touch screen to the science fair, and went on to win first place in the engineering category, and now will be competing in the International Science and Engineering Fair.





Bridger has created "Lumen", a puzzle game for OSX and Windows, and markets this through his business, Fiery Ferret.

Even More Multi-Touch, Delivered by NUI...

May 20, 2008
NUI announces delivery of their multi-touch solution for Cityscape, Abu Dhabi 2008





www.natural-ui.com
Natural User Interface Europe Ltd.

More Multi-Touch from members of the NUI group!

It is always exciting to see what members of the NUI group are doing!

Here is a new video of a multi-touch creation by some of the members of the NUI group. Although this is a proof-of-concept example, it is fun to see how it is played out, using the little iPhone-like touch-pad widgets as a navigation tool for the large screen.


Read the "Multi-touch Goodness" article in Gizmodo of an interview with Christian Moore about this demo and his Lux open-source framework. (Christian is a colleague of Harry van der Veen, both members of the NUI group.)

Here is an excerpt from the interview:
"JD: Why Flash?
CM: Because it's fast to prototype in. However, the software is broken into several segments. One C++ application that tracks hands that talks to Flash... WPF... or another C++ app... and basically everything you can imagine. You can enable multitouch in any environment, like Cocoa."

High-resolution screen shots and additional information can be found on the nuiman website.

For my tech-minded readers:
I'm pretty sure that the C++ application that track hands and fingers in the video demo uses Touchlib, a library for creating multi-touch interaction. Touchlib can work with TUIO, a protocol for tabletop tangible user interfaces. Applications such as Flash and Processing support TUIO. For more information about TUIO, read "TUIO: A Protocol for Table-Top Tangible User Interfaces".
(Information from the NUI group website mentions that OpenCV, or Open Computer Vision Library, found on SourceForge, can support blog detection and tracking.)

The people behind TUIO are from the Reactable project, of the Music Technology Group at Pompeu Fabra University in Barcelona:

Here is my plug for the NUI group, once again!

"The NUI group, or Natural User Interface Group, is an interactive media group researching and creating open source machine sensing techniques to benefit artistic and educational applications.

We offer a collaborative environment for developers that are interested in learning and sharing new HCI (Human Computer Interaction) methods and concepts. This may include topics such as: voice/handwriting/gesture recognition, touch computing, computer vision, and information visualization."


FYI
I came across the Harry van der Veen of the NUI group in early 2007 when I was working on touch-screen projects for my HCI and Ubicomp classes, and I'm inspired by all of the creativity I've seen coming from this group.

If you'd like to see more demos, visit the Natural User Interface website, a commercial out-growth of Harry and his colleague's work, where you can view a reel that includes a few touch-screen games. I love the vision statement on this site:

"Technology should enable us to interact with computers, in the same way we interact with the real world; in a way which is natural to us, namely through gestures, expressions, movements, and manipulations. Our vision is to change the way people interact with computers."

Multi-touch Crayon Physics

Also posted on the TSHWI blog:
Watch how you can draw simple shapes that can instantly turn into a game!


Multitouch Crayon Physics from multitouch-barcelona on Vimeo.
For a better version of this video, see http://www.vimeo.com/980528 For more information, see the RXSurface blog post. On May 18, Multi touch crayon physics will be offered as an alphabeta opensource! The people behind RXSurface are members of the Natural User Interface (NUI) group:
"Natural User Interface or ~ NUI Group is an interactive media group researching and creating open source machine sensing techniques to benefit artistic and educational applications."

Look, touch, listen, and play: Seth Sandler's Interactive Audio Touch Table Video; NUI Group and Google's Summer of Code

Seth Sandler's most recent video of the Audio Touch interactive table provides a good demonstration of how multi-touch on a table can work with music.



Seth is a member of the NUI group (Natural User Interface). He is finishing a Bachelors degree in Interdisciplinary Computing and the Arts, with an emphasis on Music, at the University of California, San Diego. His research and development work centers around multi-touch, multi-user musical interfaces.

Here is an update about the NUI group:

"Natural User Interface or ~ NUI Group is an interactive media group researching and creating open source machine sensing techniques to benefit artistic and educational applications."

"We offer a collaborative environment for developers that are interested in learning and sharing new HCI (Human Computer Interaction) methods and concepts. This may include topics such as: voice/handwriting/gesture recognition, touch computing, computer vision, and information visualization."

The NUI group has been selected for mentoring organization for Google's Summer of Code, for those of you who are interested in working on open-source code for multi-touch systems. The student application process begins Monday, March 24th, 2008, and ends Monday, March 31st, 2008.

NUI group's project ideas page outlines the requirements for the application, which includes a 7500 word project proposal. The project page has a long list of ideas to spark some thinking for potential Summer of Code applicants.

For those of you who aren't into coding, I encourage you to take a look at the NUI Groups project ideas page just to get an idea of th interesting ideas that are being explored. The page has a list of links to other good resources.

Share the word with anyone who might be interested in the NUI Group's projects for the Summer of Code. We need to get more people interested in STEM careers, and the project ideas outlined by the NUI group look enticing.