Cross post: Technology Supported Human World Interaction

Here is a link to two recent posts on the TSHWI blog:

Why not adopt the Google User Experience Team's principles for off-the-desktop digital interactions?

Off-the-desktop musings about future interactions: User experience, user-driven design, Universal Usability, Airports, and the "Internet of Things"

Hands-On Computing: InfiniTouch's Innovation

In response to the comment I received about my last post, I did some hunting and posted about QSI's InfiniTouch products on the Technology Supported Human-World Interaction blog. The post contains a couple of videoclips about InfiniTouch, as well as links about the various uses of the application/system.

If you are interested in touch interaction, it is worth exploring!

Hands On Computing: How Multi-Touch Screens Could Change the Way we Interact with Computers and Each Other (link to Scientific American Article)

More Multi-Touch!

Scientific American, June, 2008 Hands On Computing: How Multi-touch Screens Could Change The Way We Interact With Computers and Each Other "The iPhone and even wilder interfaces could improve collaboration without a mouse or keyboard. "

"It is easy to imagine how photographers, graphic designers or architects—professionals who must manipulate lots of visual material and who often work in teams—would welcome this multi-touch computing. Yet the technology is already being applied in more far-flung situations in which anyone without any training can reach out during a brainstorming session and move or mark up objects and plans." -Stuart Brown

Link: Emotiv System's Neural Game Controller Headset: Human-Computer Interface of the Future?

If you are looking for information about brain-computer interfaces, follow the link to my post about Emotive Systems neural interface on the Technology-Supported Human-World Interaction blog.

Emotiv System's Neural Game Controller Headset: Human-Computer Interface of the Future?

Also see:
Game Interaction via Thoughts and Facial Expressions: EPOC - Emotiv Systems Neural Interface

Oculusia, an Interactive Multi-Touch Table Game for Four People

The following video is a nice demonstration of a multi-touch table game, Oculusia. The game was created by four students from the Aalborg University, Copenhagen, for Medialogy 2008. The students, Jens Fursund, Thomas Frohlich, Thomas Miksa, and Tobias Brogaard, also created the table.



More Information:
Students Explore Multi-touch with Oculusia

Related:
The "Workbench" used in the Medialogy Lab at Aalborg University, Copenhagen:


"The Workbench displays 3D stereo images and the tracking system allows for interaction with virtual objects on the display surface. The tracking system uses infrared light to illuminate the display surface and the camera is only sensitive to infrared light. Basic image processing techniques are then applied to acquire position of the hand."

A Multi-Touch DigitalPainting Application pdf (Søren Støvlbæk, Larsen Gavin, Francis Murray & Jesper Højgaard Pedersen:Institute of Electonic Systems, Computer Vision, and Graphics, Aalborg University 2007)

The ReacTable: A Tangible Musical Instrument pdf(Martin Kaltenbrunner, Music Technology Group, Universitat Pompeu Fabra, 2006)

DigiBoard Multi-Touch Mixed Reality Game; Ideas for future design of a flexible, adjustable multi-touch surface..

This video is a demo of a multi-touch table, called the DigiBoard, running a game application called "Tower Defense". This multi-modal game takes input from physical tokens, touch, and provides auditory feedback.

According to the credits on the video-clip, the creators of this application were Andreas Hesel, Birna Run Olafsdottir, Dann Sandgreen, and Osk Hilmarsdottir. When I can track down more information about this group, I'll post it!

For more information about interactive display technologies, see Gizmo Watch's Top 15 Interactive Display Technologies. The list is full of photos and video clips of a variety of systems. It was created last year, so it might need some updating.

I am still looking for some examples of good multi-touch table games for use in education. I've created a few prototypes that are not fully fleshed out over the past year, but I haven't had the time or money to build the type of adjustable table that I want.

I'd like my table surface to adjust horizontally, vertically, and angles in-between, like a drafting table. I don't want my table to rely on a projector, since I don't want unnecessary occlusion. A rear-projection system would be heavy and bulky. My design is light and sleek, and it is mobile.

The advantage of an adjustable design is that it allows for flexibility in use, within a single environment, and across a variety of settings. It could work well in schools, libraries, museums, convention centers, exhibit halls, hospitals, visitor centers, airports, public transportation stations, board rooms, data centers, malls, supermarkets, community centers, coffee houses...just about anywhere.

I've written about this topic a few times. For more information, see "Emerging Interactive Technologies, Emerging Interactions, and Emerging Integrated Form Factors"

If you know of anyone who is creating an adjustable, multi-functional touch surface, please let me know.

I am playing around with ideas for a suite of multi-modal, adaptive game for use with young people who have suffered traumatic brain injuries, and others who have special needs. A stationary surface would not provide the flexibility my applications will require.

Fingertapps and NextWindow's Multi-Touch Firmware on the New HP Touchsmart PC,

Fingertapps is working on an application that utilizes NextWindow's multi-touch firmware, running on the second-generation HP Touchsmart PC. Fingertapps will be demonstrating their work in the NextWindow booth at Infocomm 2008 in Las Vegas, June 14-20.

Off-topic: Now I know why some people think I'm a Microsoft shill!

A while ago, I posted a response on another blog, and someone mistakenly thought that I was a Microsoft shill. Me?!

Now I know why.



When I was looking at my blog stats, I noticed that one of my visitors found a recent post via a Google search on microsoft new interactive technology. Curiously, the post was at the very top of the list:

New Interactive Technology: Tag Galaxy, Windows 7, PaperVision 3D, Microsoft Surface, Touch Screens, and Blog Posts Revisited

Results 1 - 10 of about 29,600,000 for microsoft new interactive technology. (0.12 seconds)

How did my post beat out over 29 million pages on the web?!

I've had an Apple computer, in one form or another, since 1985. When I decided to take a programming class a few years ago, I soon realized I had to adapt to the Windows world. I like C#, Silverlight, WPF, and XNA Game Studio. This does not make me a Microsoft shill.

I just happen to be overly passionate about interactive displays of all sizes, and fascinated with the possibilities of mult-touch. iPhones, touch-tables, touch-walls, devices that interact with displays (think Johnny Chung Lee), gesture + touch, urban screens, interactive exhibits, interactive way-finding... and the multimedia applications that support this sort of interaction.

I've written several posts about Apple's new technologies, like the iPod Touch and the new 3G iPhone. I'll be writing more about Apple as they research and develop flat panel displays:

"Apple is looking for a Senior Panel Process Engineer to lead the engineering activities to develop the new process and design for the multi-touch panel used in Apple products. This individual will serve as a focal point in the designing and the process development of advanced multi-touch panels from the concept to the product ramp."

I'd say I'm more of a cheerleader for all of those out there who are working so hard to create useful+cool technology for the future.

Update:
The Next Generation HP TouchSmart PC
The new TouchSmart PC's, which can display high definition content, aren't out on the market yet. HP is taking pre-orders.

NextWindow, a company that produces large interactive touch-screen displays, including those with multi-touch capabilities, was responsible for the touch-screen embedded in the TouchSmart PC.

Related:

HP Redefines Home Computing, Putting the Digital Lifestyle at People's Fingertips with New TouchSmart PC's

HP Introduces World's First Affordable Color-critical Display

MICOLE: Open source multi-modal, multi-sensory software

Here is a quick link to the following post on the TechPsych blog, via ICT Results

MICOLE: Open-source multi-modal software supports cooperative learning among sighted and visually impaired children

The new 3G iPhone: Expanding the Possibilities of Interactive Multimedia Communication

The new iPhone is filled with possibilities....In this post, I'm sharing some pictures of the iPhone that look interesting to educators and those of us who want to increase our interactive multimedia communication and entertainment when we are out and about.

The following pictures from a post I came across on the engadget website today, authored by Ryan Block, while he attended today at WWDC 2008. The new 3G iPhone, with a price of $199.00, was the focus of Steve Job's keynote. I've added a few links to the developers of new iPhone apps, where possible.


3G is fast



Apple offers MobileMe service for the iPhone, which is something most people on-the-go will appreciate:

"Everything you need. Anywhere you are."

"MobileMe stores your email, calendar, and contacts on a secure online server, or “cloud.” The cloud pushes the most current data to your iPhone, iPod touch, and computer so you’re always up to date. And your email, calendar, contacts, photos, and documents are accessible over the Internet through a set of easy-to-use web applications."

Push email, push calendar, and push contacts Web applications at me.com

MobileMe Gallery for photo sharing MobileMe iDisk for online document access

20GB of online storage

This would make the iPhone ideal for mobile learning! Here is an example of an application for science:





Great for musicians and music students who are away from their gear:


According to Ryan Block, this application is from Moo Cow Music. The application is called "Band". Here are the features, as listed on the Moo Cow Music website:

• Drums, bass, guitars, keyboards can all be played and recorded into one song from the same application.
• Multitouch - press up to five keys at once for complex chords.
• Animation shows keys being pressed, strings being struck, etc.
• Any instrument can be recorded, or overdubbed onto a previous recording.
• Erase any mistakes, then go back and overdub replacement notes.
• A configurable metronome can be enabled or disabled during recording.

Here is a video of the beta version of the application:

"Kroll", from Digital Legends Entertainment, runs on the iPhone and is an action-adventure fantasy game.


Sega's Super Monkey Ball on the iPhone


Apple iPhone Site

Link to video of Keynote


Verizon, can you hear me now? I want an iPhone!

Engaged Learning and Social Physics: Phun, an Interactive 2D Physics Sandbox

UPDATE 4/24/11:   Phun is commercially available and now optimized for use on the newer multi-touch, multi-user SMARTboards.  It is now known as Algodoo.

For those of you searching for interesting and engaging science activities for use on an interactive whiteboard or display, take a look at Phun. According to the website, "Phun is a playground for the creative mind where machines and toys can be easily created."

It was Emil Ernerfeldt's MSc Computer Science project, created at the Umeå University VR lab in Sweden, under the supervision of Kenneth Bodin.

The Phun application can be downloaded for free. The video provides a good overview of Phun's features, accompanied by soothing music that can you can download at no cost at http://phun.cs.umu.se/files/Phun_theme.mp3


From the creators of Phun:
"Phun is an educational, entertaining and somewhat (!) addictive piece of software for designing and exploring 2D multi-physics simulations in a cartoony fashion. It is part of our long term mission to bring visual physics based simulation to the masses. The application is developed for Umevatoriet, Umeås new science center, where it will run on a large interactive display, but you can also download it and run it on your own pc."

The beauty of Phun is the social aspect. Physics 2.0. There is a YouTube Phun group, where people share videos of their Phun creations. The Phun wiki provides a tutorial, a forum, and links. Phun-makers can share and rate their creations in the Phun Box.

Note: Phun is now part of Algoryx Simulation AB, a spin-off from Umea University. Algoryx focuses on interactive 3D multiphysics simulation. The free version of Phun will continue to be updated and supported, according to Emil Ernerfeldt.

Related Links:
Phun at Algoryx
Forum Thread: How Phun can be used in education
http://phun.cs.umu.se/wiki
Download Page (Mac, Win, Linux)

Emerging Interactive Technologies, Emerging Interactions, and Emerging Integrated Form Factors

Every so often I come across a blog that has been around for a while, but is a new discovery to me. If you are interested in interactive multimedia and emerging technology, and you haven't heard of Richard Banks, you will probably be pleased with his vast array of interesting posts!

After browsing around Richard Bank's blogs, I found that he works for Microsoft Research. Like me, he maintains more than one blog, and he blogs about similar topics.

Here is a view into his world:

rb.trends: tracking future technology experience
rb.work: technology, design and research
rb.log: family, photography, architecture etc.

One of Richard's recent posts brought me back to my musings about interactive surfaces:

Via Richard Banks and Gizmodo: A Touch Screen Poker Table from Hard Gaming

If it is possible to program an application to handle the metrics that support poker, then it is possible to program a multi-user application for work or education, and have this application integrated into any type display, according to need.

The form factor above would be useful for team meetings or collaborative project planning, especially during group decision-making or policy planning activities that involve data analysis or information visualization.

Could this concept be modified to fit the form of a kidney table for use in education? Think about it. We could combine the best features of surface computing with the best features of multi-user video games.

Rationale? Video-game applications can handle the metrics of multi-user interaction, which would be ideal for tracking student progress and interaction. Touch interaction would enable young children with an easy way to access the application- or all of the information that will be available on the multi-touch web of the future, without learning first to type.

(Touch and gesture interaction could help us move towards the implementation of the Internet of Things)

Could this concept be transformed for use by two people collaborating on an electronic drafting table?




If you are a NUI Group member, think about this!


Note: I came up with the phrase, "multi-touch web of the future" as I was typing this post. I guess I didn't invent it - I looked it up and found Andrew Foote's post, "The Multi-Touch Web"

I'm in.


Somewhat Related.......

More for the tech-curious:

Interactive data visualization in Second Life

| View | Upload your own

Idle together "Technology blog oriented towards good design and impressive web applications"

Apple Developing Full Multi-Touch Macs - Apple Insider

More for the tech-savvy:

Multi-touch gestures in the Factor UI (Apple)- by Pestov. Includes code.

NSResponder Modifications: Swipe, Rotate, and Magnify, by Elliott Harris, a member of the NUI Group, I think.

Web Development Guidelines for iPhone (lots of good info, with video clips and code)

Web 3D Consortium

Unity 3D kit for the

Google's Android

Multi-touch Web Consortium (in my dreams)

Gesture Web Consortium (in my dreams)

New Interactive Technology: Tag Galaxy, Windows 7, PaperVision 3D, Microsoft Surface, Touch Screens, and Blog Posts Revisited

Steven Wood's Tag Galaxy
Via Papervision3D, Richard Banks, and Flowing Data

Tag Galaxy is an application that uses Flickr and Papervision 3D. If you type in a tag, the tag is represented as the sun in the galaxy, and related tags are represented as planets. When you rotate and browse through the galaxy in 3D space, you can view the pictures according to the tag. Steven Wood created Tag Galaxy for his thesis project.

Separated at Birth?
Vacation pictures on an interactive photo-globe - April 2007

Just over a year ago, I finished some class projects that were designed for touch screens or tables. At the time, I had no idea that Microsoft was working on the Surface. I was determined that I'd make my own table if I couldn't get my hands on one, and even joined the NUI Group to learn more about the open-source DYI approach to satisfying my fascination with this interactive technology.


The first day I learned about Microsoft Surface will be etched in my memory forever...


It is not because I'm a 100% Microsoft fan. It is because I'd been thinking about touch-screen interaction since my first encounter with an interactive whiteboard several years ago. At the time, I was involved with group counseling with middle school students, seated around a table.

What would happen if we took the whiteboard put it on a table?

As noted below, the demo projects worked best on a NextWindow Human Touch. Although the large display was only "single-touch", it provided excellent resolution and touch response. Since then, NextWindow has come out with a multi-touch display, which I've had a chance to see. It is not a table, but it has possibilities.

What would happen if we took a NextWindow multi-touch display and mounted it onto an adjustible drafting board? It could be used as a vertical display, a "surface" table, and a drafting board. The best thing is that this could support quite a bit of collaborative work between two or more people, as well as learning, creative-artistic, and gaming activities.

New things are on the horizon. Windows 7, will replace Vista in the future. Bill Gates has pronounced that "every surface will be a computer". Touch screen surfaces are finding homes on tablet PC's, cell phones, and the latest OLPC "$100.00" laptops for children.

I think there will be more possibilities for using touch-screen technology for education in the very near future.

Here is my post about Microsoft's Surface from last year, with some updates:

Microsoft Surface multi touch screen table - I wish I had one for my projects last semester!

Microsoft Research recently unveiled Microsoft Surface: http://microsoft.com/surface. This multi - touch table can be used for a variety of applications, as outlined in the video from CNET and YouTube below:



I'd like to work on applications for use on a touch-table to support students with special needs, especially those who have autism spectrum disorders.

.....Last semester, I worked on prototype applications for use on a touch-screen surface -here is a photo. We used a NextWindow Human Touch large-screen display, which provided great screen resolution and touch-response.



This application was part of a travel-planning prototype developed for a course in Human Computer Interaction. The application was demonstrated on a NextWindow Human Touch large screen display. Would it work on the iPhone?







Update: Examples from some of my other posts:

Below is another demo video-clip of a globe created in GoogleEarth using photo-overlays, with links to video clips that pop up on the screen. You can spin and rotate the globe at any size, and zoom into the pictures. The above photo and the video clip show the application on a NextWindow Human Touch large-screen display.

This application would be great on a touch-table or touch-table set up on a drafting board. Although it was designed for a travel-planning application, it would work well in educational settings in subjects such as geography.


Poetry Picture Share

This was my first attempt at a "poetry picture share" application. It was designed for eventual use on a multi-touch table. It was developed using JavaScript and Ajax. It could be accessed remotely so people in different places can move things around on the screen. The video shows how the application works on a large interactive touch-screen display.

I am planning on adapting this application for use with students with special needs, such as those who have autism or other communication disorders. (Note: I've used it with several students, with success. It still is a work in progress!)



Update:

Link to Papervision 3D: Press the picture to enter a 3-D interactive underwater world...

If you have tried PaperVision 3D or Tag Galaxy on an interactive whiteboard or touch screen display, please leave a comment and share your impressions.

Interactive Data Visualization Online: NY Times Examples

All of Inflation's Little Parts

This interactive data visualization can be found in the on-line version of the New York Times.










This is what you see in the zoom view.













For those of you interested in sports data, here an interactive chart about Tiger Woods:

How Woods Wins a Major







If you look closely, the data that relates from the various years is brushed to a fader shade. By clicking on various years, you can look at patterns and trends in the data, and see how Tiger did in the various areas.












Links via Ranjit S. Mathoda: When will blogs start presenting data using interactive visuals alongside their stories?

Update 6/3/08: Another InfoGraphic from the New York Times, via Randy Krum:

Low-tech Interaction: Moving Sand Pictures

A father and son team in my region create furniture, jewelry, and beautiful moving sand pictures. These aren't computer displays, they are the real thing!

Liquid mixes with the sand as you turn the display upside down. Take a look at the video to see it in action. For more information, visit Rosey's Unique Products.