Top 15 Posts: Interactive Multimedia Technology

As of 5/31/08
Interactive multimedia for social skills, understanding feelings, relaxation and coping strategies
Engaged Learning Revisited: Four videoclips for reflection....
UMAGINE Creative -Digital Story Telling for Interactive Whiteboard or Touch Screen- free demo available
Interactive Multimedia Technology: May 2007
Dance.Draw Project : Exquisite Interaction - Collaboration between Software Information Systems -HCI- and Dance Department
Microsoft Surface multi touch screen table - I wish I had one for my projects last semester!
Programming for interactive multimedia applications: WPF, Silverlight, EduSim, NeuroVR - Musings: Learning about application development and programming at mid-life
Interactive Information Visualization: Digg Labs Website
iPod Touch Possibilities... I want one!
NASA's promotion of MMO games to support STEM learning; Space Station Sim game; EASe games for children
Interactive Multimedia Technology for Science and Math
More Multi-touch: Microsoft Windows 7 Demo on a Dell Laptop
More multi-touch from members of the NUI group!
My Nokia n800 Internet Tablet!
NUI Group Member Bridger Maxwell Receives High School Science Fair Awared for Multi-touch Screen Project

Umajin Creative -Digital Story Telling for Interactive Whiteboard or Touch Screen- free demo available

Umajin Creative is an application designed for digital storytelling. I haven't had a chance to preview it. I was impressed with the pictures on the website. If you use this application, please leave a comment. It looks kid and teacher friendly. I can see that it has potential!

According to the website, you can "compose multi-page digital documents with rich text, digital photography (including blue screen support), illustrations, sound, video, 3D models, particle fx, interactive functionality, and so much more... runs on both a Mac and PC. It also supports interactive whiteboards and touch screen PC's.. so you can interact directly with the content. With the HP Touchsmart PC you can use real brushes on the screen..to experience digital painting with variable width brush strokes!

Interactive Digital Storybook:


Below: Cool-looking digital brushes.

Above- Sample 3D models
Below- Sample of blue-screened cutout images

Below: Samples of animated particle effects

Seven Custom Functions:

I think the people from Fingertapps are responsible for UMAJIN. Below is a video highlighting multi-touch applications:

Windows 7 Demo on a Dell laptop: More Multi Touch

Video: Multi-Touch in Windows 7

Via SoapBox, Greenbush Labsand CNET

If you are new to this blog and would like to learn more about multi-touch interaction and technology, enter a keyword in the search box.

Also take a look at the Technology Supported Human-World Interaction blog.

For multi-touch DYI, check out the NUI Group!

Wet Sounds 2008: A Festival of Underwater Sounds- I want to go!

I was sitting at the kitchen table, looking at the paperwork I need to finish, and daydreaming about what I want to do this summer, when I came across information about Wet Sounds 2008: A Festival of Underwater Sounds. If you visit the website, move your curser, (or finger, if you have a touch screen) around the page. Bubbles float around, as if you are under water.
Unfortunately, the nine pools hosting Wet Sounds are in England.

WetSounds is sponsored by NewToy, a company that is involved in sound design, films, theater, new media, art installation, video art, dance, and radio.

"The audience dips in and floats around, ears submerged, absorbed in sounds coming from underwater speakers".... "Sound travels 4 times quicker than in air, due to the water's density, it is perceived not only by the ears but also by the bones and body. The sensation is similar to a floatation tank effect. The activity promotes relaxation, listening, openness to sound art and enhanced audio perception".

Phoenix Mars Mission Website

The following is part of a related post from the TechPsych blog about the Phoenix Mars Mission:

If you are an educator with access to an interactive whiteboard and can integrate a visit to Mars into your lesson plans during the last days of the school year, a trip to the Phoenix Mars Mission website is a must! The website is well designed and user-friendly.

Renderings of the Phoenix Mars Lander

Here are few places to start:

Phoenix Mars Mission News
Web Exhibit -Mars: The Search for Water, the Search for Life
Flash Video Stream (the video has relaxing ambient music, by the way)
Images
Videos and Animations
Just for Kids - this site was designed with content by kids to share with others.

From Mars to Earth: An Interactive Timeline

Enjoy!

Virtual Architecture, Virtual Archeology

It is interesting to see how interactive technology is cutting across so many disciplines!

Raising the Titanic

Ken Rigby's company, MellaniuM, developed the plugin for this project. To learn more about this project, check out Electric Archaeology: Digital Media for Learning and Research, and the related links:

Shawn Graham, Archaeologist
Agent Based Modeling in Archaeologists
http://electricarchaeologist.wordpress.com/2008/04/22/autocad-into-unreal2/

AutoCad Plugin for Unreal
http://mellanium13.blogspot.com/2008/02/mellanium-virtual-architecture.html

Dance.Draw Project : Exquisite Interaction - Collaboration between Software Information Systems -HCI- and Dance Departments at UNC-Charlotte

DANCE.DRAW: EXQUISITE INTERACTION
(Updated)

"The movement of the visualizations are artifacts in real-time of the movements of the dancers. They draw while they dance, they dance together and they draw together. Every performance generates a new visual imprint." -DanceDraw website

Interactive multimedia technology, blended with the arts!

Dr. Celene LaTulipe, from UNC-Charlotte's Software and Information Systems Department, Professor Sybil Huskey, from the dance department, dance students, and others collaborated to create an amazing performance that I had the opportunity to see performed during the Visualization in the World Symposium in April (2008).

If you look closely, you will see that each dancer holds two wireless mice, one in each hand. The mice trigger the visualization that is projected in the background. Dr. LaTulipe has focused some of her research on two-handed computer interaction. It is interesting to see how her work has been applied to this beautiful "off-the-desktop" application.

Dance.Draw is a work in progress- visit the following links for more information:

Website (Updated)
Movie
Technical Info
Dr. Kosara's Eager Eyes post about Dance.Draw

Note:
Dr. LaTulipe was my HCI professor- Dr. Kosara was my Visualization/Visual Communication professor.

Game Based Learning: Second European Conference

The 2008 Second European Conference on Game-Based Learning will be held in Barcelona, Spain, October 16-17, hosted by the Universitat Oberta de Catalunya.

Speaker Bios

Conference Program

If you are working in a K-12 setting and interested in sharing your ideas about game-based learning, please leave me a message. I'm especially interested in how interactive games support engaged, meaningful learning.

Engaged Learning Revisited: Four videoclips for reflection....

Last October, I shared a couple of videos to highlight a discussion about engaged learning in a post on my TechPsych blog. Today, I received a thoughtful comment from a teacher who was moved by the videos. I thought that I'd recycle that post and add two other videos that provide related messages in ways that can not be conveyed effectively by words alone:

From the October TechPsych post:
Those who follow my blogs know that I usually focus on the positive- engaging technologies, exciting research, interesting websites, and successes of others worth sharing. Today I came across a video that got me thinking about the importance of engaged learning, a topic I've written about in the past.

Much effort is extended in our high schools to prepare students for the "real" learning that will take place in college. With the increased emphasis on testing, it seems like the moment a child enters kindergarten, one of the main goals is to prepare for learning the skills needed in the following grade. Every student must be "ready".

Ready for....this?


In today's digital world, traditional word-based instruction doesn't always result in experiences that engage the hearts and minds of young people.

The video below was created by students involved in the Digital Ethnography group at Kansas State University. Michael Wesch is the professor behind this group. At the beginning of the video, the camera slowly peers around an empty university lecture hall, with the the following quote superimposed over the scene:

"Today's child is bewildered when he enters the 19th century environment that still characterizes the educational establishment where information is scarce but ordered and structured by fragmented, classified patterns, subjects, and schedules." -Marshall McLuhan, 1967

Although the quote is now forty years old, the message communicated in this video is clear. Despite technology, things haven't really changed, as traditional methods of teaching tend to disengage rather than engage a good number of students.



After viewing the above video, watch this video, "When I grow up I want to be a teacher", a parody of a Monster.com commercial.



For a broader perspective, take some time to reflect on the following video, originally created by Karl Fisch, and posted to his blog, the Fischbowl:
"A staff development blog for Arapahoe High School teachers exploring constructivism and 21st century learning skills. The opinions expressed here are the personal views of Karl Fisch - and various other teachers at Arapahoe - and do not (necessarily) reflect the views of Littleton Public Schools."

DID YOU KNOW? SHIFT HAPPENS


"DEBUNKING THE MYTHS OF THE THIRD WORLD" is Hans Rosling's presentation at TED 2006 is a great example of the use of presenting information supported by engaging information techniques. More videos are on the Gapminder website. This video is about 20 minutes long, but worth every second.



"With the drama and urgency of a sportscaster, Prof. Hans Rosling uses software from Gapminder debunks a few myths about the "developing" world. This global health visionary has discovered a powerful new way to communicate complex data about the world; his remarkable interactive graphs help deliver profound insights about global trends and will change forever the way you think about "us" and "them." Rosling is professor of international health at Sweden's Karolinska Institute, and founder of Gapminder, a nonprofit that brings vital global data to life."

Cross Post: One Laptop Per Child's Redesign- Dual Touchscreens, Flexibile Use

Photo from One Laptop per Child via MIT Technology Review

"Hundred-dollar laptop, revisited: The next-generation version of the One Laptop per Child machine will dispense with keypads. It can be folded flat to make one larger screen (left); here, two children could play a game, each using the touch-screen capability. Or it can be held on its side and used as an electronic book (right)."

I'm impressed with the new design of the OLPC, the dual touch screen, the support of collaboration and sharing between children, and the flexibility it will provide educators and students. It can even be used as an e-Book! This laptop would be welcomed in UDL classrooms.

For detailed information about the new OLPC laptop, which has not yet been released, read David Talbot's article in the MIT Technology Review.

I want one.

NUI-Group Member Bridger Maxwell Receives High School Science Fair Award for Multi-Touch Screen Project

Yet another post about a NUI group member... Bridger Maxwell, a high school student at the Utah County Academy of Sciences, submitted his multi-touch screen to the science fair, and went on to win first place in the engineering category, and now will be competing in the International Science and Engineering Fair.





Bridger has created "Lumen", a puzzle game for OSX and Windows, and markets this through his business, Fiery Ferret.

Even More Multi-Touch, Delivered by NUI...

May 20, 2008
NUI announces delivery of their multi-touch solution for Cityscape, Abu Dhabi 2008





www.natural-ui.com
Natural User Interface Europe Ltd.

More Multi-Touch from members of the NUI group!

It is always exciting to see what members of the NUI group are doing!

Here is a new video of a multi-touch creation by some of the members of the NUI group. Although this is a proof-of-concept example, it is fun to see how it is played out, using the little iPhone-like touch-pad widgets as a navigation tool for the large screen.


Read the "Multi-touch Goodness" article in Gizmodo of an interview with Christian Moore about this demo and his Lux open-source framework. (Christian is a colleague of Harry van der Veen, both members of the NUI group.)

Here is an excerpt from the interview:
"JD: Why Flash?
CM: Because it's fast to prototype in. However, the software is broken into several segments. One C++ application that tracks hands that talks to Flash... WPF... or another C++ app... and basically everything you can imagine. You can enable multitouch in any environment, like Cocoa."

High-resolution screen shots and additional information can be found on the nuiman website.

For my tech-minded readers:
I'm pretty sure that the C++ application that track hands and fingers in the video demo uses Touchlib, a library for creating multi-touch interaction. Touchlib can work with TUIO, a protocol for tabletop tangible user interfaces. Applications such as Flash and Processing support TUIO. For more information about TUIO, read "TUIO: A Protocol for Table-Top Tangible User Interfaces".
(Information from the NUI group website mentions that OpenCV, or Open Computer Vision Library, found on SourceForge, can support blog detection and tracking.)

The people behind TUIO are from the Reactable project, of the Music Technology Group at Pompeu Fabra University in Barcelona:

Here is my plug for the NUI group, once again!

"The NUI group, or Natural User Interface Group, is an interactive media group researching and creating open source machine sensing techniques to benefit artistic and educational applications.

We offer a collaborative environment for developers that are interested in learning and sharing new HCI (Human Computer Interaction) methods and concepts. This may include topics such as: voice/handwriting/gesture recognition, touch computing, computer vision, and information visualization."


FYI
I came across the Harry van der Veen of the NUI group in early 2007 when I was working on touch-screen projects for my HCI and Ubicomp classes, and I'm inspired by all of the creativity I've seen coming from this group.

If you'd like to see more demos, visit the Natural User Interface website, a commercial out-growth of Harry and his colleague's work, where you can view a reel that includes a few touch-screen games. I love the vision statement on this site:

"Technology should enable us to interact with computers, in the same way we interact with the real world; in a way which is natural to us, namely through gestures, expressions, movements, and manipulations. Our vision is to change the way people interact with computers."

Multi-touch Crayon Physics

Also posted on the TSHWI blog:
Watch how you can draw simple shapes that can instantly turn into a game!


Multitouch Crayon Physics from multitouch-barcelona on Vimeo.
For a better version of this video, see http://www.vimeo.com/980528 For more information, see the RXSurface blog post. On May 18, Multi touch crayon physics will be offered as an alphabeta opensource! The people behind RXSurface are members of the Natural User Interface (NUI) group:
"Natural User Interface or ~ NUI Group is an interactive media group researching and creating open source machine sensing techniques to benefit artistic and educational applications."

Virtual Math Museum: Online Math Art:

The Virtual Math Museum has a wealth of visual information useful to those in and outside the field of math. The mathmatical art section has links to the works of Paul Nylander, George Hart, Titia van Beugen, Brian Johnston, Bathsheba Grossman, Jos Leys, Paul Bourke, Jean Consant, and Luc Benard.

Each of the Math Artist pages provides information about the artist that in my opinion, would be useful to teachers who aim to encourage students to consider pursuing STEM-related careers. (Science, Technology, Engineering, Math).

The content of the entire Virtual Math Museum website is impressive on a large display or interactive whiteboard!


FYI:

"3D-XplorMath Home Page

The group in charge of the 3D-XplorMath software development project and the related Virtual Mathematics Museum website project is the 3DXM Consortium, an international volunteer group of mathematicians. The Consortium gratefully acknowledges ongoing support for these projects by The National Science Foundation (DUE Award #0514781) and is grateful to the Mathematics Department of The University of California at Irvine for hosting the 3D-XplorMath and Virtual Mathematical Museum websites."

Paul Bourke's Experiments in Interactive Visual Immersion in Second Life - and more....

"Preprocessed spherical projections from inside a crystal including the Hershfield surface"
Evaluating Second Life as a Collaborative Tool for Scientific Visualization
http://local.wasp.uwa.edu.au/~pbourke/papers/cgat08/

I thought I'd devote a post to Paul Bourke's exciting work in the area of what I call "interactive visual immersion".

According to a bio on a math art website, Paul's working life "has revolved around high performance computing and computer graphics, in particular scientific visualisation: the use of computer graphics to represent scientific data with the aim of enhancing understanding of the underlying structure and processes." Paul is also interested in the use of "novel display technologies that can further enhance the understanding of large and complicated geometric datasets...These engage capabilities of our visual system not exploited by traditional computer displays..."

Paul recently participated in the the Computer Games, Multimedia & and Allied Technology 08: International Conference & Industry Symposium on Computer Games; Animation, Multimedia, IPTV & Edutainment. (CGAT'08, Singapore, April 28-30)

The above picture is from his paper, Evaluating Second Life as a Tool for Collaborative Scientific Visualization. (pdf), taken from Paul's webpage, where you can find a link to the slides from his talk.

Bourke points out that there are few software tools available that support shared, remote collaborative scientific visualization. What is available is expensive, exploratory, and/or difficult to install and operate. Bourke proposes that Second Life might prove to be a useful tool for collaborative SciViz, despite some current limitations. Second Life is a cross-platform 3D environment that was designed specifically for interaction between many people, and it allows for texture mapping, which is a plus for visualization work. Burke notes that there are limitations, such as Second Life's low geometric complexity.

Paul Bourke's website includes a page that lists all of his papers, articles, and seminars, with impressive visuals alongside each entry. Since I recently completed a visualization class and also gave a short presentation about accessible games for health, I found many of Bourke's papers intriguing:

Vertical dome (iDome): Visualisation and Navigable movies
Visualization for scientists, museums, public outreach, and education
Immersive environments and applications to gaming
Exploiting our sense of touch for scientific visualization.
Tactile Visualization: Feel your data!
Apple technology powering displays that engage the human visual system
iDome and digital projection into hemispherical domes

Take the time to explore Bourke's entire website. Look at his texture library and his "other" section!

FYI: Here is a link to the CGAT'08 list of speakers, complete with bios and abstracts.

Seth Sandler's "How to Make a Cheap Multitouch Pad" YouTube video is going viral...

Seth Sandler, a member of the NUI group, has worked very hard at putting together low-cost multi-touch screen surfaces. Seth has a background in music, so his projects focus on interactive music applications. Over 315,000 people watched this video within the week that it was uploaded.

Thanks Seth, for sharing this vision with the world!



Seth's AudioTouch Blog: "An Interactive Multi-user, Multi-touch Musical Table and More"

Hint for high school teachers: This sort of thing would be a great project for an after-school technology club!

Two Links: Urban Screens, Urban Interfaces, Digital Media, and the Arts in Social-Public Spaces; Edward Tufte is a Smart Guy

Here are two links that I think you'll find interesting:

My post on the TSHWI blog, reflecting on interdisciplinary research and topics related to ubiquitous computing and large interactive displays in public spaces:

Urban Screens, Urban Interfaces, Digital Media, and the Arts in Social-Public Spaces


Bill Mackenty's reflections about workshop he attended that was led by Edward Tufte, known for his insights about the presentation of information graphics:

Edward Tufte is a Smart Guy

Cross post via Tech Psych: Games for Health conference

I recently presented at the Games for Health conference in Baltimore, Maryland. The attendees and presenters at the conference came from a variety of fields- game development, education, occupational and physical therapy, health, bio-mechanical engineering, media arts, and more, all interested in sharing ideas about using game, simulations, and virtual worlds to improve health.

Many of the topics covered during the recent Games for Health conference apply to K-12 settings. If you think about it, many children and teens are at risk for health problems that will adversely impact their lives as adults. If we can provide a means for young people to develop healthy behaviors and attitudes at an early age, we will help to ensure healthy futures, for individuals and communities alike.

If you have a little time, you can listen to the overview provided in an audiocast consisting of interviews with Ben Sawyer and others involved with the conference.

My presentation slides - Game Accessibility and Health Education in K-12 Settings - from the pre-conference, are posted SlideShare, where you can find slides from other presentations on the Games for Health group section.

You can find more information on the Pioneer Portfolio of the Robert Wood Johnson Foundation website, including a section about Games for Health.

Edutopia Video: No Gamer Left Behind: Virtual Learning Goes to the Next Level

The following video is from from Edutopia, part of the George Lucus Educational Foundation. It provides a good overview of the at the use of virtual worlds and interactive games in education. Some high schools provide virtual labs to teach science concepts, and others offer courses that teach students how to create games and virtual environments, and in turn, these student learn art, story-telling, math and computer programming concepts.

Teachers report that incorporating virtual learning in their classrooms increases student engagement and academic performance.

This video clip features Doug Whitley,from Kurt Squires and William Shafer, from the University of Wisconsin, students from the Digital Media program at McKinley High School (Washington D.C., and Eric Johnson, from Lucas Arts.



According to Mark Prensky, there are many barriers to the use of simulations and games in education, inlcuding the lack of money, time, and adequate technology. In addition, many educators are not aware of the power that interactive simulations hold for their students.

Oakland Blues: Virtual Preservation of Seventh Street's Jazz Scene, more about educational gaming MMO's

I recently attended the Visualization in the World Symposium sponsored by the Charlotte Visualization Center at UNC-Charlotte. I enjoyed the symposium because the presentations were interdisciplinary, drawing from fields such as psychology, architecture, journalism, scientific visualization, data visualization, and photography.

The audience consisted of people from a variety of disciplines. This added dimension to the discussions after each presentation, eliminating the "birds of a feather" feeling experienced at conferences that are narrow in focus.

Yahuda Kalay's presention:



One of the presentations was about the on-line interactive virtual preservation of Oakland California's 7th Street from the 1950's, a center of jazz at the time. The project was the result of collaboration between the Architecture and Journalism departments at the University of California-Berkeley. Preserving cultural heritage is important, but proves to be a difficult task. Digital media can assist with this problem through the use of modeling and visualization.

Because the project was designed to be used by people via the interent, decisions needed to be made regarding the quality of the graphics, since many users would not have high-end graphics cards installed in their computers. Garage Game's Torque engine was used to develop the game, because it contained a physics engine and also supported players and non-player characters (NPC), or virtual characters. The NPC's were programmed to provide interactive dialogues with players, and each NPC's dialogue contributed to telling the story of 7th street.

Below is the course description from a recent journalism class at UC-Berkeley that used the Oakland's 7th street videogame for many of the course assignments. It looks like it could be easily adapted for high school courses:

"J-298: Oakland Jazz and Blues (Spring 2008)This class is using a video game program to recreate and tell the story of the jazz and blues club scene on Oakland's 7th Street during its heyday in the 1940s and 1950s - a remarkable part of the city's history that has been all but lost to urban decay. An eight-block stretch of 7th Street is being recreated as a virtual world, which people can access over the Internet and then adopt avatar figures to walk up and down the streets, enter the clubs, listen to the music of the era and interact with other people logged onto the site. The virtual reality program used in the class was developed by the UC Berkeley Architecture Department, which is collaborating on this year-long project. This class involves reporting and research on the stories of the clubs and other establishments on 7th Street, the musicians and other characters who frequented the scene, the music played in the clubs, and the redevelopment projects that destroyed the area. And the class will work on how to tell the story of the clubs and the history of the area using video game narratives."

Further Reading:

OAKLAND BLUES. Virtual Preservation of Seventh Street's 1950s Jazz Scene by Yehuda E. Kalay and Paul Grabowicz, Center for New Media, University of California, Berkeley, USA JISC 3DVisA Bulletin, Issue 1, September 2006

RELATED: New Media Consortium

The New Media Consortium's overview of interactive virtual worlds used for education provides information about a variety of projects and the time-frames in which they will become fully implemented. It also provides an overview of massively multiplayer educational gaming, with resources for further reading. The NASA MMO project to support STEM learning is an example massively multi-player educational gaming.

I am excited about these innovations, since they will provide visual learners with educational opportunities that are suited to how they learn.

NASA's promotion of MMO games to support STEM learning; Vision-play's SpaceStationSim game; EASe games for children with autism spectrum disorders

NASA is looking for a partner to develop a massively multiplayer online learning game to support education in science, technology, engineering, and mathematics, known as STEM. The following is a quote from NASA's website regarding the advantages of providing learners with MMO games:

"Persistent immersive synthetic environments in the form of massive multiplayer online gaming and social virtual world, initially popularized as gaming and social settings, are now finding growing interest as education and training venues. There is increasing recognition that these synthetic environments can serve as powerful “hands-on” tools for teaching a range of complex subjects. Virtual worlds with scientifically accurate simulations could permit learners to tinker with chemical reactions in living cells, practice operating and repairing expensive equipment, and experience microgravity, making it easier to grasp complex concepts and transfer this understanding quickly to practical problems. MMOs help players develop and exercise a skill set closely matching the thinking, planning, learning, and technical skills increasingly in demand by employers. These skills include strategic thinking, interpretative analysis, problem solving, plan formulation and execution, team-building and cooperation, and adaptation to rapid change."

NASA's Request for Proposals document outlines the specifics for game developers who'd like to partner with NASA on this project. The MMO's target audience is teens in highschool and above, with adoption expected at the middle school level. Partners should be know how to make the game accessible to people with disabilities.

SPACESTATIONSIM

If you like games about space, or know young people who do, you might be interested in Vision-play's SpaceStationSim, which was developed in collaboration with NASA. For more information, you can visit the Vision-play website, where you can find an on-line manual for the game, screenshots, and a free demo.

EASe GAMES

Vision-Play also created four games for use with children who have autism spectrum disorders, building on the EASe CD series used by some occupational therapists to help with auditory hypersensitivity, hyperacusis, central auditory processing disorders, or sensory integration disorders. The EASe games "not only fun to play, but stimulate a child’s auditory/vestibular and visual/balance sensory inputs, and help teach them to manage noise and regulate balance." EASe games allow for three speed settings. It is not clear if they are switch-adaptible.

It will be interesting to see how these games play out in school settings!