MICOLE: Open source multi-modal, multi-sensory software

Here is a quick link to the following post on the TechPsych blog, via ICT Results

MICOLE: Open-source multi-modal software supports cooperative learning among sighted and visually impaired children

The new 3G iPhone: Expanding the Possibilities of Interactive Multimedia Communication

The new iPhone is filled with possibilities....In this post, I'm sharing some pictures of the iPhone that look interesting to educators and those of us who want to increase our interactive multimedia communication and entertainment when we are out and about.

The following pictures from a post I came across on the engadget website today, authored by Ryan Block, while he attended today at WWDC 2008. The new 3G iPhone, with a price of $199.00, was the focus of Steve Job's keynote. I've added a few links to the developers of new iPhone apps, where possible.


3G is fast



Apple offers MobileMe service for the iPhone, which is something most people on-the-go will appreciate:

"Everything you need. Anywhere you are."

"MobileMe stores your email, calendar, and contacts on a secure online server, or “cloud.” The cloud pushes the most current data to your iPhone, iPod touch, and computer so you’re always up to date. And your email, calendar, contacts, photos, and documents are accessible over the Internet through a set of easy-to-use web applications."

Push email, push calendar, and push contacts Web applications at me.com

MobileMe Gallery for photo sharing MobileMe iDisk for online document access

20GB of online storage

This would make the iPhone ideal for mobile learning! Here is an example of an application for science:





Great for musicians and music students who are away from their gear:


According to Ryan Block, this application is from Moo Cow Music. The application is called "Band". Here are the features, as listed on the Moo Cow Music website:

• Drums, bass, guitars, keyboards can all be played and recorded into one song from the same application.
• Multitouch - press up to five keys at once for complex chords.
• Animation shows keys being pressed, strings being struck, etc.
• Any instrument can be recorded, or overdubbed onto a previous recording.
• Erase any mistakes, then go back and overdub replacement notes.
• A configurable metronome can be enabled or disabled during recording.

Here is a video of the beta version of the application:

"Kroll", from Digital Legends Entertainment, runs on the iPhone and is an action-adventure fantasy game.


Sega's Super Monkey Ball on the iPhone


Apple iPhone Site

Link to video of Keynote


Verizon, can you hear me now? I want an iPhone!

Engaged Learning and Social Physics: Phun, an Interactive 2D Physics Sandbox

UPDATE 4/24/11:   Phun is commercially available and now optimized for use on the newer multi-touch, multi-user SMARTboards.  It is now known as Algodoo.

For those of you searching for interesting and engaging science activities for use on an interactive whiteboard or display, take a look at Phun. According to the website, "Phun is a playground for the creative mind where machines and toys can be easily created."

It was Emil Ernerfeldt's MSc Computer Science project, created at the Umeå University VR lab in Sweden, under the supervision of Kenneth Bodin.

The Phun application can be downloaded for free. The video provides a good overview of Phun's features, accompanied by soothing music that can you can download at no cost at http://phun.cs.umu.se/files/Phun_theme.mp3


From the creators of Phun:
"Phun is an educational, entertaining and somewhat (!) addictive piece of software for designing and exploring 2D multi-physics simulations in a cartoony fashion. It is part of our long term mission to bring visual physics based simulation to the masses. The application is developed for Umevatoriet, Umeås new science center, where it will run on a large interactive display, but you can also download it and run it on your own pc."

The beauty of Phun is the social aspect. Physics 2.0. There is a YouTube Phun group, where people share videos of their Phun creations. The Phun wiki provides a tutorial, a forum, and links. Phun-makers can share and rate their creations in the Phun Box.

Note: Phun is now part of Algoryx Simulation AB, a spin-off from Umea University. Algoryx focuses on interactive 3D multiphysics simulation. The free version of Phun will continue to be updated and supported, according to Emil Ernerfeldt.

Related Links:
Phun at Algoryx
Forum Thread: How Phun can be used in education
http://phun.cs.umu.se/wiki
Download Page (Mac, Win, Linux)

Emerging Interactive Technologies, Emerging Interactions, and Emerging Integrated Form Factors

Every so often I come across a blog that has been around for a while, but is a new discovery to me. If you are interested in interactive multimedia and emerging technology, and you haven't heard of Richard Banks, you will probably be pleased with his vast array of interesting posts!

After browsing around Richard Bank's blogs, I found that he works for Microsoft Research. Like me, he maintains more than one blog, and he blogs about similar topics.

Here is a view into his world:

rb.trends: tracking future technology experience
rb.work: technology, design and research
rb.log: family, photography, architecture etc.

One of Richard's recent posts brought me back to my musings about interactive surfaces:

Via Richard Banks and Gizmodo: A Touch Screen Poker Table from Hard Gaming

If it is possible to program an application to handle the metrics that support poker, then it is possible to program a multi-user application for work or education, and have this application integrated into any type display, according to need.

The form factor above would be useful for team meetings or collaborative project planning, especially during group decision-making or policy planning activities that involve data analysis or information visualization.

Could this concept be modified to fit the form of a kidney table for use in education? Think about it. We could combine the best features of surface computing with the best features of multi-user video games.

Rationale? Video-game applications can handle the metrics of multi-user interaction, which would be ideal for tracking student progress and interaction. Touch interaction would enable young children with an easy way to access the application- or all of the information that will be available on the multi-touch web of the future, without learning first to type.

(Touch and gesture interaction could help us move towards the implementation of the Internet of Things)

Could this concept be transformed for use by two people collaborating on an electronic drafting table?




If you are a NUI Group member, think about this!


Note: I came up with the phrase, "multi-touch web of the future" as I was typing this post. I guess I didn't invent it - I looked it up and found Andrew Foote's post, "The Multi-Touch Web"

I'm in.


Somewhat Related.......

More for the tech-curious:

Interactive data visualization in Second Life

| View | Upload your own

Idle together "Technology blog oriented towards good design and impressive web applications"

Apple Developing Full Multi-Touch Macs - Apple Insider

More for the tech-savvy:

Multi-touch gestures in the Factor UI (Apple)- by Pestov. Includes code.

NSResponder Modifications: Swipe, Rotate, and Magnify, by Elliott Harris, a member of the NUI Group, I think.

Web Development Guidelines for iPhone (lots of good info, with video clips and code)

Web 3D Consortium

Unity 3D kit for the

Google's Android

Multi-touch Web Consortium (in my dreams)

Gesture Web Consortium (in my dreams)

New Interactive Technology: Tag Galaxy, Windows 7, PaperVision 3D, Microsoft Surface, Touch Screens, and Blog Posts Revisited

Steven Wood's Tag Galaxy
Via Papervision3D, Richard Banks, and Flowing Data

Tag Galaxy is an application that uses Flickr and Papervision 3D. If you type in a tag, the tag is represented as the sun in the galaxy, and related tags are represented as planets. When you rotate and browse through the galaxy in 3D space, you can view the pictures according to the tag. Steven Wood created Tag Galaxy for his thesis project.

Separated at Birth?
Vacation pictures on an interactive photo-globe - April 2007

Just over a year ago, I finished some class projects that were designed for touch screens or tables. At the time, I had no idea that Microsoft was working on the Surface. I was determined that I'd make my own table if I couldn't get my hands on one, and even joined the NUI Group to learn more about the open-source DYI approach to satisfying my fascination with this interactive technology.


The first day I learned about Microsoft Surface will be etched in my memory forever...


It is not because I'm a 100% Microsoft fan. It is because I'd been thinking about touch-screen interaction since my first encounter with an interactive whiteboard several years ago. At the time, I was involved with group counseling with middle school students, seated around a table.

What would happen if we took the whiteboard put it on a table?

As noted below, the demo projects worked best on a NextWindow Human Touch. Although the large display was only "single-touch", it provided excellent resolution and touch response. Since then, NextWindow has come out with a multi-touch display, which I've had a chance to see. It is not a table, but it has possibilities.

What would happen if we took a NextWindow multi-touch display and mounted it onto an adjustible drafting board? It could be used as a vertical display, a "surface" table, and a drafting board. The best thing is that this could support quite a bit of collaborative work between two or more people, as well as learning, creative-artistic, and gaming activities.

New things are on the horizon. Windows 7, will replace Vista in the future. Bill Gates has pronounced that "every surface will be a computer". Touch screen surfaces are finding homes on tablet PC's, cell phones, and the latest OLPC "$100.00" laptops for children.

I think there will be more possibilities for using touch-screen technology for education in the very near future.

Here is my post about Microsoft's Surface from last year, with some updates:

Microsoft Surface multi touch screen table - I wish I had one for my projects last semester!

Microsoft Research recently unveiled Microsoft Surface: http://microsoft.com/surface. This multi - touch table can be used for a variety of applications, as outlined in the video from CNET and YouTube below:



I'd like to work on applications for use on a touch-table to support students with special needs, especially those who have autism spectrum disorders.

.....Last semester, I worked on prototype applications for use on a touch-screen surface -here is a photo. We used a NextWindow Human Touch large-screen display, which provided great screen resolution and touch-response.



This application was part of a travel-planning prototype developed for a course in Human Computer Interaction. The application was demonstrated on a NextWindow Human Touch large screen display. Would it work on the iPhone?







Update: Examples from some of my other posts:

Below is another demo video-clip of a globe created in GoogleEarth using photo-overlays, with links to video clips that pop up on the screen. You can spin and rotate the globe at any size, and zoom into the pictures. The above photo and the video clip show the application on a NextWindow Human Touch large-screen display.

This application would be great on a touch-table or touch-table set up on a drafting board. Although it was designed for a travel-planning application, it would work well in educational settings in subjects such as geography.


Poetry Picture Share

This was my first attempt at a "poetry picture share" application. It was designed for eventual use on a multi-touch table. It was developed using JavaScript and Ajax. It could be accessed remotely so people in different places can move things around on the screen. The video shows how the application works on a large interactive touch-screen display.

I am planning on adapting this application for use with students with special needs, such as those who have autism or other communication disorders. (Note: I've used it with several students, with success. It still is a work in progress!)



Update:

Link to Papervision 3D: Press the picture to enter a 3-D interactive underwater world...

If you have tried PaperVision 3D or Tag Galaxy on an interactive whiteboard or touch screen display, please leave a comment and share your impressions.

Interactive Data Visualization Online: NY Times Examples

All of Inflation's Little Parts

This interactive data visualization can be found in the on-line version of the New York Times.










This is what you see in the zoom view.













For those of you interested in sports data, here an interactive chart about Tiger Woods:

How Woods Wins a Major







If you look closely, the data that relates from the various years is brushed to a fader shade. By clicking on various years, you can look at patterns and trends in the data, and see how Tiger did in the various areas.












Links via Ranjit S. Mathoda: When will blogs start presenting data using interactive visuals alongside their stories?

Update 6/3/08: Another InfoGraphic from the New York Times, via Randy Krum:

Low-tech Interaction: Moving Sand Pictures

A father and son team in my region create furniture, jewelry, and beautiful moving sand pictures. These aren't computer displays, they are the real thing!

Liquid mixes with the sand as you turn the display upside down. Take a look at the video to see it in action. For more information, visit Rosey's Unique Products.

Top 15 Posts: Interactive Multimedia Technology

As of 5/31/08
Interactive multimedia for social skills, understanding feelings, relaxation and coping strategies
Engaged Learning Revisited: Four videoclips for reflection....
UMAGINE Creative -Digital Story Telling for Interactive Whiteboard or Touch Screen- free demo available
Interactive Multimedia Technology: May 2007
Dance.Draw Project : Exquisite Interaction - Collaboration between Software Information Systems -HCI- and Dance Department
Microsoft Surface multi touch screen table - I wish I had one for my projects last semester!
Programming for interactive multimedia applications: WPF, Silverlight, EduSim, NeuroVR - Musings: Learning about application development and programming at mid-life
Interactive Information Visualization: Digg Labs Website
iPod Touch Possibilities... I want one!
NASA's promotion of MMO games to support STEM learning; Space Station Sim game; EASe games for children
Interactive Multimedia Technology for Science and Math
More Multi-touch: Microsoft Windows 7 Demo on a Dell Laptop
More multi-touch from members of the NUI group!
My Nokia n800 Internet Tablet!
NUI Group Member Bridger Maxwell Receives High School Science Fair Awared for Multi-touch Screen Project

Umajin Creative -Digital Story Telling for Interactive Whiteboard or Touch Screen- free demo available

Umajin Creative is an application designed for digital storytelling. I haven't had a chance to preview it. I was impressed with the pictures on the website. If you use this application, please leave a comment. It looks kid and teacher friendly. I can see that it has potential!

According to the website, you can "compose multi-page digital documents with rich text, digital photography (including blue screen support), illustrations, sound, video, 3D models, particle fx, interactive functionality, and so much more... runs on both a Mac and PC. It also supports interactive whiteboards and touch screen PC's.. so you can interact directly with the content. With the HP Touchsmart PC you can use real brushes on the screen..to experience digital painting with variable width brush strokes!

Interactive Digital Storybook:


Below: Cool-looking digital brushes.

Above- Sample 3D models
Below- Sample of blue-screened cutout images

Below: Samples of animated particle effects

Seven Custom Functions:

I think the people from Fingertapps are responsible for UMAJIN. Below is a video highlighting multi-touch applications:

Windows 7 Demo on a Dell laptop: More Multi Touch

Video: Multi-Touch in Windows 7

Via SoapBox, Greenbush Labsand CNET

If you are new to this blog and would like to learn more about multi-touch interaction and technology, enter a keyword in the search box.

Also take a look at the Technology Supported Human-World Interaction blog.

For multi-touch DYI, check out the NUI Group!

Wet Sounds 2008: A Festival of Underwater Sounds- I want to go!

I was sitting at the kitchen table, looking at the paperwork I need to finish, and daydreaming about what I want to do this summer, when I came across information about Wet Sounds 2008: A Festival of Underwater Sounds. If you visit the website, move your curser, (or finger, if you have a touch screen) around the page. Bubbles float around, as if you are under water.
Unfortunately, the nine pools hosting Wet Sounds are in England.

WetSounds is sponsored by NewToy, a company that is involved in sound design, films, theater, new media, art installation, video art, dance, and radio.

"The audience dips in and floats around, ears submerged, absorbed in sounds coming from underwater speakers".... "Sound travels 4 times quicker than in air, due to the water's density, it is perceived not only by the ears but also by the bones and body. The sensation is similar to a floatation tank effect. The activity promotes relaxation, listening, openness to sound art and enhanced audio perception".

Phoenix Mars Mission Website

The following is part of a related post from the TechPsych blog about the Phoenix Mars Mission:

If you are an educator with access to an interactive whiteboard and can integrate a visit to Mars into your lesson plans during the last days of the school year, a trip to the Phoenix Mars Mission website is a must! The website is well designed and user-friendly.

Renderings of the Phoenix Mars Lander

Here are few places to start:

Phoenix Mars Mission News
Web Exhibit -Mars: The Search for Water, the Search for Life
Flash Video Stream (the video has relaxing ambient music, by the way)
Images
Videos and Animations
Just for Kids - this site was designed with content by kids to share with others.

From Mars to Earth: An Interactive Timeline

Enjoy!

Virtual Architecture, Virtual Archeology

It is interesting to see how interactive technology is cutting across so many disciplines!

Raising the Titanic

Ken Rigby's company, MellaniuM, developed the plugin for this project. To learn more about this project, check out Electric Archaeology: Digital Media for Learning and Research, and the related links:

Shawn Graham, Archaeologist
Agent Based Modeling in Archaeologists
http://electricarchaeologist.wordpress.com/2008/04/22/autocad-into-unreal2/

AutoCad Plugin for Unreal
http://mellanium13.blogspot.com/2008/02/mellanium-virtual-architecture.html

Dance.Draw Project : Exquisite Interaction - Collaboration between Software Information Systems -HCI- and Dance Departments at UNC-Charlotte

DANCE.DRAW: EXQUISITE INTERACTION
(Updated)

"The movement of the visualizations are artifacts in real-time of the movements of the dancers. They draw while they dance, they dance together and they draw together. Every performance generates a new visual imprint." -DanceDraw website

Interactive multimedia technology, blended with the arts!

Dr. Celene LaTulipe, from UNC-Charlotte's Software and Information Systems Department, Professor Sybil Huskey, from the dance department, dance students, and others collaborated to create an amazing performance that I had the opportunity to see performed during the Visualization in the World Symposium in April (2008).

If you look closely, you will see that each dancer holds two wireless mice, one in each hand. The mice trigger the visualization that is projected in the background. Dr. LaTulipe has focused some of her research on two-handed computer interaction. It is interesting to see how her work has been applied to this beautiful "off-the-desktop" application.

Dance.Draw is a work in progress- visit the following links for more information:

Website (Updated)
Movie
Technical Info
Dr. Kosara's Eager Eyes post about Dance.Draw

Note:
Dr. LaTulipe was my HCI professor- Dr. Kosara was my Visualization/Visual Communication professor.

Game Based Learning: Second European Conference

The 2008 Second European Conference on Game-Based Learning will be held in Barcelona, Spain, October 16-17, hosted by the Universitat Oberta de Catalunya.

Speaker Bios

Conference Program

If you are working in a K-12 setting and interested in sharing your ideas about game-based learning, please leave me a message. I'm especially interested in how interactive games support engaged, meaningful learning.

Engaged Learning Revisited: Four videoclips for reflection....

Last October, I shared a couple of videos to highlight a discussion about engaged learning in a post on my TechPsych blog. Today, I received a thoughtful comment from a teacher who was moved by the videos. I thought that I'd recycle that post and add two other videos that provide related messages in ways that can not be conveyed effectively by words alone:

From the October TechPsych post:
Those who follow my blogs know that I usually focus on the positive- engaging technologies, exciting research, interesting websites, and successes of others worth sharing. Today I came across a video that got me thinking about the importance of engaged learning, a topic I've written about in the past.

Much effort is extended in our high schools to prepare students for the "real" learning that will take place in college. With the increased emphasis on testing, it seems like the moment a child enters kindergarten, one of the main goals is to prepare for learning the skills needed in the following grade. Every student must be "ready".

Ready for....this?


In today's digital world, traditional word-based instruction doesn't always result in experiences that engage the hearts and minds of young people.

The video below was created by students involved in the Digital Ethnography group at Kansas State University. Michael Wesch is the professor behind this group. At the beginning of the video, the camera slowly peers around an empty university lecture hall, with the the following quote superimposed over the scene:

"Today's child is bewildered when he enters the 19th century environment that still characterizes the educational establishment where information is scarce but ordered and structured by fragmented, classified patterns, subjects, and schedules." -Marshall McLuhan, 1967

Although the quote is now forty years old, the message communicated in this video is clear. Despite technology, things haven't really changed, as traditional methods of teaching tend to disengage rather than engage a good number of students.



After viewing the above video, watch this video, "When I grow up I want to be a teacher", a parody of a Monster.com commercial.



For a broader perspective, take some time to reflect on the following video, originally created by Karl Fisch, and posted to his blog, the Fischbowl:
"A staff development blog for Arapahoe High School teachers exploring constructivism and 21st century learning skills. The opinions expressed here are the personal views of Karl Fisch - and various other teachers at Arapahoe - and do not (necessarily) reflect the views of Littleton Public Schools."

DID YOU KNOW? SHIFT HAPPENS


"DEBUNKING THE MYTHS OF THE THIRD WORLD" is Hans Rosling's presentation at TED 2006 is a great example of the use of presenting information supported by engaging information techniques. More videos are on the Gapminder website. This video is about 20 minutes long, but worth every second.



"With the drama and urgency of a sportscaster, Prof. Hans Rosling uses software from Gapminder debunks a few myths about the "developing" world. This global health visionary has discovered a powerful new way to communicate complex data about the world; his remarkable interactive graphs help deliver profound insights about global trends and will change forever the way you think about "us" and "them." Rosling is professor of international health at Sweden's Karolinska Institute, and founder of Gapminder, a nonprofit that brings vital global data to life."

Cross Post: One Laptop Per Child's Redesign- Dual Touchscreens, Flexibile Use

Photo from One Laptop per Child via MIT Technology Review

"Hundred-dollar laptop, revisited: The next-generation version of the One Laptop per Child machine will dispense with keypads. It can be folded flat to make one larger screen (left); here, two children could play a game, each using the touch-screen capability. Or it can be held on its side and used as an electronic book (right)."

I'm impressed with the new design of the OLPC, the dual touch screen, the support of collaboration and sharing between children, and the flexibility it will provide educators and students. It can even be used as an e-Book! This laptop would be welcomed in UDL classrooms.

For detailed information about the new OLPC laptop, which has not yet been released, read David Talbot's article in the MIT Technology Review.

I want one.

NUI-Group Member Bridger Maxwell Receives High School Science Fair Award for Multi-Touch Screen Project

Yet another post about a NUI group member... Bridger Maxwell, a high school student at the Utah County Academy of Sciences, submitted his multi-touch screen to the science fair, and went on to win first place in the engineering category, and now will be competing in the International Science and Engineering Fair.





Bridger has created "Lumen", a puzzle game for OSX and Windows, and markets this through his business, Fiery Ferret.

Even More Multi-Touch, Delivered by NUI...

May 20, 2008
NUI announces delivery of their multi-touch solution for Cityscape, Abu Dhabi 2008





www.natural-ui.com
Natural User Interface Europe Ltd.

More Multi-Touch from members of the NUI group!

It is always exciting to see what members of the NUI group are doing!

Here is a new video of a multi-touch creation by some of the members of the NUI group. Although this is a proof-of-concept example, it is fun to see how it is played out, using the little iPhone-like touch-pad widgets as a navigation tool for the large screen.


Read the "Multi-touch Goodness" article in Gizmodo of an interview with Christian Moore about this demo and his Lux open-source framework. (Christian is a colleague of Harry van der Veen, both members of the NUI group.)

Here is an excerpt from the interview:
"JD: Why Flash?
CM: Because it's fast to prototype in. However, the software is broken into several segments. One C++ application that tracks hands that talks to Flash... WPF... or another C++ app... and basically everything you can imagine. You can enable multitouch in any environment, like Cocoa."

High-resolution screen shots and additional information can be found on the nuiman website.

For my tech-minded readers:
I'm pretty sure that the C++ application that track hands and fingers in the video demo uses Touchlib, a library for creating multi-touch interaction. Touchlib can work with TUIO, a protocol for tabletop tangible user interfaces. Applications such as Flash and Processing support TUIO. For more information about TUIO, read "TUIO: A Protocol for Table-Top Tangible User Interfaces".
(Information from the NUI group website mentions that OpenCV, or Open Computer Vision Library, found on SourceForge, can support blog detection and tracking.)

The people behind TUIO are from the Reactable project, of the Music Technology Group at Pompeu Fabra University in Barcelona:

Here is my plug for the NUI group, once again!

"The NUI group, or Natural User Interface Group, is an interactive media group researching and creating open source machine sensing techniques to benefit artistic and educational applications.

We offer a collaborative environment for developers that are interested in learning and sharing new HCI (Human Computer Interaction) methods and concepts. This may include topics such as: voice/handwriting/gesture recognition, touch computing, computer vision, and information visualization."


FYI
I came across the Harry van der Veen of the NUI group in early 2007 when I was working on touch-screen projects for my HCI and Ubicomp classes, and I'm inspired by all of the creativity I've seen coming from this group.

If you'd like to see more demos, visit the Natural User Interface website, a commercial out-growth of Harry and his colleague's work, where you can view a reel that includes a few touch-screen games. I love the vision statement on this site:

"Technology should enable us to interact with computers, in the same way we interact with the real world; in a way which is natural to us, namely through gestures, expressions, movements, and manipulations. Our vision is to change the way people interact with computers."