Interactive Music Challenge: The Reactable & the Reactable for Children with Autism Spectum Disorders

What I'd like to share today might be of interest to educators, techies, musicians, parents, and students. If you create a composition using the Reactable Mobile app and submit it, you might have a chance of winning your very own Reactable Live!  


Information about the challenge can be found on the Reactable website.

You can find information about creating music with Reactable Mobile for your phone or tablet (Android or Apple) on the Reactable Mobile site. I especially like that the company provides a load of support through the Reactable Community.



Acquisition of Joint Attention and Social Abilities of Children with Autism Spectrum Disorders

The Reactable provided the centerpiece of Lilia Villafuerte Bazan's work for her Master thesis.  Her project focused on the acquisition  of joint attention and social abilities of children with autism spectrum disorders through music. Lilia's work was through the Music Technology Group at Universitat Pompeu Fabra, under the direction of Dr. Sergi Jorda.   

I was delighted to see Lilia's presentation of her work, along with the video, during the recent CHI 2012 conference. 


Project website: 
http://www.villafuerte.info/ascchildrenandtuis
Master Thesis:
http://www.villafuerte.info/wp-content/uploads/2012/05/LiliaVillafuerte_MasterThesis.pdf


-Lilia Villafuerte Bazan

RELATED
REACTABLE LIVE

Comment:
If you follow this blog, you probably know that I am a school psychologist who has a background in technology, music, and related arts.  I spend much of my work week with young people who have severe autism and other disabilities. I also work at a magnet high school for technology and the arts.  I'd like a Reactable for students at both schools!

Short documentary of the story behind the Reactable, a tangible user interface for creating music. (Includes an interview of Joel Bonasera, of Charlotte's Discovery Place museum.)

The following video provides a look into the history of the Reactable,  from the initial paper prototypes to the present, including the Reactable Mobile application designed for the iPad, iPhone, and iPod touch.   The video includes interviews of Sergi Jorda and Gunter Geiger, members of the original team at Pompeu Fabra University (Barcelona) that created the Reactable.  The other team members are Martin Kaltenbrunner and Marcos Alonso.




FYI:  At about 2:34 in the video, Joel Bonasera briefly discusses the Reactable installation at Charlotte's Discovery Place museum. Joel is a project manager at Discovery Place.


RELATED
How the Reactable Works
John Fuller, howstuffworks
Music Technology Group, Pompeu Fabra University
Reactable Website
Reactable Concepts
Reactable History
Discovery Place
Interactive Technology in the Carolinas: Discovery Place Science Center
(Includes a short video clip I took of the Reactable at Discovery Place)

Juggling and Music: JAM meets the ReacTable

Enjoy!
Juggling and Music Meets the ReacTable, With Carles Lopez


I could play around with a ReacTable all day long!

RELATED
Need an 8 and 1/2 minute dance/exercise break?  Get up out of your chair and dance to this  video of the Brainwater ReacTable Live Performance 1

Reactable Mobile - Music Creation and DJ-ing On-the-Go : This is why I need an iPad!

A couple of weeks ago I posted about the Reactible Mobile,  a fun "DJ" electronic music generator for the iPod and iPad, created by Reactable Systems, makers of the Reactable.  The Reactable is a multi-touch, multi-user system for music creation, sharing, and DJ-ing, but it comes in the form of a table. Even though one version of the Reactable, the Reactable Live! is portable, it is still pretty big.

In our increasingly mobile world, it's nice to know that now we can have something similar to take everywhere we go, with the Reactable Mobile that runs on the iPhone, iPod Touch, and the iPad. You can download and buy the Reactable Mobile app for the iPhone and iPad from the iTunes Store:  http://itunes.apple.com/app/reactable-mobile/id381127666?mt=8

The following videos provide a quick overview of how the Reactable app runs on mobile devices. The first video is a demonstration of "Verde", Le Freak Selector. The second video demonstrates the Reactable app on the iPhone, iPod touch, and iPad.





Reactable Mobile (Reactable Systems SL) Description  (Taken from the iTunes website)

"Improvise and create music playfully with the Reactable mobile. Based on the award winning Reactable as used by Björk on her Volta Tour, this application brings the full creative power of the famous interactive instrument onto your mobile device."

"It uses concepts of modular synthesis, sampling, digital audio effects, DJing, and combines them with modern human computer interaction and multi-touch technology.Based on the very same audio and graphics engine as the Reactable, the mobile version brings a complete range of objects to multi-touch devices:"
A set of generator objects:
- loop players, with possibility to upload your own loops,
- synthesizers, with a large range of instruments to select from,
- oscillators, to synthesise pure and complex tones,
- input, to get audio directly from the device's microphone.
A set of effects to modify generated sounds:
- wave shapers: distortion, compression, and resampling,
- delays: reverb, feedback, and ping-pong,
- modulators: ring modulation, chorus, and flanger,
- filters: low pass, high pass, and band pass.
A set of controller objects to modify other objects parameters:
- sequencers, with step-by-step, matrix, or random modes,
- low frequency oscillators (LFO), with different waveforms,
- accelerometer, to fetch data from the movement of the device.
A set of global objects to modify the settings of the entire table:
- tempo, to change the speed of the table,
- volume, to lower or increase the loudness,
- tonalizer, to change the harmony of the melody.
"IMPORTANT NOTE: A graphic resolution issue with iPod Touch 4G prevents using the version 1.0.3 on this device. An update is on its way.  Note: as this application uses both graphic and audio resources extensively, recent devices will provide the best user experience."

RELATED
There is a Reactable in an exhibit at the Discovery Science Center in Charlotte, N.C.  I had a chance to play with it during my last visit, and it was... awesome.   For more information, including pictures and video,  take a look at this post:
Reactable Live! at Sonar Barcelona 2010 (You can play with one at Discovery Place in Charlotte, NC.)

Here is a very short video-clip of the Reactable at Discovery Place. (Because of the noise in the background, it is a bit difficult to hear what I was trying to play.)


If you are interested in tangible user interfaces, including the Reactable and other similar systems, take a look my previous posts on the topic:
Tangible User Interfaces Part 11: More Examples, Resources, and Use for TUI's in Education
Tangiblie User Interfaces, Part I: Siftables

For the iPad: Reactable Multi-touch Mobile

More details later!

RELATED
History of the Reactable


How it works (Info from the Reactable website)
"The way the Reactable works is deeply inspired by modular analogue synthesizers such as those developed by Bob Moog in the early 60s and enhances them with new concepts of sampling and digital effects processing."

"While with modular synthesizers one typically had to connect these different modules with patch cables in a complex and error-prone process, with the Reactable this is attained in a much easier and more intuitive way, since connections between the blocks are managed automatically based on their types and affinities and on their proximity."


Martin Kaltenbrunner (Co-founder of Reactable)
"His research concentrates on tangible user interfaces and the development of novel human computer interaction concepts within open tools for creative production. As co-founder of Reactable Systems he had been mainly working on the interaction design of the Reactable - an electronic musical instrument with a tangible user interface. He is author of the open source tangible interaction framework reacTIVision and the related TUIO protocol, which have been widely adopted for the realization of tangible tabletop applications."


SOMEWHAT RELATED
Soundythingie for the iPad
iPad Apps for Making Music: What's Coming Later
SurfaceDJ iPhone App (Vectorform)

Reactable Live! at Sonar Barcelona 2010 - (You can see one at Discovery Place in Charlotte, NC.)

I like music technology. I like interactive, tangible table systems.  I like the Reactable!


To understand this system, see it in action!

Below is a video of Reactable Live!, Performed at Sonar Barcelona 2010 on June 18th. The music and performance is by Carles Lopez, and the video was by Marc Morera.

"Reactable in Concert by Carles López sounds like electronic house, minimal, electro, techno, idm, dnb…"

Here is a video of children playing with the Reactable, at Sonar Kids, Barcelona, 2009:



I recently learned that there is a Reactable in my hometown!
The Reactable Experience was installed at Discovery Place in Charlotte.  Discovery Place is the children's science museum for the metropolitan Charlotte, N.C. region. Below is a picture from the Discovery Place museum:


The Reactable is part of a new exhibition, "Think it up"  Here's the description from the Discovery Place website:

"The Reactable is a cutting edge electronic musical instrument merged with a visual experience. Special objects are placed on a translucent surface which trigger sound clips and audio modulation. The sound produced by the objects is represented visually on the surface, providing feedback and guidance for the creation of new music."


This version of the Reactable is for museums, science centers, schools, universities, and other public spaces.  It is an outgrowth of the original Reactable that was designed for musicians. This version is intuitive and invites people to share and create music collaboratively.


RELATED
Reactable Experience brochure (pdf)

Tangible User Interfaces Part II: More Examples, Resources, and Use for TUI's in Education

In Part I of my "mini-series" about Tangible User Interfaces, I discussed the origins of TUI and provided some examples of Siftables. In this section, I've provided some links to information about Tangible User Interfaces and the abstracts of two articles pertaining to TUI's in educational settings.

Zen Waves: A Digital (musical) Zen Garden


reactable from Nick M. on Vimeo.

Reactable

More about the Reactable
"The reactable hardware is based on a translucent, round multi-touch surface. A camera situated beneath the table, continuously analyzes the surface, tracking the player's finger tips and the nature, position and orientation of physical objects that are distributed on its surface. These objects represent the components of a classic modular synthesizer, the players interact by moving these objects, changing their distance, orientation and the relation to each other. These actions directly control the topological structure and parameters of the sound synthesizer. A projector, also from underneath the table, draws dynamic animations on its surface, providing a visual feedback of the state, the activity and the main characteristics of the sounds produced by the audio synthesizer."

The Bubblegum Sequencer: Making Music with Candy


Jabberstamp: Embedding Sound and Voice in Children's Drawings(pdf)
(A TUI application to support literacy development in children)

Affective TouchCasting (pdf)

TapTap: A Haptic Wearable for Asynchronous Distributed Touch Therapy (pdf)

BodyBeats: Whole-Body, Musical Interfaces for Children (pdf)

Telestory is a Siftables application that looks like it would be quite useful for supporting children who have communication disorders or autism spectrum disorders.

Telestory Siftables application from Jeevan Kalanithi on Vimeo.

"Telestory is an educational, language learning application created by Seth Hunter. In this video, the child is looking at a television screen. He can control onscreen characters, events and objects with the siftables. For example, he has the dog and cat interact by placing the dog and cat siftables next to each other." TeleStory Project Website

Here is a video of how Siftables can be used as equation editors:

Siftables Equation Editor from Jeevan Kalanithi on Vimeo.

RESOURCES ABOUT TUI'S:

5 lessons about tangible interfaces, GDC Lyon, December 2007(pdf) Nicolas Nova

Special Issue on Tangible and Embedded Interaction (Guest Editors: Eva Hornecker, Albrecht Schmidt, Brygg Ullmer) Journal of Arts and Technology (IJART) Volume 1 Issue 3/4 - 2008

Reality-Based Interaction: A Framework for Post-WIMP Interfaces (pdf)

Here are a couple of abstracts of articles related to the use of TUI's in education:

Evaluation of the Efficacy of Computer-Based Training Using Tangible User Interface for Low-Functioning Children with Autism Proceedings of the 2008 IEEE International Conference on Digital Games and Intelligent Toys

"Recently, the number of children having autism disorder increases rapidly all over the world. Computer-based training (CBT) has been applied to autism spectrum disorder treatment. Most CBT applications are based on the standard WIMP interface. However, recent study suggests that a Tangible User Interface (TUI) is easier to use for children with autism than the WIMP interface. In this paper, the efficiency of the TUI training system is considered, in comparison with a conventional method of training basic geometric shape classification. A CBT system with TUI was developed using standard computer equipment and a consumer video camera. The experiment was conducted to measure learning efficacy of the new system and the conventional training method. The results show that, under the same time constraint, children with autism who practiced with the new system were able to learn more shapes than those participating in the conventional method."

Towards a framework for investigating tangible environments for learning Sara Price, Jennifer G. Sheridan, Taciana Pontual Falcao, George Roussos, London Knowledge Lab, 2008

"External representations have been shown to play a key role in mediating cognition. Tangible environments offer the opportunity for novel representational formats and combinations, potentially increasing representational power for supporting learning. However, we currently know little about the specific learning benefits of tangible environments, and have no established framework within which to analyse the ways that external representations work in tangible environments to support learning. Taking external representation as the central focus, this paper proposes a framework for investigating the effect of tangible technologies on interaction and cognition. Key artefact-action-representation relationships are identified, and classified to form a structure for investigating the differential cognitive effects of these features. An example scenario from our current research is presented to illustrate how the framework can be used as a method for investigating the effectiveness of differential designs for supporting science learning"

Grafiti - a multi-touch, table-top, surface computing application, from a member of the NUI group.

Are you interested in multi-touch gestures for multimedia tabletop applications, tangible or surface computing, and interaction design? Take a look at this video of Grafiti:



Alessandro De Nardi, a student in computer science at the University of Pisa, Italy, has worked on Grafiti during Google's Summer of Code and is still involved with the project. Alessandro, a member of the NUI-group, is also in the Music Technology Group of Barcelona's UPF University. Allesandro was supervised by by Martin Kaltenbrunner and Sergi Jorda, of the reactable project.

According to De Nardi's Google Code website:

"Grafiti is a C# framework built on top of the Tuio client that manages multi-touch interactions in table-top interfaces. The possible use of tangible objects is particularly contemplated. It is designed to support the use of third party modules for (specialized) gesture recognition algorithms. However a set of modules for the recognition of some basic gestures is included in this project.

The development is in the alpha state, going to beta soon.

The goals I've been aiming at are: generality, versatility, speed of execution, extensibility, ease of programming (integration) of external modules (simple APIs and protocols) and effective interface design with regards to HCI aspects." -Alessandro De Nardi

Demos in binary form are available for download for MacOSX and Windows on the Google Grafiti site. There is a Grafiti site on Sourceforge. Apparently the alpha version will work on all 32-bit MS Windows,, BSD platforms, and POSIX (linux).

(Note: This post was updated on 12/14/08 and includes corrections.)