Interactive Music Challenge: The Reactable & the Reactable for Children with Autism Spectum Disorders

What I'd like to share today might be of interest to educators, techies, musicians, parents, and students. If you create a composition using the Reactable Mobile app and submit it, you might have a chance of winning your very own Reactable Live!  


Information about the challenge can be found on the Reactable website.

You can find information about creating music with Reactable Mobile for your phone or tablet (Android or Apple) on the Reactable Mobile site. I especially like that the company provides a load of support through the Reactable Community.



Acquisition of Joint Attention and Social Abilities of Children with Autism Spectrum Disorders

The Reactable provided the centerpiece of Lilia Villafuerte Bazan's work for her Master thesis.  Her project focused on the acquisition  of joint attention and social abilities of children with autism spectrum disorders through music. Lilia's work was through the Music Technology Group at Universitat Pompeu Fabra, under the direction of Dr. Sergi Jorda.   

I was delighted to see Lilia's presentation of her work, along with the video, during the recent CHI 2012 conference. 


Project website: 
http://www.villafuerte.info/ascchildrenandtuis
Master Thesis:
http://www.villafuerte.info/wp-content/uploads/2012/05/LiliaVillafuerte_MasterThesis.pdf


-Lilia Villafuerte Bazan

RELATED
REACTABLE LIVE

Comment:
If you follow this blog, you probably know that I am a school psychologist who has a background in technology, music, and related arts.  I spend much of my work week with young people who have severe autism and other disabilities. I also work at a magnet high school for technology and the arts.  I'd like a Reactable for students at both schools!

First Moment of Truth in digital signage environments: Bill Gerba's Words of Wisdom

Here is the link to a great discussion about FMOT and the user or customer experience in a range of environments that include digital displays: 




Examples of the First Moment of Truth in Digital Signage 
Bill Gerba, The Digital Signage Insider, 5/24/12

Leap Motion: Low Cost Gesture Control for Your Computer Display

Jessica Vascellaro, of the Wall Street Journal, reports about gesture,  motion. and even object control for computers, highlighting the work of  Leap Motion and Flutter.




Apparently the Leap Motion sensor is less expensive than Microsoft's Kinect. It can track movements down to 1/100 of a millimeter and can track fingers and movement. It handles interaction with 8 cubic feet of space.


Below is a video from the Leap Motion website:






RELATED
Leap FAQs
Leap Motion Developer Kit Application
Leap Motion: 3D hands-free motion control, unbound
Daniel Terdiman, CNET, 5/20/12
FYI:  Do a search and you'll find many more articles and posts about Leap Motion!

A fun HTML5 interactive music video: Evelyn, performed by ABBY, coded by Bleech

ABBY's recent music  video, Evelyn, was created in HTML5 to support an interactive experience. As the band plays, you can select one of three paths for each musician, and switch them around in real-time to see what unfolds. The interactive version of the video can be found at http://evelyn-interactive.searchingforabby.com/ 


Steffen Bewersdorff and Dominik Tranklein are the developers who created the video. Their web agency, Bleech, based in Berlin, Germany, focuses on providing innovative services to music and media industry clients.  (More information about the nuts and bolts of the development process can be found in a post on the Web Designer Depot site:  "An awesome HTML5 interactive music video".)


Note:  The interactive musical video performance requires a fast computer. You'll need a browser that has extended HTML5 support, such as a Safari browser if you have a Mac, or a Chrome browser if you are using Windows. Be sure to close your running applications, and enjoy creating/playing with the band! 


Below is the non-interactive version of the video, which provides a walk-through of one possible interactive musical experience:
 

UPDATE:  I used this interactive video, displayed on a SMARTBoard, with students who have special needs, in several classrooms, and it was a great hit.  They especially liked the drummer.  For some reason, we couldn't get the fifth section to show a musician.  The students didn't mind.

RELATED/SOMEWHAT RELATED
The Wilderness Downtown (Arcade Fire, Chris Milk)
OK Go's Interactive HTML5 Music/Dance/Message Video, Featuring Pilobolus
ROME: "3 Dreams of Black" An Interactive Film by Chris Milk, Music by Danger Mouse, Daniele Luppi & Norah Jones
ROME: "3 Dreams of Black" The Technology
Creating Responsive HTML5 Touch Interfaces (Stephen Woods)
Popcornjs.org (open source HTML5 media framework)

Web goes pop: HTML5 is rejuvenating music videos online
Madhumita Venkataramanan, Wired UK
Behind the Scenes: The Interactive Music Video Game built in HTML5 (MSDN - includes link to code samples, including a frame code reader to sync video and interactive elements.)

User Experience and Windows 8, link to Steven Sinofsky's post.

Steven Sinofsky is an author of an informative MSDN blog. His recent post about user experience and the new Windows 8 operating system provides an interesting, detailed "back-history" of the development of the Windows UI over the ages. Steven's post is worth setting aside a chunk of time to read and reflect!

Creating the Window 8 User Experience

 

BTW, the numerous comments from readers of Steven's post are worth skimming.

Johnny Chung Lee's Recent Words of Wisdom & Google's Open-Source Ceres Non-Linear Least Squares Solver

I have been a fan of Johnny Chung Lee since 2007 or 2008, before he finished his Ph.D in Human-Computer Interaction.  Johnny went on to work at Microsoft (Kinect) and then Google, where he works as a Rapid Evaluator. 


Johnny is known for his experiments with the Wii Remote, which he introduced to the world during a TED Talk in 2008.  He continues to maintain his Procrastineering blog, and from time-to-time, uses his blog to share his take on the world of technology.  The following quote is a good example of his viewpoint, taken from his post, "Technology as a Story":


"...what saddens me is when I encounter technologists with the brilliance to create new and wonderful things, but lack a sense of what is beautiful to people. Technology is most often known for being ugly and unpleasant to use, because technologists most often build technology for other technologists.
...But to touch millions of people, you have to tell a story - a story that they can believe in, a story that can inspire them. Technology is a tool by which new stories can be crafted." - 



Today, I came across Johnny's most recent post, which asks, "So, what exactly is a "non-linear least squares solver"?  And why should you care?   Take a moment to read his post, "Ceres: solving complex problems using computing muscle".  Google just open sourced the Ceres Non-Linear Least Squares Solver.


If Johnny Chung Lee thinks that this is "probably the most interesting code library" that he's had a chance to work with, it probably has some value. 


Even if if you don't have a clue about the Ceres Non-Linear Lest Squares Solver,  you might appreciate Johnny's examples of how would it would useful. In today's rapidly-accelerating technology-supported world, you just might need it in your future!


Here are a few examples:
---Making sense of sensor data from multiple locations (see video "SLAM 1: Viewed at 6X speed")
---Figuring out the position of a camera and the objects in view (see video "Parallel Tracking and Mapping for Small AR Workspaces")
---Combining GPS data with vehicle sensors in cars. (see video "Street View Sensor Fusion with Ceres")


RELATED
Johnny Chung Lee's Website
Excerpt from a post I wrote about Johnny Chung Lee four years ago:
I wish I could be Johnny Chung Lee for a Day! 3/2/08
I've mentioned in previous posts that I am a fan of Johnny Chung Lee, a Ph.D. student in the Human-Computer Interaction department at Carnegie-Mellon University. Johnny expects to complete his Ph.D this year. Johnny recently presented his innovative work at TED 2008. 


What impresses me about Johnny is the way that he has documented his intellectual journey in a very accessible way, by using YouTube and his well-organized, appealing website. Johnny has taken interesting ideas that most would dismiss as silly or impractical, and transformed them into useful, usable applications that hold great promise for future work. 


 In my opinion, many of Johnny's "hacks" will spark ideas related to the design and development of universally designed technologies and applications that will meet the technology needs of a wider range of people. This is important, especially now that an increasing number of "connected" interactive displays and kiosks (known by the marketing industry as interactive digital signage) in public spaces.


January 2011 post:
"Hi, Google. My name is Johnny Chung Lee": Johnny Chung Lee Leaves Microsoft. (I still wish I could be Johnny Chung Lee for a day.)