Taking a look at CreativeJS: "The very best of creative JavaScript and HTML5" -

I came across the CreativeJS website while searching for HTML5/JavaScript resources suitable for creative multimedia projects.  The team at CreativeJS includes Seb Lee-Delisle, Val Head, Paul Neave, Rob Hawkes, Hakim El Hattab, and Lisa Larson-Kelley.


What a talented team!


RELATED
Recent blog post from CreativeJS:  Spherical Harmonics Mesh Builder
Seb Lee-Delisle
Neave Interactive
Rob Hawke's book:  Foundation HTML5 Canvas
Hakim El Hattab's Experiments
Lisa Larson-Kelley's Website:  Learn from Lisa (Web video publishing)

SOMEWHAT RELATED
Interactive Digital Fireworks

Digital Fireworks in Brighton from Seb Lee-Delisle on Vimeo.

OK Go's Interactive HTML5 Music/Dance/Message Video, Featuring Pilobolus - behind the scenes and more. Delight.

Not long after the earthquake and nuclear disaster in Japan, the creative team at Google Japan collaborated with the band OK Go and Pilobolus, a dance troupe,  to explore the capabilities of HTML5 and multimedia, while at the same time providing people from all over the world a means to send a message to others, including the people of Japan.  


I encourage you to visit the link to the interactive version of All Is Not Lost.   For your convenience, here is the traditional version:

Before the video begins, viewers can enter a message in a text box.  At the end of the video, the message will appear, with a few options for sharing the message. At a loss for words, I quickly typed "I love summer", and was delighted to see how the dancers gathered to form my message, as pictured in the screen shot.


Quite a bit of creative thinking went into the making of All Is Not Lost, as described in following article and video clips:
Behind the Work: Trish Sie on OK Go's "All Is Not Lost"
Shareen Pathak, Creativity, 7/27/11
"The trifecta also partnered with Google to create an HTML5-powered interactive experience, which leverages the Chrome browser to showcase the clip through 12 separate windows of dances that shift along with the music. Users can also type in messages in Roman letter or Japanese and watch the band create the letters with their bodies. The result? A visually arresting intersection of dance, technology and direction." -Shareen Pathak

A LOVE LETTER TO JAPAN....


THE MAKING OF ALL IS NOT LOST


RELATED
All Is Not Lost lyrics
OK Go's Human Kaleidoscope, All Is Not Lost, And How It Translates Into Sales
Teressa Iezzi, Fast Company, 7/28/11
O.K. to Go Dance Crazy
Tony Gervino, NY Times Magazine 7/31/11

OPEN SOURCE MULTI-TOUCH SOFTWARE FOR YOUNG PEOPLE WITH AUTISM SPECTRUM DISORDERS- Featured in the WSJ! (Video and links)

Well before the iPad entered the market, researchers such as Dr. Juan Hourcade at the University of Iowa, began to develop multi-touch software to  support young people with autism spectrum disorders and other disabilities.  I've been following this research for a while, and was happy to discover that Dr. Hourcade's work was  recently featured in a recent Wall Street Journal article and video.  


If you are a parent or work with a young person with special needs, you might be interested in learning more about Windows-7 based touch-screen software.  The software shown in the video below, as well as related resources, are available for free on the Open Autism Software website.




RELATED

Article:

Playing on a Tablet as Therapy

Shalini Ramachandran, Wall Street Journal, 7/26/11

Previous Post: Interactive Multi-Touch for Children with Autism Spectrum Disorders:  Research and Apps by Juan Pablo Hourcade, Thomas Hanson, and Natasha Bullock-Rest, University of Iowa 4/26/10

Interaction Design and Children Remixed, CHI 2011

CHI 2011 Child Computer Interaction: 2nd Workshop on UI Technologies and Impact on Educational Pedagogy

Why is this important to me?   

I work with young people with a range of disabilities, including autism, as a school psychologist.  One of my schools is a program that is technology-rich.  Although we don't have iPads, there is a SMARTBoard in every classroom and in other rooms around the school. We have a Wii,  a few multi-touch all-in-one PC's,  and recently received two Dell multi-touch tablet PCs. 

I noticed that Dr. Hourcade is looking for people who'd like to create similar apps for the iPad.  I'd be interested in knowing anyone who might be working on this.  I'd like to help in some way, but I don't have a lot of free time to do this on my own. 

I have a new iPad2, and after playing around with a ton of apps, I was inspired to downloaded XCode4 and think about what I could create. I'm a Mac person at heart, but I learned to program within a Windows environment (C#).    I need a way to maintain my programming skills and gradually learn a few new ones. This sort of project would be useful to me in my "day job", too!

SOMEWHAT RELATED
3D Interactive Heart, by 22Miles, on a Dell Latitude Xt2

Multimedia, Multi-Touch, and Visitor Participation at the Levine Museum of the New South

I had the pleasure of visiting the Levine Museum of the New South with my daughter, a history buff, and my 7 month old grandbaby. It was his first trip to a museum, and he loved it - the pictures, the hands-on exhibits, and of course, the multimedia technology. Future historian?  Museum curator?  Digital archivist?  We'll see...


We had a chance to explore the Cotton Fields to Skyscrapers exhibit and Whirlwind of Growth, updated to take a look at the past decade and how recent events, such as rapid population growth in the region (69% between 1990, the year I moved to Charlotte, and 2010), recession and financial crisis, have impacted the Charlotte region and the Carolinas.


I especially liked the New South TalkBack part of the exhibit. I think the TalkBack feature was originally part of the Changing Places exhibit, a multi-part project that focuses on how people in the Charlotte region deal with the growing cultural diversity and steady stream of newcomers who come from just about everywhere, and how newcomers experience their transition to the area.  The exhibit was unveiled in February of 2009, and due to positive feedback, was extended well past the planned ending date.  The exhibit's impact was extended to the virtual world through  the Changing Places Project website, as well as the museum's YouTube channel. Both are great places to explore. 


I'd like to share a bit more about the Levine Museum of the New South "experience" from content related to the Changing Places: From Black and White to Technicolor® exhibit:


This overview video of the Changing Places exhibit at the Levine Museum provides a good dose of civic pride.  I've lived in the Charlotte area since 1990 and have come to appreciate the rich tapestry that makes up our community, reflected in the short stories (and songs!) shared in the visitors' video clips.



Changing Places "Our History":

(I especially liked the uplifting background music in this video.)


Changing Places: "The Out of Towners" is a mashup of clips created by visitors to the Levine Museum who come from other places in the US and around the globe:



For more videos from Levine Museum visitors, take a look at the Levine Museum's YouTube channel.   The videos are much better than what I took using my cell phone video cam, below:


New South Talkback, Levine Museum of the New South




What I didn't like so much...


Below is a video of a multi-touch picture/info display, part of the exhibit.  I didn't like it very much, as the screen was very cluttered and the content was "jumpy" at the slightest touch. Given my interest in large displays, multi-touch, and gesture interaction, my expectations were pretty high:


REMIX HISTORY
.


Perhaps it is my civic duty to volunteer with a few other multi-touch techies to create a "redo" of this application!  


RELATED
The following links are to some of the organizations/people who work or have worked in partnership with the Levine Museum of the New South:
Darcie Fohrman, Exhibit Developer and Designer, Changing Places
Irene Morris , Graphic Design, Changing Places
Dr. Pamela Grundy, curator, Changing Places
Dr. Tom Hanchett, assistant curator, Changing Places
Sarah Bryan, researcher, Changing Places
New Granada Productions, edited video footage from the Talkback Booth at the Levine Museum, spanning the course of 14 months.
Studio Displays, Inc.  (Changing Places)
Brad Larson Media "Using technology to encourage family learning in museums, zoos, and other public places" (Changing Places)
Community Building Initiative
Emulsion Arts, Film production, Changing Places
Luquire George Andrews, Changing Places (PR, brand strategy, media planning, digital solutions, located in Charlotte, NC)
Nancy Pierce, Photographer, Changing Places
Michael Daul, Kaplan & Associates Cultural Resources, Online Curator, Website Design and Development, Changing Places


Video preview of the new section of the exhibit, including some shots of "Remix History":

A few interesting tech links from Experienta/Putting People First and HCI 596 Blog

No time for reflective blogging today, so here are a few interesting links!


The future of the TV Experience 
(Article discusses Blink, a media industry magazine)


Designing Pleasurable Products and Interfaces Conference (DPPI 11)

Cambridge Consultants: Patients Want More User-Friendly Medical Devices
Levent Ozler, Dexinger, 6/3/11

Internet of things blurs the lines of bits and atoms
Katia Moskvitch, BBC News, 6/2/11


NESTA: Hot Topics: Digital You-Discussion about Telepresence

Hot Topics - Digital You from NESTA UK on Vimeo.


NESTA is the UK's National Endowment for Science, Technology, and the Arts


RELATED
The following links provide a wealth of resources related to emerging technologies and human-computer interaction:
Experientia - Putting People First blog
HCI  596 blog 
"This blog is for the HCI 596 course being taught at Iowa State University through it's Human-Computer Interaction program."
NESTA 

Quick Links: On-line interactive science resources for families from the Charlotte Observer (Better viewed on a large screen TV!)

The following information was compiled by Alicia W. Roberts as a feature related to her recent article in the Charlotte Observer, "Parents find fun online for kids: How to keep boredom at bay and help youngsters learn more about science tis summer".

To enhance the visual impact of many of the following websites, parents should consider  connecting the family computer to their a large HDTV display, if they have one.   This will make it easier to include all members of the family in the process!

ONLINE SCIENCE FOR KIDS, PARENTS

YouTube -Videos of science experiments:  youtu.be/773Rv8pZeOs. 

National Geographic for Kids: www.Kids.nationalgeographic.com/kids .

Free: Science, math and engineering: www.brainpop.com/free_stuff.

Funology - go to the Weird Science tab: www.funology.com.

PBS Kids: www.pbskids.org.

Fun activities: www.edheads.org.

Resources for Science and Parenting
Geology and Earth science: www.all-geo.org/highlyallochthonous/2011/04/backyard-science-isotope-hydrology-style .

Science and math activities: www.parentingscience.com.

"Helping Your Child Learn Science," a reference booklet with experiments: www2.ed.gov/pubs/parents/Science/index.html.

U.S. Government Science websites, with downloadable resources: www.science.gov/browse/w_133A.htm.

Citizen Science Websites

Nature's Notebook ( www.usanpn.org/how-observe): Help scientists collect data on how plants and animals respond to climate change.

Firefly Watch ( www.mos.org/fireflywatch/how_to_participate): Are fireflies disappearing?

Science for Citizens ( www.scienceforcitizens.net): Find a project that suits your family.

I'll add a few more resources when I get a moment!

Interactive Multimedia: Music, Videos and Ads -Choose your POV, Scenes, Timelines, and More -HTML5, JavaScript, WebGL

I thought I'd share some examples of interesting interactive multimedia sites on the web.  It seems that artists, musicians, and ad agency folks have been experimenting with tools such as HTML5, SVG, Canvas, and Web GL.  Some of this work is featured on Google's Chrome Experiments website, and other examples can be found on websites promoting Wrangler Jeans or Ikea furniture.   This sort of content is great on a larger display.

Take some time to watch the videos and explore the links below.  Enjoy!

ARCADE FIRE: "The Wilderness Downtown" - Interactive Video by Chris Black 

"Choreographed windows, interactive flocking, custom rendered maps, real-time compositing, procedural drawing, 3D canvas rendering... this Chrome Experiment has them all. "The Wilderness Downtown" is an interactive interpretation of Arcade Fire's song "We Used To Wait" and was built entirely with the latest open web technologies, including HTML5 video, audio, and canvas."

ROME:  "3 Dreams of Black", an Interactive Film by Chris Black (The link leads to the interactive site.)

WebGL Aquarium Site

WRANGLER

Blue Bell SS11 Collection Interactive Video

Wrangler: WE ARE ANIMALS GALLERY STUNTMAN (Interactive)

Wrangler: WORN ACROSS AMERICA (Interactive multimedia 

map - go to "Choose Scenes")

Wrangler Collection fw10(Interactive)

Demo of interaction on an iPad

Interactive Ad for IKEA: "A Better Sleep for Everyone"
The website for this amusing and creative ad campaign was created using HTML5. Here is a description of what you'll find if you follow the above link:


"IKEA is now launchig the Kokokaka produced A Better Sleep for Everyone campaign site, which features IKEA's bedding catalog.  6 different mattresses are shown by 6 Swedish artists, each interpreting a classic lullaby performed in a dreamy and surreal music video.  By scrolling up and down the user can change between the artist's music videos and the different mattresses.  Experience, for instance, a soulful Tingsek having problems falling asleep.  ust like the princess from the famous fairly tale he gets annoyed by something hard under the pile of mattresses.  But guess what? It's not a pea, it's Tingsek's band!  Let yourself fall asleep to beautifully performed lullabies!" 


Agency: Forsman and Bodenfors;  Film Production: Social Club;     Director: RBG6;   

Music: Music Super Circus;  Web Production: Kokokaka;   Photographer: Carl Nilsson

Below are videos of two of the lullabies featured in the interactive ad:





Here is a video from the interactive IKEA Come Into The Closet  website from a year or two ago:



"This is a campaign to promote IKEA's wardrobe solutions. IKEA wanted to show their huge range of styles and all the smart features on the inside. All the movements on the web site are controlled by sound and music. So change songs, upload your own music, play on your keyboard or sing into the microphone."


RELATED
Cacophony: An interactive video player in HTML5 and Javascript
"The basic elements of a Cacophony video are: An HTML5 Video on the base layer, a series of HTML5 Canvas layers above that, a timeline of effects to be triggered to the beat of the song, images and other elements to be used by the effects"

Top 10 All-Time Posts on the Interactive Multimedia Technology Blog

I'm finishing up the last couple weeks of the school year, so I'll have little time to post this week.  I hope you enjoy exploring the following links!


Revised Post 8/1/06: Interactive multimedia for social skills, understanding feelings, relaxation and coping strategies


Teliris Interact TouchTable and TouchWall: Immersive Collaboration & Telepresence; DVE's Holographic Tele-Immersion Room


Games to lift stress away: Flower, flOw, (and Cloud), from thatgamecompany


Power to the Pixel Cross-Media Forum Streaming Live from London Today #PttP


HACKED KINECT MULTITOUCH using libFreenect and libTISCH (via Florian Echtler)


Link to iTV Doctor Rick Howe's post about 2D to 3D, 3D TV data points, and 3D content distributers


Temple Grandin - A gifted visual thinker, who also has autism, featured in HBO movie starring Claire Danes.  Update: Video of Claire Danes' acceptance of a Golden Globe for her performance


Algodoo physic app. for the SMART Board 800 series, supports multi-user interaction!


Wii Just Dance2 and Kinect Dance Central:  UI and Usability Approaches; Challenges for Developoing Accessible Games


Interactive Touch-Screen Technology, Participatory Design, and "Getting It" -Revisited

Quick Link: Florida's Technology Integration Matrix for K-12 Tech Integration and Transformation

Technology Integration Matrix 

"The Technology Integration Matrix (TIM) illustrates how teachers can use technology to enhance learning for K-12 students. The TIM incorporates five interdependent characteristics of meaningful learning environments: active, constructive, goal directed (i.e., reflective), authentic, and collaborative (Jonassen, Howland, Moore, & Marra, 2003). The TIM associates five levels of technology integration (i.e., entry, adoption, adaptation, infusion, and transformation) with each of the five characteristics of meaningful learning environments. Together, the five levels of technology integration and the five characteristics of meaningful learning environments create a matrix of 25 cells...."

Here is a list of the people behind the development of the 2010-11 Technology Integration Matrix, as listed on the TIM website.  I will provide links to the team member's websites/blogs in the future.

DEVELOPMENT TEAM
Kate Kemker, Ph.D., Florida Department of Education
Roy Winkelman, Ph.D., FCIT Director
James Welsh, Project Manager
Allison Papke, Coordinator, Lesson Plan Editor, and School Liason
Roberto Herrera, Videographer and Video Editor
Kyle Wahling, Videographer
Francisco Flores, Video Editor
James Basom Seaman, Web Design and Development
J. Christine Harmes, Ph.D., Indicator Development
Ryan Dial, Programming and Development

Special Thanks:
Michael Sweeney, Florida Digital Educator Program
C. Kris Mathews, Florida Digital Educator Program
G. Gordon Worley, Florida Digital Educator Program
Jeanine Gendron, Ph.D., Broward County Schools
John Lien, Orange County Schools
Dawn Howard, Manatee County Schools
Don Manderson, Escambia County
Kara Dawson, Ph.D., University of Florida
Mijana Lockard, Lincoln Avenue Academy, Lakeland, FL
Virginia Richard, School Technology Services, Polk County
Trevor Honohan, Principal, Audubon Park Elementary
Kristine Haller
Nancy Kuznicki
Jeanne Rogers
Crystal Gasell
David Schneider
David Futch
Alvin Olivo
Sylvia Hernandez
Maria Alves
Lois Hooper
Cheryl Woolwine
Jennifer O’Dell
Ana McMoran

The TIM was developed through the Enhancing Education Through Technology (EETT) program under Title II of NCLB.

Musical Multitouch/Gestural Interfaces by Osmosis

I've been pleasantly surprised by the increase of interesting multi-touch and gesture-based applications developed for musical interaction on large displays.  This topic is dear to my heart -  I took a computer music technology class back in 2003, and think it would have been great if this sort of thing was available back then.  Who wants to point and click around a music app for hours on end?!


The video below shows what is new from Osmosis, a company based in NY that focuses on the development of multi-touch and gesture based software for a range of uses, including music applications.


Performance Systems for Stage/Studio from Osmosis on Vimeo.

Transparent Stage System Specs
Design
• Floating, transparent HD displays from 32"
• Haptic surface with tempered glass backing
• High gain image with wide viewing angle
• Rugged aircraft-grade aluminum build
• Enclosed projector and computer
• Minimalist style, compact footprint
• Disassembles for easy transportation


Interactivity
• Projected capacitive foil or IR bezel options
• Up to 32 simultaneous touch points
• Precise, responsive touch tracking (3mm)
• Immune to external light conditions
• Use of fingers, gloves or stylus


Technical
Display:
• DLP portable projector
• 1280×800 HD resolution
• 2500 ANSI-lumen, 1800:1 contrast
Computer:
• Mini-ITX, Core 2 Quad, 4GB RAM
• ATI Radeon X1250 graphics card
• Wireless keyboard and mouse
• Windows 7 Pro

Stuart McClean, the founder of Osmosis, shared the following information about his company:

"Osmosis is a consulting firm based in the NYC area with deep experience in interactive technology. Although we cater to a range of markets, we’re especially passionate about music production and performance. Working closely with artists, we build customized interactive systems for stage and studio. HCI technology is integrated into a range of designs including stands, desktop rigs, tables, carts and vertical screens. Interfaces are tailored to specific artist needs and combine controls, generative audio and visuals, instruments, and gestural input. Our unique and flexible systems take full advantage of multi-touch interaction and offer seamless control of Ableton Live, Traktor, or other DAWs via midi and OSC..."





For more information about the applications developed by the Osmosis team, take a look at their showcase page.  

Pervasive Retail Part I: Web UX Meets Retail CX - Screens Large and Small at the Mall, Revisited

If you follow my blog(s), you know that I have a passion for interactive displays in public spaces, and that I enjoy watching how various technologies converge, jump across platforms and devices, inter-operate, and re-purpose over time.  

The best places for watching this unfold, in my opinion, are airports, malls, shopping districts,  and larger "big box" establishments, where the Web meets Digital Out of Home (DOOH), old-fashioned kiosks morph into multi-touch screens and gesture-based windows, and visual merchandising meets technology, digital culture, architecture, and consumer metrics.  At the center of it all is the user/consumer - regular people, moms, dads, kids, teens, the elderly, the disabled, the hurried and the worried. Adding to the complexity is that an increasing number of people who are out-and-about are tethered to various mobile devices.

In scholarly tech circles, the concept of DOOH is known "Pervasive Retail".  The explosion of mobile devices and ubiquitous screens has fueled the fire for research, and is the focus of the current issue of IEEE's Pervasive and Ubiquitous Computing.   

Despite the influx of technology, no-one is exactly sure how to do it quite right.  (I have some ideas, which I'll save for a future post.)

If you are interested in learning more about concepts related to "pervasive retail", the Retail Customer Experience website is a treasure trove of information related to DOOH, digital signage, multi-channel retailing, in-store media, kiosks, interactive touch screens and windows, related metrics, and more, with stories about real-life technology implementation.

Mall Video

The following video, taken with my handy HTC Incredible, provides a quick sampling of the screens I encountered during a recent visit to South Park Mall, in Charlotte, N.C.  The last screens in the clip were taken in the Brookstone store, and will be included in another clip that focuses solely on all of the screens that were scattered about the retail space.  

I have a hunch that some of the smaller displays in the Brookstone store were iPads.  iPads and tablets have great potential for use for shelf-level in-store interactive visual merchandising deployments, given the right apps and mounting systems. (See iPads as Cheap Digital Signage, by Tony Hymes of DOOHSocial and the video about Premier's iPad mounts, for more information.)

Much of what you'll see in the following video, taken at the same mall in December of 2009, wasn't around during my most recent trip:

Screens Large and Small at the Mall

Interactive Coke Machine and Kid at the Mall

I was sad to see that the interactive screen on the Coke machine  had been replaced by an ordinary one.  Part of the problem, I think, is that the interactive display was too busy and as a consequence, made the goal getting a quick drink a bit too complicated for the average thirsty customer, as seen in the video below:

Touch Screen Coke Machine at the Mall: 90 seconds to get a coke!

RELATED

Previous Posts:

What is DOOH and Why Should We Care?

Quick Link:  Bill Gerba's Updated Graphic of the Digital Signage Ecosystem

Digital Signage and Digital Out of Home (DOOH), via The Digital Signage Insider, Digital Signage Today, and more

Interactive Visual Merchandising, Interactive Window-Shopping at Nordstrom, Downtown Seattle:  "Writing with Light"

Window Shopping in the Web Outside:  Interactive Window Displays (videos and links)

Close Encounter with "Best Buy On": Example of a multi-channel marketing approach using in-store media and an online magazine 

Video: Gesture Interactive Window Display at Repetto Store in France, by the Marcel Agency
Interractions (ACM) Cover "Proximic Interactions: The New Ubicomp?"; Plus - Close encounters with displays at the airport and JC Penney. (Briefly touches on the topic of accessibility/usability/universal design of displays in public spaces, an area in need of further research.)

The World Is My Web Browser: Interactive Technology in Public Spaces

References and Resources (Partial List)

Fujitsu's Pervasive Retailing (pdf)

Ron Brunt, InTouch with Retailing Whitepaper, 1/15/06

When all the world is a screen

Brian Monahan, IPG Emerging Media Blog, 4/15/11

When all the world is a screen (The video is worth taking the time to watch.)

Narayanswami, C.,  Kruger, A.,  Marmasse, N. Pervasive Retail, IEEE Pervasive Computing

April-June 2011 (Vol. 10, No. 2) pp. 16-18 1536-1268/11/$26.00 © 2011 IEEE 

References from the Pervasive Retail article:

Mobile Retail Blueprint, Nat'l Retail Federation; www.nrf.commodules.php?name=Pages&op=viewlive&sp_id=1268 .

G. Belkin, Pervasive Retail Business Intelligence, Aberdeen Group, Apr. 2010; www.slideshare.net/AxiomConsultingAustralia pervasive-retail-business-intelligence .

R. Wasinger, A. Krüger, and O. Jacobs, "Integrating Intra and Extra Gestures into a Mobile and Multimodal Shopping Assistant,"Proc. 3rd Int'l Conf. Pervasive Computing (Pervasive), Springer, 2005, pp. 297–314.

A. Meschtscherjakov et al., "Enhanced Shopping: A Dynamic Map in a Retail Store," Proc. 10th Int'l Conf. Ubiquitous Computing(UbiComp 08), ACM Press, 2008, pp. 336–339.

C. Stahl and J. Haupert, "Taking Location Modelling to New Levels: A Map Modelling Toolkit for Intelligent Environments," Proc. Int'l Workshop Location- and Context-Awareness (LoCA), LNCS 3987, Springer, 2006, pp. 74–85.

Cross-posted: The World Is My Interactive Interface blog