Is the answer Voronoi? Looking for possible solutions to an art+dance+music+tech idea from a recurring dream....

If you are a long-time reader of this blog, you probably know that I sometimes have some unusual dreams about technology.  I don't blog about my dreams very often, but last night, I had another technology dream, a continuation of a dream I had one night last week.  
Voronoi Diagram (Wikipedia)

I'm pretty sure that the last two dreams were sparked by playing an online interactive demonstration of a Voronoi application before going to sleep one night, and also reading an article about "extracting ordered patterns from a triangular mesh surface" in the Novemeber/December IEEE Potentials magazine before turning in last night. 

The dance probably was influenced by my recent viewing of the North Carolina Dance Theater's performance of Innovative Works with my mom, someone who encouraged my  study of music, art, and dance at an early age.

Some of my tech dreams are sort of...practical. For example, in one recurring dream, I find myself coding for a flexible mesh/grid application. Sometimes the mesh/grid has something to do with wireless sensor networks on curved terrain, perhaps related to something like the Smart Grid, and sometimes I find myself working on an application that analyzes streaming data from a variety of sources, for security prediction purposes.  At other times, I'm coding for something more artistic, my preference.

Last night, my dream focused on creating a flexible mesh fabric that used in a multimedia dance/graphic arts/music performance.  I was coding for this performance using a Voronoi-like algorithm.


This is the best I can do to explain this: The fabric is carried by the dancers, and is both reactive and generative. In essence, the fabric is intertwined/embedded in the dance, the music, and the graphics. In my dream, everything looked/sounded/felt awesome and otherworldly, and the music that merged and morphed during the dance was so beautiful, not only the melodies, but the sounds.  (In a previous dream, the mesh contained a "nanotechnology" component, but I'll save that quest for the future.)


I thought I'd look at some of my web book marks and search a bit more for information related to this topic.  For now, here is the "brain dump".  I have more to add to this post, and plan to port it to a reference page for this blog in the future.  I hope that this post will be useful to some of my art/music/dance/tech readers!

RELATED AND SOMEWHAT RELATED


Update:  Right after uploaded this post, I came across a link to a WebGL demo for a 3D music video of pop singer Ellie Goulding's song, 'Lights', by HelloJoy.   Visitors to the webpage can click to interact with the environment. If you keep the button pressed, you fly faster.  If you tweet the link, you'll see your name crop up as you fly around in the soundspace.  For more information about the making of 'Lights' - take a look at  Behind the scenes of 'Lights": the latest WebGL sensation!  (Carlos Ulloa, 11/9/11)


After I watched the 'Lights' video, I recalled Radiohead's 'House of Cards' video, which I wrote about back in 2008: 
 "We were rolling computers all day"...The Making of Radioheads House of Cards using imaging and info visualization software.   The process behind the making of the House of Cards video was described in detail in Chapter 10 of the book, Beautiful Data.

Bradley, E., Capps, D., Luftig, J, & Stuart, J.M. Toward Stylistic Consonance in Human Movement Synthesis.(pdf)  The Open Artificial Intelligence Journal, 2010, 4, 1-19
Bradley, E., Stuart, J.  Using Chaos to Generate Variations on Movement Sequences (pdf) Chaos, 8:800-807 (1998)
Bradley, E., Stuart, J.  Learning the Grammar of Dance.(pdf)  Proceedings Fifteenth International Conference on Machine Learning, Madicson, WI, 1998
E. Bradley, D. Capps, and A. Rubin, "Can computers learn to dance?," Proceedings International Dance & Technology (IDAT), Tempe AZ, Feb 1999.
Chaotic Dance: Using mathematics to generate choreographic variations
Schedl, M., Hoglinger, C., Knees, P. Large-Scale Music Exploration in Hierarchically Organized Landscapes Using Prototypicality Information (pdf)
Fournel, N. Procedural Audio for Video Games: Are we there yet? (pdf) GDC 2010


Voronoi Cells, created by Nathan Nifong.  A version of this interactive work was used in a DanceDraw performance

Patterns in the Noise (Nathan Nifong's site - FYI, Nathan worked with Celine Latulipe with the Dance.Draw project while completing his bachelor degree in computer science at UNC-Charlotte)
Voronoi Dance (Christian Gross, using OpenFramework)
Voronoi art: Slow Trip (Oktalist/Mat)


The above video, by Mat/Oktal, was inspired by his viewing of Thomas Ruff's Substrat images. 
Scott Snibbe Studio (Intearctive art, music, and animation for iPhone, iPad, iPod, and Mac)

Interactive Voronoi Diagram Generator with WebGL (Alex Beutel)
 
The above video was found in Alex Beutel's blog post, "Interactive Voronoi Diagrams with WebGL"

Posts about DanceDraw and related work at UNC-Charlotte:
News from the HCI lab at UNC-Charlotte- Creative Interactions (Videos)
Exploring the Design Space in Technology-Augmented Dance at CHI 2010:  Celine Latulipe's team from UNC-Charlotte
Interactive Surveillance:  Live digital art installation by Annabel Manning and Celine Latulipe

SIGCHI  Digital Arts and Interaction Community:  Building Bridges
The Interdisciplinary World of Dance and Interactive Technology

HTML5Voronoi  (HTML5Code website)

HTML5 Voronoi, Live Version
Update to code to compute Voronoi diagrams (Raymond Hill, 5/22/11)
William Forsythe's "Synchronous Objects-One Flat Thing, Reproduced" - Multidisciplinary online interactive project: Translating choreography into new forms.
BCS HCI 2011 Workshop: When Words Fail:  What can Music Interaction tell us about HCI?
Woven Sound (Alex McLean)
Real DJs Code Live (Robert Andrews, Wired, 7/3/06)
Visualization of Live Code (Alex McLean)
Voronoi diagrams of music (pdf)  (Alex McLean, 2006)
WebGL
Sylvester: Vector and Matrix Math for JavaScript
Generative Art Links (Mikael Hvidtfeldt Christensen)
Schacher, J.C. Motion to Gesture to Sound:  Mapping For Interactive Dance (pdf) Proceedings of the 2010 Conference on New Interfaces for Musical Expression (NIME 2010), Sydney, Australia
Code & Form:  Computational Aesthetics (Marius Watz)
Werghi, N. Extracting ordered patterns from a triangular mesh surface.  IEEE Potentials, Nov/Dec 2011
Last night I dreamt about haptic touch-screen overlays


Hyun-Seok Kim's Voronoi Yacht

Revisiting Good Blogs: IDEUM update

Jim Spadaccini is the director of Ideum and is responsible for the Ideum blog, which is well worth reading if you are interested in creative use of multi-touch, gesture, and interactive multimedia applications.  Jim also serves as the principal investigator of the Open Exhibits project, a non-profit open source, multi-touch, multiuser software initiative, which includes a free software development kit (SDK) for museums, students, nonprofits, and educational use.


Here are a few links to posts from the Ideum blog:
Building a High-Resolution Multi-touch Wall: Part 1, Part 2, Part 3,  Part 4 and Part 5.
Ideum Blog
Events at the The Tech and the Maxwell Museum
Open Exhibits, Year Two

The Open Exhibits Research and Design blog is worth reading, as it often includes Jim Spadaccini's words of wisdom and some interesting video clips.


More about IDEUM
Ideum will be involved in an upcoming day-long conference at the Tech Museum on Tuesday, November 15th (2011), Building Interfaces for the New Decade  Conference San Jose, California, 11/15/11

Ideum is working on interconnectivity between devices, as shown below, where people are connecting their iPhones to a multi-touch table:


Here is a video about Ideum's Tiny Drifters exhibit,  7 foot multi-touch wall at the Monterey Bay Aquarium:




Here is a promotional video about Ideum's rugged MT-55 multi-touch table. It holds up to the interaction of thousands of museum visitors.  

MindHabits Online Demo: Useful Serious Game for Social Skills Group Activities

I'd like to share the on-line demo of MindHabit's suite of serious games that I've found useful in my work with teens and young adults who need support in the area of social-emotional skills. 


What I like about the online demo is that it adjusts to the player's responses. This feature made it fun to use during the last few social skills groups I facilitated at work, since it could be played by students with a range of cognitive abilities. I had students take turns playing the game using a SMARTboard, and found that all of the students paid attention to what was going on. In my opinion, using the interactive whiteboard supported "off-the-shoulder" learning among the students who were not at the board. 


MindGames is available for Windows and Macs, and the full version is just $19.99 and provides 100 game levels.  The full version tracks progress and includes four games.


Here's some information from the company's website:  
"Based on social intelligence research conducted at McGill University, these stress busting, confidence boosting games use simple, fun-to-play exercises designed to help players develop and maintain a more positive state of mind." 
 "Based on the principles of social intelligence: Inhibition - uses game mechanics to promote positive habits; Association - connects personal info to positive feedback; Activation - uses personal references"

You are playing the MindHabits Trainer online demo. Your progress will not be logged beyond this session. Copyright © 2008 MindHabits inc. All rights Reserved.

Upcoming: 

• Interview with Guillaume Largillier about Stantum's multi-touch tablet.
• Collaboration with Stevens Institute of Technology, focusing on a serious game project to support learning of "on-the-job" social skills for teens and young adults with autism spectrum disorders and related challenges.
• More news about large interactive displays, multi-touch, and gesture applications/installations.
• Cross-platform/device interactive "TV".
• Cathy Davidson's new book:  Now You See It: How The Brain Science of Attention Will Transform The Way We Live, Work, and Learn (Viking)

Multimedia, Multi-Touch, and Visitor Participation at the Levine Museum of the New South

I had the pleasure of visiting the Levine Museum of the New South with my daughter, a history buff, and my 7 month old grandbaby. It was his first trip to a museum, and he loved it - the pictures, the hands-on exhibits, and of course, the multimedia technology. Future historian?  Museum curator?  Digital archivist?  We'll see...


We had a chance to explore the Cotton Fields to Skyscrapers exhibit and Whirlwind of Growth, updated to take a look at the past decade and how recent events, such as rapid population growth in the region (69% between 1990, the year I moved to Charlotte, and 2010), recession and financial crisis, have impacted the Charlotte region and the Carolinas.


I especially liked the New South TalkBack part of the exhibit. I think the TalkBack feature was originally part of the Changing Places exhibit, a multi-part project that focuses on how people in the Charlotte region deal with the growing cultural diversity and steady stream of newcomers who come from just about everywhere, and how newcomers experience their transition to the area.  The exhibit was unveiled in February of 2009, and due to positive feedback, was extended well past the planned ending date.  The exhibit's impact was extended to the virtual world through  the Changing Places Project website, as well as the museum's YouTube channel. Both are great places to explore. 


I'd like to share a bit more about the Levine Museum of the New South "experience" from content related to the Changing Places: From Black and White to Technicolor® exhibit:


This overview video of the Changing Places exhibit at the Levine Museum provides a good dose of civic pride.  I've lived in the Charlotte area since 1990 and have come to appreciate the rich tapestry that makes up our community, reflected in the short stories (and songs!) shared in the visitors' video clips.



Changing Places "Our History":

(I especially liked the uplifting background music in this video.)


Changing Places: "The Out of Towners" is a mashup of clips created by visitors to the Levine Museum who come from other places in the US and around the globe:



For more videos from Levine Museum visitors, take a look at the Levine Museum's YouTube channel.   The videos are much better than what I took using my cell phone video cam, below:


New South Talkback, Levine Museum of the New South




What I didn't like so much...


Below is a video of a multi-touch picture/info display, part of the exhibit.  I didn't like it very much, as the screen was very cluttered and the content was "jumpy" at the slightest touch. Given my interest in large displays, multi-touch, and gesture interaction, my expectations were pretty high:


REMIX HISTORY
.


Perhaps it is my civic duty to volunteer with a few other multi-touch techies to create a "redo" of this application!  


RELATED
The following links are to some of the organizations/people who work or have worked in partnership with the Levine Museum of the New South:
Darcie Fohrman, Exhibit Developer and Designer, Changing Places
Irene Morris , Graphic Design, Changing Places
Dr. Pamela Grundy, curator, Changing Places
Dr. Tom Hanchett, assistant curator, Changing Places
Sarah Bryan, researcher, Changing Places
New Granada Productions, edited video footage from the Talkback Booth at the Levine Museum, spanning the course of 14 months.
Studio Displays, Inc.  (Changing Places)
Brad Larson Media "Using technology to encourage family learning in museums, zoos, and other public places" (Changing Places)
Community Building Initiative
Emulsion Arts, Film production, Changing Places
Luquire George Andrews, Changing Places (PR, brand strategy, media planning, digital solutions, located in Charlotte, NC)
Nancy Pierce, Photographer, Changing Places
Michael Daul, Kaplan & Associates Cultural Resources, Online Curator, Website Design and Development, Changing Places


Video preview of the new section of the exhibit, including some shots of "Remix History":

Revisiting CHI 2011: Videos of Interactive Touch, Gesture, Large Surface, and Mobile Apps with Potential for Use in Education (CHI = Computer Human Interaction)

One of my interests is how the power and potential post-WIMP interactive technologies can be harnessed for use for formal and informal education purposes, including life-long collaborative learning.  


In May, I had a chance to meet with a number of like-minded people during the CHI 2011 conference at the 2nd Workshop on UI Technologies and Impact on Educational Pedagogy.  I was impressed with the depth and breadth of the presentations at the workshop.   Since then, I've been looking through other papers and videos from CHI 2011 to find interesting applications that hold potential for use in educational settings.  


I've come across a good number of interesting applications and prototypes, so be sure to check back for future posts on this topic.  For now, here are a few applications that I'd like to share.  


Below are a few videos from Phillip Chi-Wing Fu.  (He doesn't know it yet, but I've admitted his videos into the Post-WIMP Explorers' Club.)


Interactive Multi-touch Sketching Interface for Diffusion Curves

"A novel multi-touch sketching interface enabling interactive and practical design with 2D diffusion curves is proposed; featured interaction techniques include simultaneous sketching of multiple diffusion curves and at-the-spot colors tuning."


Distinguishing Multiple Smart-Phone Interactions on a Multi-touch Wall Display using Tilt Correlation

"This paper proposes a novel matching technique, called tilt correlation, which employs the built-in tilt sensor on smart-phones to identify their concurrent contacts on a common multi-touch wall display."


WYSIWYF: Exploring and Annotating Volume Data with a Tangible Handheld Device (CHI 2011)


"Integration of a multi-touch wall display with a tangible handheld device with multi-touch and tilt sensing capabilities to provide intuitive what-you-see-is-what-you-feel visual exploration and annotation of volume data."


The following videos were uploaded by alucero:


Pass-Them-Around: Collaborative Use of Mobile Phones for Photo Sharing (CHI 2011)

"Pass-Them-Around is a phone-based application that allows a small group of collocated people to share photos using the metaphor of passing paper photos around. The prototype encourages people to share their devices and use them interchangeably while discussing photos face-to-face. The prototype supports ad-hoc photo sharing in different contexts by taking into account the spatial arrangement of users around a table, measured with sensors embedded in their mobile phones."


The next video was part of MobileHCI '10:
MindMap: Collaborative Use of Mobile Phones for Brainstorming

In case you missed this: Microsoft Releases Kinect SDK Beta for PC

Kinect for Windows SDK Beta!   IT IS TRUE!!!!!


My Kinect and PC are waiting for my summer project.    What a great opportunity to "practice" programming over my 5 week summer break..... I already know C#, and I've done a little game programming (ie. AI for Game Development - using XNA Game Studio Express- it has been a while).  

-Photo credit: Microsoft Research


I have some cool ideas for basic games that might be good for the students I work with who have autism spectrum disorders... and some ideas that might be fun for my grand-baby.  I can't wait to have time to code again!   


Here's some info from the Microsoft Kinect for Windows SDK Beta website:


"The Kinect for Windows SDK beta is a programming toolkit for application developers. It enables the academic and enthusiast communities easy access to the capabilities offered by the Microsoft Kinect device connected to computers running the Windows 7 operating system."


"The Kinect for Windows SDK beta includes drivers, rich APIs for raw sensor streams and human motion tracking, installation documents, and resource materials. It provides Kinect capabilities to developers who build applications with C++, C#, or Visual Basic by using Microsoft Visual Studio 2010."


This SDK includes the following features:


Raw sensor streams
Access to raw data streams from the depth sensor, color camera sensor, and four-element microphone array enables developers to build upon the low-level streams that are generated by the Kinect sensor.
Skeletal tracking
The capability to track the skeleton image of one or two people moving within the Kinect field of view make it easy to create gesture-driven applications.
Advanced audio capabilities
Audio processing capabilities include sophisticated acoustic noise suppression and echo cancellation, beam formation to identify the current sound source, and integration with the Windows speech recognition API.
Sample code and documentation
The SDK includes more than 100 pages of technical documentation. In addition to built-in help files, the documentation includes detailed walkthroughs for most samples provided with the SDK.
Easy installation
The SDK installs quickly, requires no complex configuration, and the complete installer size is less than 100 MB. Developers can get up and running in just a few minutes with a standard standalone Kinect sensor unit (widely available at retail outlets)."

RELATED
Microsoft Releases Kinect for Windows SDK Beta

Nicholas Kolakowski, Application Development News, 6/16/11

"Playful Interface Cultures" exhibit video showreel, just released (Ars Electronica Festival 2010)

Playful Interface Cultures from Interface Culture on Vimeo.


"This video showreel provides a walkthrough through the "Playful Interface Cultures" exhibition at the Ars Electronica Festival 2010. The works shown were created within the Master and Ph.D. program at the Interface Culture Lab, University of Art and Industrial Design in Linz, Austria." http://www.interface.ufg.ac.at/​

"The exhibition design has been based on traditional Austrian tobacco brands in reference to the historic venue of the former tobacco factory built by the German designer and architect Peter Behrens." -Interface Culture Lab


Instructors:
Georg Russegger
Martin Kaltenbrunner
Michaela Ortner
Varvara Guljajeva
Design
Vesela Milhaylova
Henning Schulze
Production
Reinhard Gupfinger
Technical Support
Mar Canet
Anika Hirt

University of Art and Industrial Design, Linz


SOMEWHAT RELATED
TuioKinect, by Martin Kaltenbrunner: "A simple Tuio hand gesture tracker for Kinect"
Reactable 

Multitouch and Gesture News (Part I) : NUITEQ Updates

In this post, I'd like to share some news from NUITEQ, a privately held multi-touch software technology that is based in Sweden.  NUITEQ is known for Snowflake Suite, a multi-touch application that is used in a variety of situations.  


"NUITEQ is as happy as a kid in a candy shop to announce that it has released version upgrade 1.9.4 of its award-winning, industry leading multi-touch software product Snowflake Suite" - Harry van der Veen, NUITEQ


The screen shot below gives a nice view of the range of applications that included in Snowflake Suite:



-Photo from NUITEQ


DEMONSTRATION OF SNOWFLAKE SUITE




RELATED
NUITEQ's CTO Mikael Bauer nominated for the Mer-IT award in the category IT-person of the year
NUITEQ Awarded CONNECT Norr Company of the Year 2011
NUITEQ's Partners
From the NUITEQ Gallery, via Harry van der Veen

Cute TV Commercial! Intel Gesture Recognition Technology

This was the first television commercial I saw on TV this morning. I had to stop my morning routine to watch it : )



RELATED
From ChannelIntel: Interview of Francis MacDougall, CTO of GestureTek

Window Shopping in the Web Outside: Interactive Window Displays (videos and links)

This is one of my favorite interactive window display projects:

Interactive Window Concept made for the module Advanced Interface Design at Hyper Island hosted by North Kingdom. Shot with a RED ONE camera at 120fps - Viralmente

Scion Interactive Window Campaign - Monster Media


Starbuck's Interactive Window Display PSFK Salon, 2/7/11

Starbucks Interactive Storefront from The Media Merchants on Vimeo.
Creative Agency:  Blast Radius;  Media Agency: MediaCom; Execution and Interactivity: The Media Merchants














Video: Gesture Interactive Window Display at a Repetto Store in France, by the Marcel Agency

http://www.repetto.com/boutique/home

Chanel Interactive Windows

"Some of the interactive shopping windows conceived & deployed by Vanksen (www.vanksen.com) for Chanel Coco-mademoiselle perfume relaunch."
\
LM3Labs' Ubiq'window

Ubiq'window installation from Nicolas Loeillot on Vimeo.


Hamleys Interactive Summer Window Display - One Off, Kingston University

Interactive Window Shopping: Just Wave Your Hands -Science Daily, 1/17/11

(Credit: Fraunhofer HHI)

EyeTouch Interactive Window Solution (Eyeclick)


Interactive Window Projection- Diesel


Interactive Catalog Window, by YDreams


Windows 7 Phone Launch: Interactive Window Display


Yourwindow, Vision2Watch

Interactive window from Vision2Watch on Vimeo.


Please leave a comment and a link of any other interactive window displays for my "collection"!

Video: Gesture Interactive Window Display at a Repetto Store in France, by the Marcel Agency

The video below features an interactive window created by the Marcel agency for a Repetto store in France.  The display allows passers-by the ability to interact with the content in the display through arm gestures via motion sensors.  


Repetto - Vitrine interactive. from Marcel on Vimeo.

Why bother switching from GUI to NUI? - Asked and Answered by Josh Blake; My 2-cents; Stevie B’s Microsoft Research Video; Marco Silva’s NUI-HCI Presentation (and links)

In Chapter 1 of Natural User Interfaces in .NET,  Josh Blake asks and answers a question posed by many people who have been under the spell of keyboard input and GUI/ WIMP interaction: 


Why bother switching from GUI to NUI?  The answer?  Read Chapter 1 (pdf) of the book - the chapter is free.


Here are a few of my personal reasons:  
1.  I want to buy the next version of the iPad or something like it.
2.  I want to buy a new large-screen Internet HD TV.

3.  I want to buy a Kinect.

4.  I do NOT want to interact with my new TV with a Sony remote.  Too many tiny buttons!

5. I do NOT want to interact with my new TV with a keyboard,  because it reminds me of...work.

6.  Most importantly: 

I want to design apps for the people I care about, and others with similar needs:

    My mom.  

    My grandson.

Moms and dads with kids in tow.
People with special needs and/or health concerns, and the people who care and guide them.
Knowledge sharers and (life-long) learners....

RELATED

"Smart" Interactive Display Research

How Natural User Interfaces are changing Human Computer Interaction

 

View more presentations from Marco Silva

My YouTube Playlist:

Post-WIMP Explorers' Club Videos

"Natural user interfaces, gesture interaction, multi-touch, natural interaction, post WIMP examples and more... "

Interactive Multimedia Technology Post:
Interactive Touch-Screen Technology, Participatory Design, and "Getting It": Revisited

RELATED - and somewhat related   

Moms with Apps

Design for Care

SparkOn

NUI Group

UX Matters      "Why don't usability problems get fixed"

Encyclopedia:  Human Computer Interaction, Interaction Design, User Experience, Information Architecture, Usability and More (Interaction-Design.org)

Josh Blake's Blog: Deconstructing the NUI    Book: Chapter 1 (pdf)  Free!
Blake.NUI

"Blake.NUI is a collection of helpful controls, utilities, and samples useful for multi-touch and NUI development with WPF, Surface, and Silverlight."

OpenKinect Wiki

OpenKinect Community Founder Joshua Blake Comes Up With His Own Version of Win 7 Multitouch on Kinect   (Madhav K, KinectHacks 1/14/11)

Kinect SDK Could Open New Worlds for Motion-Controlled Interfaces (Mike Martin, TechNews World, 2/22/11)

Seth Sandler

Seth Sandler's Blog

Seth Sandler's  Ultimate List of Interactive Tables

Dan Saffer

Bill Buxton

Johnny Chung Lee's Procrastineering Blog
Adam Greenfield's Speedbird Blog

 (This is not an inclusive list.)


GUI to NUI Post-WIMP Manifesto:  TBA

NCSU D.H. Hill Library has a Perceptive Pixel Multitouch Display (updated)

NCSU 'sandbox' lets students touch the technology

Jay Price, Charlotte Observer, 2/22/11

"The Perceptive Pixel display wall, which was switched on this week, is the $100,000 centerpiece of the new Technology Sandbox at the D.H. Hill Library. The sandbox is a room of cutting-edge, interactive hardware that's aimed partly at familiarizing students with the latest high-tech hardware, particularly large displays and gesture-based computing. It was paid for with a federal grant from the Institute of Museum and Library Services that is administered by the State Library of North Carolina."

Credit: Corey Lowenstein - clowenst@newsobserver.com

RELATED
Technology Sandbox introduces NCSU students to latest gadgets
WRAL Techwire, 8/25/10
New Learning Spaces for New Learning Styles (pdf)
Terry B. Hill, Mohan Ramaswamy,  NCSU

HUNT LIBRARY

North Carolina State University Libraries
Perceptive Pixel