Interactive Technology from CHI 2012 Supporting Literacy, Storytelling, and Narrative

I spent the last several days at CHI 2012 and was so immersed in the experience, I held off blogging until I arrived home.  


I was pleasantly overwhelmed by the experience at CHI 2012, There was so much to see, hear, and touch, there were so many bright, creative people all around - I was on mind/sensory overload.  Omar L. Gallaga, the author of Digital Savant (Austin 360), attended the conference, and the quote below from his recent post nicely sums it up: "When your mind gets blown multiple times in a very short period of time, it begins to feel as is your brain is quickly rewiring itself to accept that the reality is that anything is possible and that the continual, pleasurable surprise of discovery is the new norm."


For this post, I'm sharing a few things that were presented at CHI 2012 that focus on literacy, storytelling, and narrative.  I'll share more in future posts.


Creating and Using Interactive Narratives: Reading and Writing Branching Comics 
The short video below provides a good overview of the project, which was presented at CHI 2012 during the Out of the Box" session, chaired by Shahram Izadi, of Microsoft Research USA

"We employ comics and combine paper with a multi-touch interface to explore an approach to reading and writing interactive narratives."
Dan Andrews and Chris Baber from the University of Birmingham (UK) 
Sergei Efremov and Mikhail Komorov from the Moscow State Institute of Electronics and Mathematics (Russia).


If you are interested in learning more about this process, take a look at the references that were reviewed in the paper about this creative, thoughtful work.

Tap & Play: An End User Toolkit for Authoring Interactive Pen and Paper Language Activities 
The following video was created several months before CHI 2012:




Tap & Play:  An End User Toolkit for Authoring Interactive Pen and Paper Language Activities
Ann Marie Piper, Nadir Weibel, James Hollan, 


The following video is related to the Textual Tinkerability paper, presented during the Literacy on the Margin session, chaired by Juan Pablo Hourcade, at CHI 2012:
 


Textual Tinkerability:  Encouraging Storytelling Behaviors to Foster Emergent Literacy    Angela Chang, Cynthia Breazeal, Fardad Faridi, Tom Roberts, Glorianna Davenport, Henry Lieberman, Nick Montfort, Massachusetts Institute of Technology


Related: TinkRBook: Shared Reading Interfaces for Storytelling (IDC 2011)
TinkRBook

RELATED

CHI 2012:  brain expanding bits of brilliance
Omar L. Gallaga, Digital Savant, 5/11/12


FYI:   I was one of the organizers for the EIST (Educational Interfaces, Software & Technology) workshop, held on May 5th and 6th. The participants came from all around the world to spend two days of presenting, sharing, and brainstorming.  Everyone worked hard to make the workshop a success. (I'll share more about EIST soon, but if you are curious, the above link will provide you with more information about the program and papers that were presented.) 

Revisiting CHI 2011: Videos of Interactive Touch, Gesture, Large Surface, and Mobile Apps with Potential for Use in Education (CHI = Computer Human Interaction)

One of my interests is how the power and potential post-WIMP interactive technologies can be harnessed for use for formal and informal education purposes, including life-long collaborative learning.  


In May, I had a chance to meet with a number of like-minded people during the CHI 2011 conference at the 2nd Workshop on UI Technologies and Impact on Educational Pedagogy.  I was impressed with the depth and breadth of the presentations at the workshop.   Since then, I've been looking through other papers and videos from CHI 2011 to find interesting applications that hold potential for use in educational settings.  


I've come across a good number of interesting applications and prototypes, so be sure to check back for future posts on this topic.  For now, here are a few applications that I'd like to share.  


Below are a few videos from Phillip Chi-Wing Fu.  (He doesn't know it yet, but I've admitted his videos into the Post-WIMP Explorers' Club.)


Interactive Multi-touch Sketching Interface for Diffusion Curves

"A novel multi-touch sketching interface enabling interactive and practical design with 2D diffusion curves is proposed; featured interaction techniques include simultaneous sketching of multiple diffusion curves and at-the-spot colors tuning."


Distinguishing Multiple Smart-Phone Interactions on a Multi-touch Wall Display using Tilt Correlation

"This paper proposes a novel matching technique, called tilt correlation, which employs the built-in tilt sensor on smart-phones to identify their concurrent contacts on a common multi-touch wall display."


WYSIWYF: Exploring and Annotating Volume Data with a Tangible Handheld Device (CHI 2011)


"Integration of a multi-touch wall display with a tangible handheld device with multi-touch and tilt sensing capabilities to provide intuitive what-you-see-is-what-you-feel visual exploration and annotation of volume data."


The following videos were uploaded by alucero:


Pass-Them-Around: Collaborative Use of Mobile Phones for Photo Sharing (CHI 2011)

"Pass-Them-Around is a phone-based application that allows a small group of collocated people to share photos using the metaphor of passing paper photos around. The prototype encourages people to share their devices and use them interchangeably while discussing photos face-to-face. The prototype supports ad-hoc photo sharing in different contexts by taking into account the spatial arrangement of users around a table, measured with sensors embedded in their mobile phones."


The next video was part of MobileHCI '10:
MindMap: Collaborative Use of Mobile Phones for Brainstorming

CHI 2011, Bill Buxton, and the Buxton Collection: Explore 35 years of interactive devices, online!

Bill Buxton is a researcher at Microsoft who focuses on Human-Computer Interaction and is known for his work in user experience design and natural user interfaces such as multi-touch surfaces.   His talk at the recent CHI 2011 conference, held in Vancouver, Canada, was an overview of the Buxton Collection, an on-line historical archive of interactive input devices spanning over the past 35 years.  


It was interesting to note that at the time of the presentation, the Vancouver Conference Center, where the conference was taking place, was having serious problems with the network/internet connections, and as a consequence, Buxton was not able to demonstrate the on-line version of his collection as planned.  

Not to worry.  The physical version of the Buxton's archive was on display during the conference, along with Buxton, who was happy to tell the story behind every device and gadget in the archive, with much enthusiasm. The slideshow below provides a glimpse of the Bill Buxton archive displayed at CHI 2011:

My Buxton Collection Slideshow, CHI 2011, Vancouver, Canada

Buxton's archive of gadgets comes with a rich history, accumulated over the years. Much of this history, until now, has resided in Buxton's head.   Holding and touching the items in the archive while listening to Buxton's passionate stories about each one, was unlike anything I had ever experienced.  His archive is a labor of love, and the interactive, on-line version of the Buxton Collection is his way of sharing his knowledge with the world.


During his talk, Buxton pointed out that in computer science programs, students are not required to have much exposure to the "history of ideas" in the field.  Huge chunks of work are often ignored in the literature,  and in some cases, the wheel is unknowingly reinvented, and this is something that must be addressed within the CHI community, according to Buxton. 


I agree.

RELATED
Previous IMT posts about Bill Buxton
Bill Buxton's Presentation Video: "A Little Tale about Touch" (Microsoft Worldwide Partner Conference, 2010)
Two good articles by Bill Buxton: The Mad Dash Towards Touch Technology; The Long Nose of Innovation
Buxton Collection
Buxton Collection, PivotViewer
Back to the Past: Bill Buxton Shows Off 36 Years of Tech Devices
Microsoft News Center, 5/9/11
Microsoft's Bill Buxton exhibits gadget collection 35 years in the making
Donald Melanson, engadget, 5/9/11
Bill Buxton's Haptic Input References (pdf)
Bill Buxton's website
Multi-touch Systems that I Have Known and Loved (Bill Buxton)
CES 2010: NUI with Bill Buxton

On Engineering and Design: An Open Letter Microsoft Research Principal Scientist Bill Buxton calls for engineers and user experience designers to learn to appreciate one another

Bill Buxton, Bloomberg Businessweek, 4/29/09

humanaquarium at CHI 2011

I had the opportunity to attend the presentation session of humanaquarium: Exploring audience, participation, and interaction at CHI 2011, and also experience humanaquarium's compelling interactive performance along with a crowd of CHI 2011 attendees. It was one of my favorite events.



What I liked about humanaquarium is that it is a good example of how music, the arts, and technology are converging, on many levels.  It is encouraging to know that this sort of interdisciplinary, creative collaboration is evident at a number of university research centers. humanaquarium was developed with the support of Culture Lab, Newcastle University (UK),  the Advanced Man-Machine Interface Laboratory, University of Alberta, Edmonton, Canada, and the Alberta Foundation for the Arts.


According to the abstract for the CHI 2011 paper, "humanaquarium is a movable performance space designed to explore the dialogical relationship between artist and audience. Two musicians perform inside the cube-shaped box, collaborating with participants to co-create an aesthetic audio-visual experience. The front wall of the humanaquarium is a touch-sensitive FTIR window. MaxMSP is used to translate the locations of touches on the window into control data, manipulating the tracking of software synthesizers and audio effects generated in Ableton Live, and influencing a Jitter visualization projected upon the rear wall of the cube."

To learn more about humanaquarium and catch a glimpse of the performance, take a look at the following video clips.  The first video clip was produced by Nick Barber, who was covering the CHI 2011 conference for IDG News Services. (Nick is also the host and co-producer of World Tech Update.)  I produced the second video clip.

humaniquarium:  Robyn Taylor/Guy Schofield/John Shearer

RELATED
humanaquarium: exploring audience, participation, and interaction
CHI EA '11 Proceedings of the 2011 annual conference extended abstracts on Human  Factors in Computing Systems
CHI 2011 The User in Flux
humanaquarium YouTube channel
Humanaquarium blends art and tech, attracts crowds
Nick Barber, Computerworld, 5/13/11
Humanaquarium: Two musicians, one box
Nate Lanxon, Wired, 3/16/10

Johannes Schoening's CHI 2011 Highlights, Linkfest, and More!

Johannes Shoening has posted some great videos, pictures, and links about ACM-CHI 2011, a conference that recently took place in Vancouver, Canada.  Johannes is the author of the Perspectivevoxel blog, where he shares information about interactive and emerging technologies.


Weekly Linkfest (CHI 2011)


CHI 2011 Highlights in the Area of Interactive Tables and Surfaces




Take a look at a recent article about CHI 2011, by Terry Lavender,  in the on-line version of the Vancouver Observer:


CHI: Computer-Human Interaction shows stunning use of interface technology (5/14/11)

Note:  I have several post about CHI in the works.  Sorry for the delay, I got behind when Blogger was down.