Interactive Multimedia Journalism: NY Time's 5-part series, "Held by the Taliban"

I came across a link to this information below  via the Innovative Interactivity blog:



-NY Times

David Rohde, a NY Times reporter, was travelling in Afghanistan to interview a Taliban commander when he was kidnapped with two Afghan colleagues on November 10, 2008.  He reported his story in traditional format, Held by the Taliban:  7 Months, 10 Days in Captivity  in print and on-line in a series that spanned over five days, along with an epilogue

I found that in just about 10 minutes,  the on-line multimedia version of the Rohde's story provided me with a deeper understanding of the complex nature of the conflicts in the region, as well as a sense of what what he might have felt during his long ordeal. 

Rohde tells the story through narrative, photographs, animated maps, video scenes of his experiences, propaganda-like content from the videos watched by young men of the Taliban, and a 3D representation of how his escape from captivity unfolded.The on-line interactive feature included five video clips, each corresponding to one of the articles of the series.  What I appreciated about the video clips was the opportunity to  view each of the video clips sequentially, or in an any order.   The animated maps provided a meaningful geographic context to the story, and the fact that the story was narrated by the author gave it a sense that the conflict in and around the Afghan region is immediate, real, and urgent.

Rohde's story continued with a Q & A on a NY Times blog, further enhancing the meaningfulness and life of the story.   Before the digital era, a story like this would have been thrown out with the paper, or watched by a few on a television news documentary, remembered by a handful of historians, and forgotten by most people. 

Interactive multimedia journalism provides a chance for readers/viewers/users a glimpse of history and culture related to the news story,  and also provides a means of documenting history-in-the-making for current and future generations.  

As I read Rohde's story, watched the video clips, and viewed related multimedia content, I thought about the  increase in the number of people who now access the internet, including the NY Times, from their mobile devices.  I wondered how story might be interpreted through the small screen, and also wondered how Rhode's multimedia story and others like it could be played out on  screens installed in public spaces.  

With some tweaking, this form of multi-media journalism could be accessed on large screens, or even touch-walls, in variety of locations.  Airports, trains, visitor centers, museums, libraries, and shopping malls come to mind as places where this might be useful.   Similar interactive screens are out there, such  the GoBoard digital concierge at the Courtyard Marriott, created by Four Winds Interactive using Microsoft's interactive technologies.

-Mariott GoBoard, Microsoft



-Hard Rock Cafe's Memorabilia Wall, Engaget

RELATED

Charlie Rose interview with David Rohde

SOMEWHAT RELATED
Sensory-Mind's Ring Wall, an interactive multi-touch wall you don't even have to touch!
The Ring Wall is an interactive information display that looks like it could support an interactive multimedia news story.

How to recreate past events with multimedia

The link above is to a post by Tracy Boyer, the author of the Innovative Interactivity blog, and discuss techniques that multimedia content producers can use to recreate past events, along with some examples.

Tracy Boyer's Blogroll

I'm sure that many of my readers will appreciate this list!

• 10,000 words
• Adam Westbrook
• Design Reviver
• Duckrabbit
• FlowingData
• Interactive Narratives
• Journerdism
• Kennedy Media
• KobreGuide
• Mashable
• Mastering Multimedia
• MediaShift
• MediaStorm
• Multimedia Standards
• MultimediaShooter
• News Videographer
• ReadWriteWeb
• Smashing Magazine
• Teaching Online Journalism
• Visual Editors
• Visual Journalist

Sensory-Mind's Ring Wall, an interactive multi-touch wall you don't even have to touch!

Sensory-Minds is a small company in Germany that is focused on research and design in the field of Natural User Interfaces. If you visit the SENSORY-MINDS website, you'll find that it has been designed for touch interaction.

ring°wall from SENSORY-MINDS on Vimeo.

Information about the Ring Wall from Sensory-Mind's Vimeo site:

".....The two-piece ring°wall consists of a LED display and a multitouch information-wall and impresses by its size: a total surface of 425 square meters, which equals more than 6000 computer displays, is the biggest of its kind. An interactive World emerges out of 34 million pixels generated by 15 high definition projectors and is supported by 30 directional speakers.


Multitouch sensors basing on laser technology, convert the usage of the natural user interface into an experience. By direct touching, more than 80 users can simultaneously get informed about news and activities around the ringworld.


The interactive wall is not only a central information system, but also an innovative advertising tool and medium for public viewings."

Heiko Hoffman of Sensory-Minds recently joined the NUI-Group.  Here is a response to a question on the NUI-group forum about the way the system's sensors work:


"The sensor works like spinning radar gun, or like a wiper in a car, that means like a radar gun you get the distance to the object.  It’s not quiet difficult when you know the position of the sensor and the degree from the beam to get the X,Y position.  Yes, you don’t need to touch the surface but we arranged it that the radar beam is very close to the screen so it seems that you have to touch the surface.

At the moment the system (has) limitations because each sensor (gets) coordinates and this means that you got shadows.This is no problem because the people interact very fast. We are working on a system where the data from the sensors were put together and that would be the solution for this problem."


Note:

I'll be writing a few more posts updating the work of past and present NUI-group members as well as news from the commercial multi-touch & gesture community.  Be sure to check back soon, or subscribe to this blog!

Locative & Mobile Media and Martijn de Waal's The Mobile City blog

I subscribe to Martijn de Waal's blog, The Mobile City:  Locative & Mobile Media/Urban Culture/Identity, partly because of my interest in ubiquitous/pervasive computing (specifically how technology can support interaction and collaboration between people across screens of all sizes in public spaces), and partly because I am interested in looking at the ways different disciplines adopt, adapt, and transform emerging interactive technologies.

If you are interested, take a look at my recent post about Martijn de Waal and topics related to locative/mobile media on The World Is My Interface blog.

UX of ITV: The User Experience and Interactive TV (or Let's Stamp Out Bad Remote Controls)

I prefer to watch our flat-panel HDTV in the dark, and usually I watch something I've DVR'd.  Because of the nature of my remote control, I often have to interrupt the immersive experience, turn on the light, poke at the buttons, and start over again.


Not long ago, I had the flu and I thought I'd try out the interactive channel from my satellite TV provider.  My interactive experience was about the same as interacting with the DVR!

What the satellite TV company offered was not really ITV.  It was BIRC.  Bad Interactive Remote Control.  You know what I'm talking about.  In the era of the WiiMote, most of us still have to interact with our TV systems as if were 1982.

There is hope!
A good number of researchers are working hard to make the UX of ITV, including mobile TV, a reality.   If you are reading this post, you most likely will appreciate some of the articles and links below:
Konstantinos Chorianopoulos. Research Methods in Interactive TV (pdf)
(Konstantinos Chorianopoulos is a lecturer and  Marie Curie Fellow in the Department of Informatics at the Ionian University, Corfu, Greece.  He founded UITV.INFO,  a site that has research and information about interactive television.)
Enhancing Social Sharing of Videos: Fragment, Annotate, Enrich, and Share (Proceedings of the 16th ACM international conference on Multimedia) Pablo Cesar, Dick C.A. Bulterman, David Geerts, Jack Jansen, Henrick Knoch and William Seager (This research paper includes a discussion of the concept of "personal secondary screens" on mobile devices that display information that can be shared and annotated while watching content on a large display.)


Sample of presentations and workshops at EuroITV 2009:
The Connected Home Redefines the TV Experience  Jan Van Bogaert (Alcatel-Lucent) 
The Internet Revolution Will Be Televised   Rich Exekiel (Yahoo! Conntected TV)
Needs, emotions, experience!  Marc Hassenahl (Folkwang University)
Marian F. Ursu, Pablo Cesar, and Doug Williams.Enhancing Social Communication and Belonging by Integrating TV Narrativity and Game Play
Rodrigo Laiola Guimaraes. Telling Stories and Commenting on Media:  The Next Generation of Multimedia Authoring Tools (pdf)
Ana Vitoria Joly.  Designing iTV Interfaces for Preschool Children
Claus Knudsen and Roel Puijk. Television and Presence: Experiments in interaction and mediation in a digital environment

Marianna Obrist, Henddrik Knoche, Damien Alliez Tutorial: User-Experience in TV-centric Services: What to consider in the Design and Evaluation?(pdf) 
David Geets Tutorial: Designing and Evaluating the Sociability of Interactive Television (pdf)
Artur Lugmayr Tutorial:  Ambient Media - An Introduction by Case-Studies(pdf)
Janez Zaletelj, Mladen Savic and Marko Meza. Real-time Viewer Feedback in the iTV production.
Skylla Janssen. Interactive Television Format Development – Could Participatory Design Bridge the Gap?
Jan Hess and Volker Wulf. Explore Social Behaviour around Rich-Media: A Structured Diary Study 
Dimitri Schuurman, Tom Evens and Lieven De Marez. A living lab research approach for mobile TV

RELATED 
Ana Vitoria Joly. Design and Evaluation of Interactive Cross-platform Applications for Pre-literate Children.  IDC 2007 Proceedings: Doctoral Consortium
Ana Vitoria Joly. Interactive Cross-platform Environments for Young Children (pdf)
C. Hesselman, W. Derks, J. Broekens, H. Eertink, M. Guelbahar, and R. Poortinga, "Facilitating an Open Ecosystem to Enhance Interactive TV Experiences", Workshop on Sharing Content and Experiences with Social Interactive Television, co-located with the European Interactive TV Conference (EuroITV2008), Salzburg, Austria, July 2008
R. Kernchen, P. Cesar, S. Meissner, M. Boussard, K. Moessner, C. Hesselman, and I. Vaishnavi, "Intelligent Multimedia Presentation Delivery in Ubiquitous Multi-Device Scenarios," IEEE MultiMedia (IEEE MM), 17(2), April-June, 2010 [in print]
P. Cesar, D.C.A. Bulterman, and J. Jansen, "Leveraging the User Impact: An Architecture for Secondary Screens Usage in an Interactive Television Environment," in Springer/ACM Multimedia Systems Journal (MSJ), 15(3): 127-142, 2009

P. Cesar, D.C.A. Bulterman, and Luiz Fernando Gomes Soares, "Introduction to special issue:  Human-centered television-directions in interactive digital television research"  ACM Transactions on Multimedia Computing, Communications, and Applications, October 2008

Somewhat Related
My preliminary thoughts about  Adobe's Open Screen project

libTISCH, a multi-touch development framework with multi-touch widgets and more!

For techies and the tech-curious who like technologies that support collaboration and multi-touch interaction,  this is great news!

Florian Echtler announced the first stable releas of libTISCH, a multi-touch development framwork, which can be found on Sourceforge.  TISCH stands for Tangible Interaction Surfaces for Collaboration between Humans.  libTISCH, a C++ software framework, is included in this project.  It provides a means for creating GUIs based on multi-touch and/or tangible input devices.

Here is how it works:

Here is information from libTISCH announcement:


Highlights of this release are, among others, the following features:

- ready-to-use multitouch widgets based on OpenGL
- reconfigurable, hardware-independent gesture recognition engine
- support for widely used (move, scale, rotate..), pre-defined gestures
 as well as custom-defined gestures

- hardware drivers for FTIR, DI, Wiimote, DiamondTouch..
- TUIO converters: source and sink

- cross-platform: Linux, MacOS X, Windows (32 and 64 bit)
- cross-language: C++ with bindings for C#, Java, Python

libTISCH has a lot to offer for the multitouch developer. For example, 
the textured widgets enable rapid development of applications for many
kinds of multi-touch or tangible interfaces. The separate gesture
recognition engine allows the translation of a wide range of highly
configurable gestures into pre-defined or custom events which are then
acted on by the widgets. While the lower layers of libTISCH provide
functionality similar to tbeta, touche etc. (you can interface existing
TUIO-based software with libTISCH in both directions), it goes far
beyond.

More information about the library and underlying architecture can be
found on http://tisch.sf.net/ and in the Sourceforge wiki at
http://sourceforge.net/apps/mediawiki/tisch/

TISCH Project Wiki

RELATED
Florian is on the scientific staff at the Technisch Universitat Munchen in Germany. Be sure to check out his  webpage.

I especially like the concept of the MeTaTop: "A Multi-Sensory Table Top System for Medical Procedures" that is linked from Florian's website.

Interactive Multimedia Across Platforms and Screens: Adobe's Open Screen Project; MEX Mobile User Experience Manifesto.... (Please don't annoy the user!)

I'm not sure what I think about ubiquitous Flash 10.1. and Adobe's Open Screen project. I like the idea of anything that is seamlessly cross-platform, but I shudder to think that this might let out a wave unwanted or annoying "push" advertising on on-the-go screens of all sizes. I'm assuming web developers, along with TV ad producers, will be jumping on this train without fully thinking about how their applications and designs will play out in the off-the-desktop, digital-out-of home world.

I decided to take a look, drill down through the hype, and share a few links related to this topic.

Adobe Pushes for a Flash-ier Mobile Web
Rob Pegararo, Faster Forward, Washington Post (10/5/09)
"Are you anxious to bring Flash to the mobile Web, even if it means being subjected to some over-eager Web coder's song-and-dance routine? Or would you rather do without it on the go, even if that means having to switch to a "real" computer to use some Web sites' features?"

Hopefully the "over-eager web coders" will heed the MEX Manifesto:

MEX:  Mobile User Experience 2009 Manifesto (pdf)
"The Manifesto sets out our beliefs as to how user-centred design principles can enhance the experience of multi-platform digital services."

A framework for user journeys in a multi-platform world:  Marek Poawlowski, founder of MEX

MEX: User experience journeys in a multi-platform environment from Marek Pawlowski on Vimeo.

"User experiences are evolving into increasingly complex sets of interactions between multiple devices.  In this video presentation, Marek Pawlowski of the MEX Mobile User Experience strategy forum, shows how a framework can be used to map user journeys through the multi-platform environment."

"Unencumbered by wires, information is flowing into every corner of our world at an ever increasing rate and through an ever increasing range of digital platforms. The single greatest challenge facing digital industries is understanding how this explosion of data will be woven into the fabric of consumers' lives." -- Marek Pawlowski, founder of MEX.

MEX Blog 

OPEN SCREEN VIDEO

Open Screen Project from Vyshak V on Vimeo.

"The Open Screen Project is an industry-wide initiative, led by Adobe and backed by other industry leaders who all share one clear vision: Enable consumers to engage with rich Internet experiences seamlessly across any device, anywhere. Partners in the Open Screen Project are working together to provide a consistent runtime environment for open web browsing and standalone applications — taking advantage of Adobe® Flash® Player and, in the future, Adobe® AIR®. This consistent runtime environment will remove barriers to publishing content and applications across desktops, mobile phones, televisions, and other consumer electronics." Learn more

Reinventing Storytelling in the Digital Age Across Platforms, Across Screens

NAB 2009 presentation by Shantanu Narayen of Adobe and A.D. Albers, of Disney Interactive Media Group, from NAB 2009
Adobe and NVIDIA Deliver Rich Web Experiences on Netbooks and Mobile Devices
Reuters (10/5/009)
"At Adobe MAX, Adobe's worldwide developer conference, Adobe Systems Incorporated and NVIDIA Corporation..announced that both companies are bringing uncompromised browsing of rich Web content to netbooks, smarphones and smartbooks built with NVIDIA GPUs. The companies have been working closely together as part of the Open Screen Project to optimize and dramatically improve performance of Flash Player 10.1 by taking advantage of GPU video and graphics acceleration on a wide range of mobile Internet devices. NVIDIA customers embracing Flash Player 10.1 for their new devices include HP, Lenovo, Samsung, Acer, Asus and more..."


RIM Joins Open Screen Project  Reuters (10/4/09 )


Honey I Shrunk the Flash Player Simon Bisson and Mary Branscome, ZDNET, 10/12/09

Teaming up with Adobe and the Open Screen Project -Google Blog   Bill Coughran (10/5/09)


Paramount Digital Entertainment Launches Interactive Thriller on MySpace  Tracy Sedlow, InteractiveTV Today (10/28/09)
"The company says that it can deliver the show's multiple interactive elements to viewers across devices using Adobe Flash Player and Adobe AIR, "because of efforts by the Open Screen Project, an industry-wide initiative led by Adobe and supported by PDE and close to 50 other industry leaders, to enable people to engage with rich Internet experiences across any device, anywhere." -