Digital Storytelling, Multimodal Writing, Multiliteracies...

Digital storytelling, multimodal writing, and multiliteracies are overlapping concepts that weren't around during my first round as a university student. As more people of all ages create and share digital content on the web in new and imaginative ways, teachers and university scholars have taken notice. Is there a consensus that the printed word, as we've known it, is in the middle of a digital transformation?

Let's start out with digital storytelling.

By now, everyone knows about YouTube and vlogs as new means of communication. There is more to digital storytelling than uploading a few hastily put-together video clips from the family camcorder, or slapping together a PowerPoint presentation with a few bells and whistles. There are now some standards. Digital storytelling is an art.

The following definition is from an article from EduCause, 7 things you should know about Digital Storytelling.:

• "Digital storytelling is the practice of combining narrative with digital content, including images, sound, and video, to create a short movie, typically with a strong emotional component. Sophisticated digital stories can be interactive movies that include highly produced audio and visual effects, but a set of slides with corresponding narration or music constitutes a basic digital story. Digital stories can be instructional, persuasive, historical, or reflective. The resources available to incorporate into a digital story are virtually limitless, giving the storyteller enormous creative latitude. Some learning theorists believe that as a pedagogical technique, storytelling can be effectively applied to nearly any subject. Constructing a narrative and communicating it effectively require the storyteller to think carefully about the topic and consider the audience’s perspective."

Petter Kittle, from the Northern California Writing Project, Summer Institute 2008, touches on the topic of multimodal writing in Multimodal Texts: Composing Digital Documents. Related to this is the concept of digital writing.

"Multiliteracies is an approach to literacy which focuses on variations in language use according to different social and cultural situations, and the intrinsic multimodality of communications, particularly in the context of today's new media."

• "...it is no longer enough for literacy teaching to focus solely on the rules of standard forms of the national language. Rather, the business of communication and representation of meaning today increasingly requires that learners are able figure out differences in patterns of meaning from one context to another. These differences are the consequence of any number of factors, including culture, gender, life experience, subject matter, social or subject domain and the like. Every meaning exchange is cross-cultural to a certain degree." -from Kalantzis and Cope's Multiliteracies website

Here is a short list of resources:
The Center for Digital Storytelling
Multimedia Storytelling
What are multimodality, multisemiotics, and multiliteracies?
(Ben Williamson, Futurelab)
Reading Images: Multimodality, Representation, and New Media
(Gunther Kress)
New Learning: Elements of a Science of Education
(Mary Kalantzis & Bill Cope)
Multiliteracies
The Multiliteracy Project
Multimodal Writing
http://multimodalwriting.com/
(new website, under development)
Multimedia Blogging
(a post from 2004, worth reading for historical context)
Thinking about multimodal assessment
(Digital Writing, Digital Teaching)
Standards related to digital writing
(from Teaching Writing Using Blogs, Wikis...)

I conclude this text-based post with a promise to incorporate more multimedia experiences in my upcoming posts....stay tuned.

MellaniuM: Virtual Reality, History, and Digital Heritage

Joe Rigby, from MellaniuM, focuses on the use of interactive virtual reality technology to create environments that support the learning of history. This concept is also known as "Digital Heritage". Applications are in the works that combine high-polygon modeling with scaled photo-realistic textures, incorporating multi-user avatar interaction within 3D archaeological visualizations.

MellaniuM will be presenting at VSMM' 08: Conference on Virtual Systems and Multimedia Dedicated to Digital Heritage. Instead of a PowerPoint presentation, participants will be provided with a walk-through of the Theatre of Pompeii.

A future workshop, sponsored by ADSIP, the Applied Digital Signal and Image Processing Research Centre, will feature an EONREALITY multi-wall immersive system to display the latest version of the Theatre of Pompeii district. Public VR and Anne Weis ,from the Department of Art History at the University of Pittsburgh, are collaborating on this project.

The original Pompeii Project, was built at Carnegie Mellon's Studio for Creative Inquiry, during the mid-1990's. Links to the Virtual Theater District (VMRL) and 3D models can be found at http://artscool.cfa.cmu.edu:16080/~hemef/pompeii/project.html

If you have an interactive whiteboard, download the 3D models of Pompeii. You'll have to install a free VRML plug-in in order to view them on a web browser.

Wouldn't it be great if all students could learn about history via interactive virtual reality someday?

Microsoft Surface's Hotel Concierge Application: Let's see an affordable Surface, deployed in classrooms and libraries!

Microsoft Surface at the Sheraton Hotel:



Think how this could play out in classrooms, libraries, information centers, and other public spaces.

Need I say more?

(Cross posted on TechPsych and Technology-Supported Human-World Interaction blogs)

Microsoft Research project: MouseMischief - Multi-user, Multi-Mice Interaction on Large Displays

This is an interesting demonstration of the use of multiple mice, controlled by children on an interactive whiteboard. The collaborative application uses Microsoft's Multi-Point technology. For more information and free downloads, go to MouseMischief.org.

The Internet of Surfaces? Microsoft's Pete Thompson discusses screens and surfaces of all sizes.

I came across this video of Microsoft Surface's general manager, Pete Thompson, on the GottaBeMobile website. According to Thompson, the people who worked with TouchWall discussed in the video, were also involved with Microsoft Surface.

Apparently, the Surface and Wall folks at Microsoft aren't sure of what they are doing with screens of all sizes.

(If you are interested in surface form factors, see my previous blog post, Emerging Interactive Technologies, Emerging Interactions, and Emerging Integrated Form Factors.)

There are many unanswered questions from my perspective.

Bill Gates portends that every surface will be a computer, a concept that is echoed in the video. If so, what are people doing to ensure that surface-supporting environments are universally designed?

Before Microsoft and other companies unleash "surface" technology to the masses, they must get a few things right.

Will they?

Will the researchers at Microsoft find out how various screens play out in classrooms, in the community settings, on-the job, and in-between?

For example, large interactive displays in urban and retail settings have the potential to provde people with a rich amount of information about what is around them. These displays serve little purpose if they are user-unfriendly, and no purpose at all if they are not accessible.

If developers and designers are not following basic user-centered design guidelines and usability standards now, how can we expect the nextgen systems of display surfaces to support universal usability? As our population ages, this will be more of a problem.

From what I can tell, there will be more opportunities for people to use their mobile devices to interact with larger screens and surfaces when they are out and about. For example, when I was at the airport recently, I noticed that there was a large display that offered cell-phone ringtone downloads. Microsoft was behind this display.

Interconnectivity and interoperability between devices and screens of all sizes is important to think about. If universal usability guidelines are not followed, our mobile devices will be difficult to use in the world of surfaces.

It isn't much of a leap to see the big picture. Just think about the problems we have with our remote controls and entertainment centers in our homes! We might have to carry all sorts of devices just to get from point A to point B. I'm not kidding. It will not be a pretty sight, especially if the privacy and security issues are not resolved as we move to a world that supports the internet of things.

Or shall I say, the internet of surfaces?

Creative Programming: openFrameworks - AWESOME for interactive multimedia applications!

openFrameworks: Better Tools, Enhanced Creativity, Better Projects: YES. Artists can make tools at the same time they make artwork.

To learn all about this, delve into the video. It highlights interviews with creative people who are using openFrameworks, including their innovative work.


made with openFrameworks from openFrameworks on Vimeo.

If you are working with openFrameworks, or thinking about it, let me know.


This looks like a great tool to use for projects I'm creating for my new HP TouchSmart....

.....and my multi-touch thought experiments ; }














I learned about openFrameworks from Seth Sandler, aka "cerupcat", a member of NUI-group who was chosen to participate in Google's Summer of Code. He's posted about his progress on his AudioTouch blog.

Here is a screenshot of Seth's tracking application, still under development, is the result of porting touchlib, the main tracker used by NUI-Group members, to openFrameworks: