SynergyNet Project: Multi-touch tables and displays designed for education...

Rich White at Greenbush Labs is a strong supporter of interactive whiteboards and displays through his work with EduSim, an interactive 3-D virtual collaborative environment. Here is a link to one of his most recent posts about SynergyNet, a multi-touch table that is designed for use in education:

In the Classroom of the Future: Every Desk is an Interactive Surface



"SynergyNet will integrate ICT into the fabric of the classroom. The new desk with a ‘multi-touch’ surface will be the central component; the desks will be networked and linked to a main smartboard offering new opportunities for teaching and collaboration."

"Several students will be able to work together at a desk as the desks allow simultaneous screen contact by multiple users using fingers or pens. Durham researchers want to create a ‘natural way’ for students to use computers in class. The system encourages collaboration between students and teachers, and a move away from teacher-centric learning."

More about the SynergyNet project:

Smart desks make sci-fi a reality in the classroom

Research is underway at Durham University, located in the UK, to see how the interactive tables can support learning IT concepts. According to the SynergyNet project website, the goals of the research are as follows:

• Aim 1: To create a radically new technology-rich learning environment that integrates with traditional classroom layouts and collective activities.
• Aim 2: To design and implement a new form of user interface for educational multi-touch systems.
• Aim 3: To formulate a new pedagogy that eases transition and movement between teacher-centric and pupil-centric interaction.
• Aim 4: To analyse pupils’ learning strategies to inform fundamental research by capturing data as pupils use the SynergyNet environment.

"The system will also boost equal access in school. In IT, we have found that males have been the dominant actors - interactive classrooms will encourage more females to take part in lessons. It will also enable more disabled students to participate in lessons and allow more personalized learning." - Dr. Elizabeth Burd, Principle Investigator, Director of Active Learning in Computing

Related:
jME Physics - the open-source java-based physics engine used in the SynergyNet system

jMonkeyEngine - the open-source java-based game engine use in the SynergyNet system (the video below contains some clips from the SynergyNet project)

Pupils test multi-touch screens
(BBC Article)

Videogames, Teens, and.... Civic Education?

Videogames have become a serious target for study by researchers. If youhaven't played a video game in years, you might be surprised at the results of a recent study by the Pew Internet & American Life Project and the Civic Engagement Research Group at Mills College.

Major New Study Shatters Stereotype of Teens and Video Games: Game playing is universal, diverse, often involves social interaction, and can cultivate teen civic engagement
- MacArthur Foundation

From the above article:

"A focus of the survey was the relationship between gaming and civic experiences among teens. The goal was to test concerns that gaming might be prompting teens to withdraw from their communities. It turns out there is clear evidence that gaming is not just an entertaining diversion for many teens; gaming can be tied to civic and political engagement. Indeed, youth have many experiences playing games that mirror aspects of civic and political life, such as thinking about moral and ethical issues and making decisions about city and/or community affairs. Not only do many teens help others or learn about a problem in society during their game playing, they also encounter other social and civic experiences:

• 52% of gamers report playing games where they think about moral and ethical issues.
  
• 43% report playing games where they help make decisions about how a community, city or nation should be run.
  
• 40% report playing games where they learn about a social issue. "

"Moreover, the survey indicates that youth who have these kinds of civic gaming experiences are more likely to be civically engaged in the offline world. They are more likely than others are to go online to get information about current events, to try to persuade others how to vote in an election, to say they are committed to civic participation, and to raise money for charity."

Teens, Video Games, and Civics pdf
-Pew Internet & American Life

The Civic Potential of Video Games pdf

I'll post more later!

Special Issue on Multimedia, Media Convergence, and Digital Storytelling (RCETJ)

Mark van 't Hooft, the chair of ISTE's SIGHC, recently shared information and links to an interesting special issue of on-line publications about multimedia related topics:

An Introduction to the RCETJ Special Issue on Multimedia, Media Convergence, and Digital Storytelling (Journal of the Research Center for Educational Technology, Fall 2008 Special Issue)

Video-Based Additional Instruction (Marc Franciszkowicz)

Design and Assessment of an On-line Pre-lab Model in General Chemistry: A Case Study (Juan Antonion Llorens-Molina)

The iPod Project: A Mobile Mini-Lab (Nikhil Sathe and Jorg Waltje)

Media Convergence in a College Newsroom: A Longitudinal Study of Identification and Commitment to a Collaborative Web Site (Fred Endres)

The Educators Coop Experience in Second Life: A Model for Collaboration (Leslie Jarmon and Joe Sanchez)

The Creation and Refinement of a Sustainable Multimedia Process in a Higher Education Environment (Megan R. Bell and Larissa Biggers Schraff)

Using Virtual Worlds to Launch Ideas, Research and Teaching (Anne-Marie Armstrong, Darryl Shreve, and Josh Neds-Fox)

Electronic Portfolios: Engaged Students Create Multimedia-Rich Artifacts (Gail Ring, Barbara Weaver, and James H. Jones, Jr. )

Digital Storytelling as a Gateway to Computer Science (Amy Csizmar Dalal)

Interesting reading!

FotoViewr 3D - Nice on my TouchSmart...

I had a chance to play with my Flickr photos using the beta version of FotoViewr on my HP TouchSmart. Nice.

Here is a video from the FotoViewr website:

Lazybrains 3D game: Another Brain-Computer Interface!

I came across an article about the BCI (Brain-Computer Interface) 3D game, Lazybrains, on the Wired website today. "Brain Scanners, Fingercams Take Computer Interfaces Beyond Multitouch"

LazyBrains was a Digital Media Senior Project of Aaron Bohenick, James Borden, Sachary Brooks, Kenneth Oum, and Jordan Santell, students at Drexel University.

Here is a video:

Game Teaser


Description of the BCI, a fNIR:

• "The Functional Near-Infrared Imaging Device (fNIR) is a technology that was developed at the University of Pennsylvania, but is currently being used by the Drexel University biomedical department. The device shines infrared light into the user's forehead, and records the amount of light that gets transmitted back. The change in the amount of light can be used to deduce information about the amount of oxygen in the blood. When the user concentrates, their frontal lobe needs more oxygen and this change can be detected by the device."

For more information, see the Voxel6 website.


Here is a link to a post about a similar BCI system:
Emotive System's Neural Game Controller Headset: Human-Computer Interface of the Future?

It will be interesting to see how this technology unfolds. In my opinion, it will be quite useful for cognitive rehabilitation, as well as providing access to games for people who have significant physical limitations.

Interactive Touch-Screen Technology, Participatory Design,and "Getting It"....

PLEASE SEE THE UPDATED VERSION OF THIS POST:
Interactive Touch Screen Technology, Participatory Design, and "Getting It", Revisited

There's been some discussion over the reasons why so many people don't understand touch screen, or "surface" computing, even though research in this area has been going on for years.

As the new owner of the HP TouchSmart, I know that I get it.

The research I've conducted in this area suggests that people will "get-it" only if there is a strong commitment to develop touch-screen "surface" applications through a user-centered, participatory design process. In my view, this should incorporate principles of ethnography, and ensure that usability studies are conducted outside of the lab.

This approach was taken with Intel's Classmate PC. Intel has about 40 ethnographic researchers, and sent many of them to work with students and teachers in classrooms around the world. (A video regarding ethnographic research and the Intel Classmate project can be found near the end of this post.)



Where to start?
K-12 classrooms and media centers. Public libraries. Malls. Hospital lobbies and doctor's offices. Any waiting room. Staff lounges in medical centers, schools, and universities. Community festivities and events. Movie theater lobbies. Museums and other points of interests.

I believe we need to take a "touching is believing" approach.

Here are some thoughts:

When I try to explain my fascination with developing touch-screen interactive multimedia applications, (interactive whiteboards, multi-touch displays and tables, and the like), many of my friends and family members eyes glaze over. This is particularly true for people I know who are forty-ish or over.

Even if you are younger, if you never saw the cool technology demonstrated in the movie Minority Report, or if you have limited experience with video games, or if you haven't came within touching distance of an interactive whiteboard, the concept might be difficult to understand.

The reality?

Even people who have the opportunity to use surface computing technology on large screens do not take full advantage of it. Multi-touch screens are often used as single-touch screens, and interactive whiteboards in classrooms are often serve as expensive projector screens for teacher-controlled PowerPoint presentations.

Most importantly, there are few software developers who understand the surface computing approach, even with the popularity of the iPhone and iPod Touch. Most focus on traditional business-oriented or marketing applications, and have difficulty envisioning scenarios in which surface computing would be a welcome breath of fresh air.

Another factor is that not all people entrusted to market surface or touch screen computing fully understand it.


Despite a cool website showing off the goods, Microsoft's Surface multi-touch table has been slow to take off, limiting hope of bringing down the price tag to a price most families or schools could afford. (The picture above depicts an application for the Surface designed for health care professionals, not K-12 science education.)

Although you can't buy a Surface table for your family room, it is possible to buy a TouchSmart.
HP's TouchSmart website is engaging and highlights some examples of touch-screen interaction, but most people don't seem to know about it.

Unfortunately, you wouldn't have a clue that the HP TouchSmart exists browsing the aisles at Circuit City or Best Buy!

When I was shopping for my new TouchSmart, I noticed that from a distance, the TouchSmart looked just like the other larger flat-screen monitors filling up the aisles. The salespeople at both stores were not well-informed about the system. The only reason I knew bout the new TouchSmart was related to my obsession with interactive multimedia touch-screen applications- designing them, developing them, studying them, reading about them, blogging about them.... ; }

More thoughts:

After studying HCI (Human-Computer Interaction), and relating this knowledge to what I know as a psychologist, my hunch is that the "Window Icon Mouse Pointing-device" (WIMP) and keyboard input mind-set is embedded in our brains, to a certain extent. Like driving a car, it is something automatic and expected. This is true for users AND developers.

Think about it.

Suppose one day, you were told that you no longer were allowed to control your car by turning on the ignition, steering the wheel, or using your feet to accelerate, slow down, or stop the car! Instead, you needed to learn a new navigation, integration, and control system that involved waving your hands about and perhaps speaking a few commands.

For new drivers who'd never seen a car before, this new system would be user-friendly and intuitive. Perhaps it would be quite easy for 16-year-old kids to wrap their heads around this concept. For most of us, no. Imagine the disasters we would see on our streets and highways!

When we think about how newer technologies are introduced to people, we should keep this in mind.

In my mind, spreading the word about surface computing is not a "if you build it, they will come" phenomenon, like the iPhone. We can't ignore the broader picture.

From my middle-age woman's vantage point, I believe that it is important that the those involved with studying, developing, or marketing surface computing applications realize that many of us simply have no point of reference other than our experiences with ATMs, airline kiosks, supermarket self-serve lanes, and the like.

(The video clip at the very end of this post provides a good example of touch-screen technology gone wrong.)

Be aware that there are substantial numbers of people who might benefit from surface computing who prefer to avoid the ATMs, airline kiosks, and self-serve grocery shopping.

Realize that the collective experience with technology, in many cases, has not been too pretty. Many people have had such user-unfriendly experiences with productivity applications, forced upon them by their employers, that any interest or desire to explore emerging technologies has been zapped.

My own exposure to interactive "surface" related technology was somewhat accidental.

A few years ago, a huge box was deposited into the room I worked in a couple of days a week as a school psychologist at a middle school. After a week or so, I became curious, and found out that it was a SmartBoard. Until then (2002!), I did not know that interactive whiteboards existed.

The boxed remained unopened in the room for the entire school year, but no worry. I played with the only other SmartBoard in the school, and found a couple at the high school where I also worked. I hunted for all of the applications and interactive websites that I could find, and tried them out. That is when I was hooked. I could see all kinds of possibilities for interactive, engaging subject area learning activities. I could see the SmartBoards potential for music and art classes. With my own eyes, I saw how the SmartBoard engaged students with special needs in counseling activities.

(By the way, if you are working with middle school students, PBS Kid's ItsMyLife website activities work great on an interactive whiteboard.)

A few years have passed, and reflecting on all of my fun experiences with interactive whiteboards, with and without students, I now understand that many teachers still have had limited exposure to this technology.

This school year, many teachers are finding themselves teaching in classrooms recently outfitted with interactive whiteboards, scrambling along with educational technology staff development specialists, to figure out how it works best with various groups of students, and what sort of changes need to be made regarding instructional practice.

For the very first time, interactive whiteboards were installed in two classrooms at one of the schools I work at. One of the teachers I know thanked me for telling her about interactive whiteboards and sharing my resources and links.

If I hadn't let her know about this technology, she wouldn't have volunteered to have one installed in her classroom. It has transformed the way she teaches special needs students.

In the few months that she's used the whiteboard, I can see how much it has transformed the way the students learn. They are attentive, more communicative, and engaged. The students don't spend the whole day with the whiteboard - the interactive learning activities are woven into lessons at various times of the day, representing true technology integration.

Now let's see what happens when all-in-one touch-screen PC's are unleashed in our schools!

Some resources:
HP TouchSmart PC website, with demo
HP's TouchSmart YouTube videos
lm3labs (catchyoo, ubiq'window)
NUI Group (See member's links)
NextWindow
Fingertapps
thirteen23
SmartTechnologies
Perceptive Pixel - Jeff Hans
Microsoft Surface
iPhone
(More can be found by doing a search on this blog or The World Is My Interactive Interface.)

Value of ethnographic research:
Ethnographic Research Informed Intel's Classmate PC
"Intel looked closely at how students collaborate and move around in classroom environments. The new tablet feature was implemented so that the device would be more conducive to what Intel calls “micromobility”. Intel wants students to be able to carry around Classmate PCs in much the same way that they currently carry around paper and pencil." -via Putting People First and Ars Technica

The video below is from Intel's YouTube Channel. Information about Intel's approach to ethnographic research in classrooms during the development of the Classroom PC is highlighted. This approach uses participatory design and allows the set of applications developed for the Classmate PC to reflect the needs of local students and teachers. Schools from many different countries were included in this study.




FYI:

Need for Improvement: User-Unfriendly Information Kiosk Interactive Map

Here are some interesting pictures from lm3labs, which are in my interactive usability hall of fame:

Samsung's new Omnia SDG i900 was re-created in a much larger size, using lm3lab's Ubiq'window touchless technology.

For more about lm3labs, including several videoclips, take a look at one of my previous posts:
Lm3Labs, Nicolas Leoillot, and Multimedia Interaction

Link to "Internet of Surfaces" Photo Examples

Take a look!
INTERNET OF SURFACES: Photo Examples of Screens of All Sizes

More pictures will be added.

Surface Computing, Health, and Hands-on Science Education

"....from the first time I saw surface in Andy Wilson's lab at Microsoft Research, I knew it had healthcare written all over it. It has taken some time to bring together the right developers and partners to apply Surface technology in health, but we are finally there." -Bill Crounse, MD, Microsoft HealthBlog

As I've previously suggested, surface computing would be useful in education K-12 settings. One look at the graphics posted below at a demonstration about Microsoft Surface in Health.

We know that one of the challenges in our public schools is to to encourage more students to take STEM-related courses. (If you are not familiar with the acronym, STEM stands for Science, Technology, Engineering, and Mathematics.) One look at the hands-on graphics below might convince reluctant students to sign up for class!






















The video of the 3D interactive heart simulation can be found at the bottom of the following post:

Microsoft Health Blog

Digital Lighbox for Hospitals - The Multi-touch Future of Electronic Medical Records?

Is this what the future holds for electronic medical records?



I came across this on Richard Bank's blog, rb.trends. This multi-touch display is from BrainLAB AG, a company located in Germany. Here is a quote from Ubergizmo:

"Digital Lightbox replaces the conventional light box used to observe analog x-ray images. Connected to the hospital PACS, the new digital platform can be installed both in meeting rooms and in operating rooms, where clinicians can then access, manipulate, and utilize data for surgery planning. By displaying the human body in 3D, Digital Lightbox helps clinicians to more clearly demonstrate to patients what effects a disease can have and which procedures may be necessary. Digital Lightbox enables clinicians to select the most valuable images from large amounts of existing medical data. Ergonomic touchscreen technology with zoom functionality makes working with data easy and effective. Clinicians can intuitively navigate within pictures and between settings. Image scrolling can be performed with one finger; zooming in and out of images with two. Images from different sources can also be fused easily. A measure functionality enables clinicians to set size and other dimensions."

Something like this would be good for high school science classrooms.

Update:
For more photos of the Digital Ligthbox and the iPlan Net software that supports remote collaboration, visit the Future-Making Serious Games blog.

Digital Students@Analog School videoclip from 2004: Do the sentiments of the students still ring true?

It is the beginning of the school year, the best time of the year to for educators to seriously reflect on the many ways they can play an important role in engaging and inspiring their students.

I learned about the above video today from "Back-to School Tech Ideas for K-5", written by C.C. Long on her Tech Integration in Schools blog, and I thought it was worth sharing. The video was created by several college students in 2004, and can be found on TeacherTube. It is similar in spirit to the videos I included on a post about engaged learning earlier this year.

Here are a few quotes from the students in video:

"We are more visual learners, we use different technologies to express ourselves, we don't use just pen and paper".

"99 % of the teachers do it the old fashioned way of ...sit down, you listen to someone lecture for 40-50 minutes...

"Just lectures, it limits my learning."

"There are several options in expressing yourself and expressing your viewpoints. And I think the university limits that."

"I was given one way, and that was how I had to do it."

"The professor still wants to teach it the same way they learned.."

"I figured I'd have to to write papers, I figured I'd have to do problem sets, but I thought there would be more options."

"It is frustrating, because it doesn't seems that college is accommodating the visual learner"

"Anywhere you go outside of the classroom, the technology is being used. I don't understand why we aren't applying it to class."

"Listen. Sit down and talk with me. Give me a choice to express myself in their class."

"I think it would make it more exciting for them."

"When I become a teacher, I am going to have to learn and assess that students are going to have even more that is accessible to them. And if I don't adapt to that, I'm going to start to lose my students."

My hunch is that many educators still do not feel comfortable keeping up with world of the tech-savvy. To do so takes quite a bit of effort, time, and determination. And frustration. If you've worked in public schools for a while, you know what I mean. Much of the technology that educators have been handed over the years has been teacher-unfriendly.
.
Things are changing.

It has been six years since CAST and the Association for Supervision and Curriculum Development created an on-line "how-to" book about technology and the concept of Universal Design for Learning (UDL). This online multimedia book, Teaching Every Student in the Digital Age, still provides a good foundation for teachers who plan to integrate technology into teaching and learning activities to support all learners.

It is exciting to know that many school districts have initiated study groups provide on-line resources for their teachers to support the implementation of UDL. . At the university level, the concept of using technology to support universal design for instruction is not as alien as it might have been 10 years ago.

Mindful, reflective use of technology, including interactive multimedia technology, can support multiple means of learning, communication, collaboration, and knowledge sharing among among all learners, no matter what age. In turn, an engaging and meaningful environment for learning can be sustained.


So what now?

If you are an educator, it wouldn't hurt to see what new educational applications have arrived at your school. Volunteer to be the teacher who teaches with the new interactive whiteboard. Sign up for the Wi-Fi laptop cart once a week for a semester. Hunt down the digital video cameras and do a search for where the video editing software might be hiding. Don't let the tech-savvy teacher down the hall hog it all, even if you consider yourself to be a technophobe!

Most importantly, establish a relationship with the technology "go-to" person at your school or university department, and see if it is not too late to order a few new technology tools. Find a few other people who have decided to do more with technology this year. Then sign up for a few workshops before your calendar is filled. There are no guarantees, but you just might have the best school year ever.

If you are new to this blog, do a search for what might interest you. I am sure you will find links to information that will help. Be sure to visit C.C. Long's Tech and Integration blog for specific technology related activities you can implement right away.

For reflection, take the time to watch the video clips on C.C. Long's blog that were used during Thinkfinity training.

For more inspiration, you might enjoy following a few of the links below:

Engaged Learning Revisited: Four videoclips for reflection....

Response to Intervention, Universal Design for Learning: Resources for Implementation

Visual Literacy and Multimedia Literacy Quotes - Odds and Ends PART TWO

Engaged Learning and Social Physics: Phun, an Interactive 2D Physics Sandbox

Updated MegaPost - Resources For All: Interactive Multimedia and Universal Design for Learning

Natural User Interface new website shares information about the company's innovative multi-touch solutions...

Frequent visitors to this blog know that I've been following the development of the NUI multi-touch system since it was in the gestational stage, a university project of Harry van der Veen. I'd like to mention that as the NUI multi-touch table has evolved, so has the NUI website.

Take a look at the user-friendly, visually appealing website that showcases the Natural User Interface company's accomplishments, new products, and services. Here is some information from the site:

• "Natural User Interface (NUI) is an innovative emerging technology company specializing in advanced multi-touch software and service solutions. NUI's solutions can convert an ordinary surface into an interactive, appealing and intelligent display that creates a stunning user experience."

• "NUI provides both standardized as well as customized horizontal, vertical and angled multi-touch hardware solutions. With the wide variety of market leading suppliers and partner, NUI gives warranty on all our products."

For those of you who can't wait to dive into programming multi-touch, collaborative applications, take a look at NUI's Snowflake 1.0 software for OEM partners:

• "NUI Suite 1.0 Snowflake is an easy to use, robust, fast and reliable gesture recognition, computer vision, image processing, motion sensing multi-touch software package.It has been tested and developed for over 1,5 year and valued as best in the industry by our global hardware partners."

The Hidden Geometry of Hair: Computer Generated "Hollywood Hair" Unveiled at SIGGRAPH 2008

I came across a link to an article in the UCSD School of Engineering News today and just had to post it.

"HOLLYWOOD HAIR IS CAPTURED AT LAST: DETAILS IN SIGGRAPH 2008 PAPER"

Technology now exists that will enable animators, video-game makers, and film-makers the opportunity to capture images of real hairstyles that look and behave realistically. Below is a series of side-by-side comparisons of computer generated heads of hair and the real thing.

From the article:

"By determining the orientations of individual hairs, the researchers can realistically estimate how the hairstyle will shine no matter what angle the light is coming from. “You can’t just blend the highlights from two different angles to get a realistic highlight for a point in between,” said Chang.
“Instead of blending existing highlights, we create new ones.”


....One possible extension of this work: making an animated character’s hair realistically blow in the wind. This is possible because the researchers also developed a way to calculate what individual hair fibers are doing between the hairstyle surface and the scalp. They call this finding the “hidden geometry” of hair.

“Our method produces strands attached to the scalp that enable animation. In contrast, existing approaches retrieve only the visible hair layer,” the authors write in their SIGGRAPH 2008 paper. An animation of a hairstyle is available as a download from the “hair photobooth” Web site created by Sylvain Paris: http://people.csail.mit.edu/sparis/publi/2008/siggraphHair/

What we need is a real-life hair photobooth. Have a bad hair day? No problem. Walk in, "zap!", and walk out with real Hollywood hair. Maybe the hair scientists are working on it....


SIGGRAPH 2008 Paper citation: “Hair Photobooth: Geometric and Photometric Acquisition of Real Hairstyles,” by Sylvain Paris and Wojciech Matusik from Adobe Systems, Inc., Will Chang, Wojciech Jarosz and Matthias Zwicker from University of California, San Diego; and Oleg I. Kozhushnyan and Frédo Durand from Massachusetts Institute of Technology

Digital Storytelling, Multimodal Writing, Multiliteracies...

Digital storytelling, multimodal writing, and multiliteracies are overlapping concepts that weren't around during my first round as a university student. As more people of all ages create and share digital content on the web in new and imaginative ways, teachers and university scholars have taken notice. Is there a consensus that the printed word, as we've known it, is in the middle of a digital transformation?

Let's start out with digital storytelling.

By now, everyone knows about YouTube and vlogs as new means of communication. There is more to digital storytelling than uploading a few hastily put-together video clips from the family camcorder, or slapping together a PowerPoint presentation with a few bells and whistles. There are now some standards. Digital storytelling is an art.

The following definition is from an article from EduCause, 7 things you should know about Digital Storytelling.:

• "Digital storytelling is the practice of combining narrative with digital content, including images, sound, and video, to create a short movie, typically with a strong emotional component. Sophisticated digital stories can be interactive movies that include highly produced audio and visual effects, but a set of slides with corresponding narration or music constitutes a basic digital story. Digital stories can be instructional, persuasive, historical, or reflective. The resources available to incorporate into a digital story are virtually limitless, giving the storyteller enormous creative latitude. Some learning theorists believe that as a pedagogical technique, storytelling can be effectively applied to nearly any subject. Constructing a narrative and communicating it effectively require the storyteller to think carefully about the topic and consider the audience’s perspective."

Petter Kittle, from the Northern California Writing Project, Summer Institute 2008, touches on the topic of multimodal writing in Multimodal Texts: Composing Digital Documents. Related to this is the concept of digital writing.

"Multiliteracies is an approach to literacy which focuses on variations in language use according to different social and cultural situations, and the intrinsic multimodality of communications, particularly in the context of today's new media."

• "...it is no longer enough for literacy teaching to focus solely on the rules of standard forms of the national language. Rather, the business of communication and representation of meaning today increasingly requires that learners are able figure out differences in patterns of meaning from one context to another. These differences are the consequence of any number of factors, including culture, gender, life experience, subject matter, social or subject domain and the like. Every meaning exchange is cross-cultural to a certain degree." -from Kalantzis and Cope's Multiliteracies website

Here is a short list of resources:
The Center for Digital Storytelling
Multimedia Storytelling
What are multimodality, multisemiotics, and multiliteracies?
(Ben Williamson, Futurelab)
Reading Images: Multimodality, Representation, and New Media
(Gunther Kress)
New Learning: Elements of a Science of Education
(Mary Kalantzis & Bill Cope)
Multiliteracies
The Multiliteracy Project
Multimodal Writing
http://multimodalwriting.com/
(new website, under development)
Multimedia Blogging
(a post from 2004, worth reading for historical context)
Thinking about multimodal assessment
(Digital Writing, Digital Teaching)
Standards related to digital writing
(from Teaching Writing Using Blogs, Wikis...)

I conclude this text-based post with a promise to incorporate more multimedia experiences in my upcoming posts....stay tuned.

MellaniuM: Virtual Reality, History, and Digital Heritage

Joe Rigby, from MellaniuM, focuses on the use of interactive virtual reality technology to create environments that support the learning of history. This concept is also known as "Digital Heritage". Applications are in the works that combine high-polygon modeling with scaled photo-realistic textures, incorporating multi-user avatar interaction within 3D archaeological visualizations.

MellaniuM will be presenting at VSMM' 08: Conference on Virtual Systems and Multimedia Dedicated to Digital Heritage. Instead of a PowerPoint presentation, participants will be provided with a walk-through of the Theatre of Pompeii.

A future workshop, sponsored by ADSIP, the Applied Digital Signal and Image Processing Research Centre, will feature an EONREALITY multi-wall immersive system to display the latest version of the Theatre of Pompeii district. Public VR and Anne Weis ,from the Department of Art History at the University of Pittsburgh, are collaborating on this project.

The original Pompeii Project, was built at Carnegie Mellon's Studio for Creative Inquiry, during the mid-1990's. Links to the Virtual Theater District (VMRL) and 3D models can be found at http://artscool.cfa.cmu.edu:16080/~hemef/pompeii/project.html

If you have an interactive whiteboard, download the 3D models of Pompeii. You'll have to install a free VRML plug-in in order to view them on a web browser.

Wouldn't it be great if all students could learn about history via interactive virtual reality someday?

Microsoft Surface's Hotel Concierge Application: Let's see an affordable Surface, deployed in classrooms and libraries!

Microsoft Surface at the Sheraton Hotel:



Think how this could play out in classrooms, libraries, information centers, and other public spaces.

Need I say more?

(Cross posted on TechPsych and Technology-Supported Human-World Interaction blogs)

Microsoft Research project: MouseMischief - Multi-user, Multi-Mice Interaction on Large Displays

This is an interesting demonstration of the use of multiple mice, controlled by children on an interactive whiteboard. The collaborative application uses Microsoft's Multi-Point technology. For more information and free downloads, go to MouseMischief.org.

The Internet of Surfaces? Microsoft's Pete Thompson discusses screens and surfaces of all sizes.

I came across this video of Microsoft Surface's general manager, Pete Thompson, on the GottaBeMobile website. According to Thompson, the people who worked with TouchWall discussed in the video, were also involved with Microsoft Surface.

Apparently, the Surface and Wall folks at Microsoft aren't sure of what they are doing with screens of all sizes.

(If you are interested in surface form factors, see my previous blog post, Emerging Interactive Technologies, Emerging Interactions, and Emerging Integrated Form Factors.)

There are many unanswered questions from my perspective.

Bill Gates portends that every surface will be a computer, a concept that is echoed in the video. If so, what are people doing to ensure that surface-supporting environments are universally designed?

Before Microsoft and other companies unleash "surface" technology to the masses, they must get a few things right.

Will they?

Will the researchers at Microsoft find out how various screens play out in classrooms, in the community settings, on-the job, and in-between?

For example, large interactive displays in urban and retail settings have the potential to provde people with a rich amount of information about what is around them. These displays serve little purpose if they are user-unfriendly, and no purpose at all if they are not accessible.

If developers and designers are not following basic user-centered design guidelines and usability standards now, how can we expect the nextgen systems of display surfaces to support universal usability? As our population ages, this will be more of a problem.

From what I can tell, there will be more opportunities for people to use their mobile devices to interact with larger screens and surfaces when they are out and about. For example, when I was at the airport recently, I noticed that there was a large display that offered cell-phone ringtone downloads. Microsoft was behind this display.

Interconnectivity and interoperability between devices and screens of all sizes is important to think about. If universal usability guidelines are not followed, our mobile devices will be difficult to use in the world of surfaces.

It isn't much of a leap to see the big picture. Just think about the problems we have with our remote controls and entertainment centers in our homes! We might have to carry all sorts of devices just to get from point A to point B. I'm not kidding. It will not be a pretty sight, especially if the privacy and security issues are not resolved as we move to a world that supports the internet of things.

Or shall I say, the internet of surfaces?

Creative Programming: openFrameworks - AWESOME for interactive multimedia applications!

openFrameworks: Better Tools, Enhanced Creativity, Better Projects: YES. Artists can make tools at the same time they make artwork.

To learn all about this, delve into the video. It highlights interviews with creative people who are using openFrameworks, including their innovative work.


made with openFrameworks from openFrameworks on Vimeo.

If you are working with openFrameworks, or thinking about it, let me know.


This looks like a great tool to use for projects I'm creating for my new HP TouchSmart....

.....and my multi-touch thought experiments ; }














I learned about openFrameworks from Seth Sandler, aka "cerupcat", a member of NUI-group who was chosen to participate in Google's Summer of Code. He's posted about his progress on his AudioTouch blog.

Here is a screenshot of Seth's tracking application, still under development, is the result of porting touchlib, the main tracker used by NUI-Group members, to openFrameworks:

Video modeling software for visual learners and those who have autism spectrum disorders

Activity Trainer, video modeling software from Accelerated Educational Software, supports the following activities:

• Academic
• Communication
• Daily Living
• Non-Verbal Imitation
• Recreation
• Social
• Vocational

A free 30-day trial of the software can be downloaded from the Accelerations Educational Software website.

On the Accelerations Educational Software website, you can find other products, such as Storymovies, which is the product of a collaboration between Carol Gray (social stories), Mark Shelley, and the Special Minds Foundation.

"The Next Big Thing in Humanities, Arts, and Social Science Computing: Cultural Analytics": Link

I could remain an undeclared graduate student for life!

Last semester, I took a course in information visualization and visual communication. What next?!

Cultural Analytics.

I just heard of this area of study today. Here is a link to an article, written by Kevin D. Franklin and Karen Rodriguez, in HPCwire:

The Next Big Thing in Humanities, Arts, and Social Science Computing: Cultural Analytics


"some see the reciprocal and perhaps limitless possibilities of emergent technologies and humanities scholarship -- how digital technology cuts across disciplines, creates new ways of looking at artifacts, as well as producing new forms itself."

This speaks to my multidisciplinary soul.


-photo via HPCwire