How soon will we see interactive information visualization for multi-touch & gesture systems?

The field of information visualization is growing. Until recently, most visualizations were created for use on a single PC or larger screen, allowing for interaction by only one user at a time. I have a feeling that this will be changing in the very near future.

Why? Interactive duo and multi-touch interfaces are becoming more common, and now come in all sizes of screens, from the iPhone, the Surface, and CNN's multi-touch "Magic Wall". People of all ages play interactive games on the Wii, often on large flat-panel displays. In my opinion, the time is right for those developing applications for the InfoViz world to think about harnessing the power of multi-touch.

Below is a picture of the front page of the Visual Complexity website. If you go to the site, you can select a visualization, and then explore it more in detail, as each picture links to a web-page that provides background information about the visualization project, the artist or team behind the project, and links to the project's website.

I took a look at a variety of the examples posted on the Information Complexity website, and think many would be enhanced by a transformation to a multi-touch, gesture, bi-manual, or duo/multi user system. I'm interested in learning what others think about this. If you are working on a collaborative information visualization project, feel free to add a comment and post a link.

Here is a a nice quote from the website:

"Functional visualizations are more than innovative statistical analyses and computational algorithms. They must make sense to the user and require a visual language system that uses colour, shape, line, hierarchy and composition to communicate clearly and appropriately, much like the alphabetic and character-based languages used worldwide between humans." Matt Woolman Digital Information Graphics

Update: I did a search for "multi-touch" on the Visual Complexity website and found a couple of interactive applications:

Reactable (I've posted about this system a few times!)
(Reactable website)

Prototouch

(Wirmachenbunt Website)

Visual Culture

What is Visual Culture? It is a fairly new discipline of study that supports the point of view that understanding our world goes beyond words and text.

I'll be posting a bit on this topic, since I'm interested in figuring out how the concept of visual culture fits into my own theories about multimedia literacy and technology.

As I dig deeper into this field of study, I hope to gather and share what I find. One of my tasks will be to find a "short list" of visual culture scholars and see if there are is any cross-pollination between this group and people in the field of information/data visualization.

In searching for definitions of Visual Culture, I came across this description from the Visual Culture website from the University of Wisconsin-Madison.

What is Visual Culture?

"(It) is concerned with everything we see, have seen, or may visualize-paintings, sculptures, movies, television, photographs, furniture, utensils, gardens, dance, buildings, artifacts, landscape, toys, advertising, jewelry, apparel, light, graphs, maps, websites, dreams-in short, all aspects of culture that communicate through visual means...."

Visuality and Textuality

"Visual Culture Studies presents a challenge to the textual model of the world that dominates so much thinking about culture. While analyzing cultural experiences, artistic practices, and even social interactions and history itself as so many texts to be read is an undeniably powerful approach, this textual bias ignores a vast realm of perception, experience and meaning in reducing culture to "reading." Visual Cultural Studies counters this bias by emphasizing the importance of the visual-even in textual objects, as when scholars consider the impact of design on the meaning of literature. In countering this earlier thinking, Visual Culture Studies does not set aside textuality, but draws it into dialogue with visuality, performativity, aurality, tactility, and so on. At the same time, Visual Culture scholars are beginning to question the broader dominance of visuality itself (what some have termed "ocularcentrism") and to look for ways to consider vision in the broadest cultural context without introducing new hierarchies of social practice."

What is Visual Culture? (University of Aberdeen)

"Visual culture has emerged as a constellation of various interests and numerous critical intersections in the visual and performing arts here at the University of Aberdeen. Because of its dynamic nature it constitutes less of discipline than a platform for exploring theoretical and creative collaborations between Film, Art History, Modern Thought, Museum Studies, Visual Anthropology, Scientific Imaging practices, new media, and computer engineering. The taught M.Litt reflects the diverse interests of faculty and students participating in visual studies. The M.Litt in Visual Culture is divided into two tracks: one designed to engage with critical thinking (the history of art and film criticism, theories of the visual, philosophies of perception and new media); and the other concentrating on creativity and media production (the internationally acclaimed filmmaker Raul Ruiz advises on avant-garde filmmaking, and courses are taught in film production)."

RELATED
(This list is not inclusive and will be updated.)

Visual Culture Website (University of Wisconsin, Madison)

Visual Culture Website (University of Aberdeen)

Georgtown University's Visual Culture Page (Martin Irvine)
Warning- it is ALL TEXT!

Aesthetics and Visual Culture (Bruce B. Janz, Director, Center for Humanities and Digital Research, University of Central Florida)
This site has an extensive number of links that cross numerous disciplines, last updated in 2006.

Journal of Visual Culture

Visual Culture at Middlesex University People Page

Visual Culture Blog (Oberholtzer-Creative)

The Visual Culture Awards - Info about awarded individuals

Infosthetics

Capella Web Capella Garcia Arquitectura- see picture below:




Via Visual Culture:

Pathway to a Sustainable Environment
"Heaven by Capella Garcia Arquitectura is an ephemeral space whose purpose is to show off the virtues of a constructional system using a plastic material, developed by the Resyrok Company, at the latest Casa Decor Fair in Barcelona, Spain, whose theme was “Pathway to a sustainable environment"

I came across the following video, created by Eirik Solheim, on the Visual Culture blog:

2008 in 40 Seconds

One year in 40 seconds from Eirik Solheim on Vimeo.

Data and Information Visualization Resources for Economics, Finance, and the Current Crisis

I posted a variety of links to data and information visualization resources that might be helpful to those of you who are interested in exploring our current economic crisis:

Economic Sights and Sounds: Links to Data and Information Visualization resources for Economics, Finance, and the Current Crisis

I'm using the Economic Sights and Sounds blog as a repository of information and links for use in a collaborative interactive multimedia time-line of events surrounding the current situation.

Visualization software used to create Radioheads "House of Cards"- no cameras used.

Take a look at my recent Technology Supported Human-World Interaction post, "We were rolling computers all day: The making of Radiohead's House of Cards Video". The music video was made without the use of cameras, and relied on visualization software and scanner/sensors. Follow the links from the post for additional information. ( I came across the video on the Kinetic Interface blog)

Two Links: Urban Screens, Urban Interfaces, Digital Media, and the Arts in Social-Public Spaces; Edward Tufte is a Smart Guy

Here are two links that I think you'll find interesting:

My post on the TSHWI blog, reflecting on interdisciplinary research and topics related to ubiquitous computing and large interactive displays in public spaces:

Urban Screens, Urban Interfaces, Digital Media, and the Arts in Social-Public Spaces


Bill Mackenty's reflections about workshop he attended that was led by Edward Tufte, known for his insights about the presentation of information graphics:

Edward Tufte is a Smart Guy

MyMap email visualization, related link

I came across a link to the MyMap project via Andrea Gaggioli's Positive Technology Journal, reblogged from Information Aesthetics:

"Christopher Baker is an artist whose work engages the rich collection of social, technological and ideological networks present in the urban landscape. Baker creates artifacts and situations that reveal and generate relationships within and between these networks."

Here is a link to his MyMap project website. MyMap is an e-mail visualization. It won "Best in Show" during the Spring 2007 University of Minnesoa College of Design Student Exhibition.

Below are pictures from the site:



Description from Christopher Baker's website:

"I created My Map, a relational map and alternative self portrait. My Map is a piece of custom designed software capable of rendering the relationships between myself and individuals in my address book by examining the TO:, FROM:, and CC: fields of every email in my email archive. The intensity of the relationship is determined by the intensity of the line. My Map allows me to explore different relational groupings and periods of time, revealing the temporal ebbs and flows in various relationships. In this way, My Map is a veritable self-portrait, a reflection of my associations and a way to locate myself."

Looking at MyMap, I could see the potential for use of this program on an interactive whiteboard or display. It would be good if optimised for a mobile device, too!

FYI
Visit Andrea's blog for more posts about information visualization.

Link to a great post about serious games and virtual reality in business

Elaine Alhadeff's Future-Making Serious Games blog has a wealth of interesting information for people interested in how interactive multimedia technology can impact the present- and the future. Take the time to visit her blog and take a look!

Here is a slice from one of her latest posts:

Via: daytaOhio - Barco and daytaOhio Equipped Collaborative Visualization Complex at Wright State’s Joshi Research Center
"Barco, a leading global provider of visual display systems, has partnered with daytaOhio to implement the industry’s most advanced virtual collaboration and visualization complex at Wright State University’s Joshi Research Center. "

It is exciting to learn about the growth in collaborative visualization! The businessmen below seem to be enjoying the experience:

For more interesting ideas, look at Eliane's archived posts about serious games.

Interactive information visualization: Digg labs website

I thought I'd share a view examples of information visualization as it relates to on-line news. Stories are grouped in various configurations over time. Click on the images below to see each example in real-time:

NEWSMAP

Click on the image to link to Newsmap.

" Newsmap is an application that visually reflects the constantly changing landscape of the Google News news aggregator. A treemap visualization algorithm helps display the enormous amount of information gathered by the aggregator. Treemaps are traditionally space-constrained visualizations of information. Newsmap's objective takes that goal a step further and provides a tool to divide information into quickly recognizable bands which, when presented together, reveal underlying patterns in news reporting across cultures and within news segments in constant change around the globe."

DIGG

Click an image to see digg visualizations in action!
STACK
"Diggers fall from above and stack up on active stories." If you click on a story, you'll get a pop-up that provides more details about the story, including information over time.


SWARM

Digger swarm around different categories of stories and make them grow. You can look at popular stories, newly submitted stories, or all activity. Click on a circle, and you'll get more information, and the option to keep or "kill" the story.

BIGSPY

"Bigspy: Active stories appear at the top when people digg them. Bigger stories have more diggs..." If you click on a story, you'll link to more information about it from the digg website.


ARC

"Stories arrange themselves as users digg them. Larger stories have more diggs." If you click on a story, you'll link to more information about it from the digg wesbsite.

Check back for more examples- and leave a comment if you have others to share.

Related Link:

Article from PBS.org's Idea Lab, by Rich Gordon, about information visualization and journalist-programmers. "Idea Lab is a group blog by innovators who are reinventing community news for the digital age..."

About: Programming for interactive multimedia applications- WPF, Silverlight, EduSim, NeuroVR..

Part I Musings: Learning about application development and programming at mid-life:

I thought I'd write on a more personal level this time.

I'm a school psychologist, so in 2003, my motivation for taking computers at mid-life stemmed from my desire to create engaging interactive multimedia games for learning, games that could be played on hand-held devices as well as on the interactive whiteboards that I noticed were inching into my schools.

It was difficult for me to figure out how to get from Point A to Point B.

I shouldn't have been shocked to learn that most introductory programming classes provide instruction, as well as endless lab assignments, that are geared for people who want to make business forms and manipulate business-related data, build e-commerce websites, or create relational databases for... banks!

I now can make a mortgage calculator forms that adjust for various scenarios and provide cute error messages, in beginning Visual Basic.Net, C#, and Java. I can create a database that will let users look up part numbers for all sorts of widgets, in all sorts of combinations, and ensure that client data can be easily accessed in a nice looking form.

Why should I learn all of the old stuff when there are so many new avenues to explore?

Over the past few years, I've been fortunate to take a variety of classes that were not readily available just 8-10 years ago:

Computer/Internet Multimedia. Computer Music Technology. Game Design/Development. AI for Games. Ubiquitous Computing. Web Development Tools. Virtual Reality for Education and Training. These courses have motivated me to learn more about programming. The traditional programming courses had the opposite effect.


PART II Keeping up

I recently attended a day-long code camp at Central Piedmont Community College to learn more about Windows Presentation Foundation (WPF) and Silverlight. WPF and Silverlight provide the foundation for applications that can run on Microsoft's Surface. WPF and Silverlight provide designers and developers with more efficient ways of developing visualization and interactive multimedia applications.

The architecture behind WPF and Silverlight seems to align more with the way people think and envision, which in my opinion, gives hope for those of us who have toyed with the idea of programming, but were frightened off the first time they opened up a traditional programming textbook.


Part III Visualization and Interactive Multimedia

For more information about WPF, Silverlight, design, etc, read Sam Batterman's (a Microsoft Evangalist) recent blog post: Some thoughts about WPF and Data Visualization

Here are some pictures and text that I lifted from Sam's blog that will give you a picture of what I'm talking about:

"Here's an application that one of our partners built in a few weeks - all WPF and actually, not complicated code...getting that heart rendered was probably less than 100 lines of code. This app is used in a hospital for documenting heart surgery procedures. You can draw and annotate the 3D surface, rotate the heart, etc."





Can you imagine how kids would love to manipulate something like this in a science class?! This would be great on an interactive whiteboard or display.

This focuses on user experience, much more so than applications in the past. For more information about designing for user experience, take a look at the link to Bill Buxton's book, "Sketching User Experiences."

Bill Buxton's webiste, "Multi-touch systems that I have known and loved" is a good resource for those of you who'd like to get a better picture of multi-touch systems and interaction.

(I'll write more about WPF and Silverlight after I get my laptop repaired and have a chance to experiment some more.)

Croquet and EduSim

Right now, I've been experimenting with Croquet, which uses Squeak, and EduSim, which is powered by Croquet, to put together some learning activities for students.The best part is that Croquet is open-source, and all of the EduSim applications are free.

Here are some pictures that link to short video clips from the Greenbush EduSim website:






Neuro VR

Another application that I'm working with is NeuroVR.

NeuroVR is a free virtual environment that was designed for use in clinical settings. If you don't have access to virtual-reality hardware, you can still use NeuroVR on a desktop or large-screen display. Available 3D environments include an office, a supermarket, a park, a classroom, a poolside setting, and a home.

Andrea Gaggioli, Ph.D., is the Chief Technical Officer of the this project. He's also behind the Positive Technology blog, which is a great resource. Andrea is a researcher working at the intersections of psychology, neuroscience, and emerging technologies.

NeuroVR allows the clinician (or educator) to easily insert pictures, objects, and videoclips into each virtual world. Doors open and close, and you can move items around in the environment. For example, fruit set on a table can be moved over to a counter.

In my opinion, NeuroVR has potential not only in the area of rehabilitation and therapy, but in special education as well, particularly for students who have multiple special needs, including severe autism. A variety of "how-to" videos are posted on DaevornLi's YouTube channel.


Here are some videos clips to give you a better picture of the application:

Microsoft UK: "ThinSight" Multi-touch Technology on a Laptop

(Photos from New Scientist)

Tom Simonite's article in a recentNewScientist article reviews a prototype for a touch-screen lap-top using "ThinSight", developed by Microsoft Research. Other companies are researching similar systems.

Multi-touch screens and surfaces support the work of people who increasingly rely on visualization or interactive multimedia to do their jobs. In addition, 3D multi-touch applications created for laptops and tablet PC's might be good for education, training simulations, and serious games.

Visualizing Content on the Web: Great pictures, descriptions, and links on Max Kiesler's blog

"Content Visualizations: The Next Wave of Interaction Design" is the topic of Max Kiesler's recent blog post.

Searches conducted on the Web are primarily word-based, and the same is true for searches conducted on PC's. What results is a list of phrases, which are often difficult to sort and prioritize quickly.

The increased bandwidth available on the web now allows for content that contains a range of multimedia components, such as photos and video clips. Many on-line journal articles, blog posts, and of course, information visualization websites, contain important visual-oriented information that might be missed through traditional searches.

Kiesler discusses some solutions to this problem in his post. Take a look at it soon- it contains great visualizations, as well as links to additional information and resources.

Although Keisler does not state it directly, his post makes the case for the importance visual and multimedia literacy in our society.

Note: For those of you who follow my blog, you'll know that this sort of visual approach would be a great tool in education, especially when presented on interactive large-screen displays.