Focused on interactive multimedia and emerging technologies to enhance the lives of people as they collaborate, create, learn, work, and play.
Dec 7, 2008
Demo of Duke University's multi-touch wall at RENCI, running the Cobalt Metaverse Browser
"This video shows the Cobalt metaverse browser being tested on a 13-foot by 5-foot multi-touch visualization wall equipped with six high-definition projectors located at the Renaissance Computing Institute engagement center at Duke University. The input drivers are being developed by Dr. Xunlei Wu so that users can directly manipulate high-resolution data using both hands and multiple fingers for a more natural and intuitive data exploration experience. In the video, Dr. Wu is using both gesture and touch to navigate through, and rearrange content between, two Cobalt virtual worlds."
Related:
The Open Cobalt Project (on ning)
Cobalt Website & link to download to the latest pre-alpha build
Cobalt Community,
Cobalt Google Group
EduSim (A 3D multi-user virtual world platform and authoring toolkit for K-12 interactive whiteboards. The latest version is powered by Cobalt)
Cobalt can import objects from the Google 3D warehouse as well as Google Sketch-up:
Video Tutorial: Using Google 3D Warehouse to build Cobalt & Edusim Virtual Worlds
People:
Julian Lombardi, Duke University
Xunlei Wu, Senior Visualization Researcher, RENCI, Duke University
Rich White, EduSim; Greenbush Education Service Center, Girard, KS
Kids using Cobalt-based EduSim on desktop computers, via Rich White:
Dec 6, 2007
iSocial: A Virtual Social Space for Youth with ASD, built with Croquet
iSocial runs on Croquet, an open-source 3D SDK/tool designed to build 3D virtual collaborative environments. (EduSim applications also run on Croquet.)
iSocial Video
Nov 17, 2007
About: Programming for interactive multimedia applications- WPF, Silverlight, EduSim, NeuroVR..
I thought I'd write on a more personal level this time.
I'm a school psychologist, so in 2003, my motivation for taking computers at mid-life stemmed from my desire to create engaging interactive multimedia games for learning, games that could be played on hand-held devices as well as on the interactive whiteboards that I noticed were inching into my schools.
It was difficult for me to figure out how to get from Point A to Point B.
I shouldn't have been shocked to learn that most introductory programming classes provide instruction, as well as endless lab assignments, that are geared for people who want to make business forms and manipulate business-related data, build e-commerce websites, or create relational databases for... banks!
I now can make a mortgage calculator forms that adjust for various scenarios and provide cute error messages, in beginning Visual Basic.Net, C#, and Java. I can create a database that will let users look up part numbers for all sorts of widgets, in all sorts of combinations, and ensure that client data can be easily accessed in a nice looking form.
Why should I learn all of the old stuff when there are so many new avenues to explore?
Over the past few years, I've been fortunate to take a variety of classes that were not readily available just 8-10 years ago:
Computer/Internet Multimedia. Computer Music Technology. Game Design/Development. AI for Games. Ubiquitous Computing. Web Development Tools. Virtual Reality for Education and Training. These courses have motivated me to learn more about programming. The traditional programming courses had the opposite effect.
PART II Keeping up
I recently attended a day-long code camp at Central Piedmont Community College to learn more about Windows Presentation Foundation (WPF) and Silverlight. WPF and Silverlight provide the foundation for applications that can run on Microsoft's Surface. WPF and Silverlight provide designers and developers with more efficient ways of developing visualization and interactive multimedia applications.
The architecture behind WPF and Silverlight seems to align more with the way people think and envision, which in my opinion, gives hope for those of us who have toyed with the idea of programming, but were frightened off the first time they opened up a traditional programming textbook.
Part III Visualization and Interactive Multimedia
For more information about WPF, Silverlight, design, etc, read Sam Batterman's (a Microsoft Evangalist) recent blog post: Some thoughts about WPF and Data Visualization
Here are some pictures and text that I lifted from Sam's blog that will give you a picture of what I'm talking about:
"Here's an application that one of our partners built in a few weeks - all WPF and actually, not complicated code...getting that heart rendered was probably less than 100 lines of code. This app is used in a hospital for documenting heart surgery procedures. You can draw and annotate the 3D surface, rotate the heart, etc."
Can you imagine how kids would love to manipulate something like this in a science class?! This would be great on an interactive whiteboard or display.
This focuses on user experience, much more so than applications in the past. For more information about designing for user experience, take a look at the link to Bill Buxton's book, "Sketching User Experiences."
Bill Buxton's webiste, "Multi-touch systems that I have known and loved" is a good resource for those of you who'd like to get a better picture of multi-touch systems and interaction.
(I'll write more about WPF and Silverlight after I get my laptop repaired and have a chance to experiment some more.)
Croquet and EduSim
Right now, I've been experimenting with Croquet, which uses Squeak, and EduSim, which is powered by Croquet, to put together some learning activities for students.The best part is that Croquet is open-source, and all of the EduSim applications are free.
Here are some pictures that link to short video clips from the Greenbush EduSim website:

Neuro VR
Another application that I'm working with is NeuroVR.
NeuroVR is a free virtual environment that was designed for use in clinical settings. If you don't have access to virtual-reality hardware, you can still use NeuroVR on a desktop or large-screen display. Available 3D environments include an office, a supermarket, a park, a classroom, a poolside setting, and a home.
Andrea Gaggioli, Ph.D., is the Chief Technical Officer of the this project. He's also behind the Positive Technology blog, which is a great resource. Andrea is a researcher working at the intersections of psychology, neuroscience, and emerging technologies.
NeuroVR allows the clinician (or educator) to easily insert pictures, objects, and videoclips into each virtual world. Doors open and close, and you can move items around in the environment. For example, fruit set on a table can be moved over to a counter.
In my opinion, NeuroVR has potential not only in the area of rehabilitation and therapy, but in special education as well, particularly for students who have multiple special needs, including severe autism. A variety of "how-to" videos are posted on DaevornLi's YouTube channel.
Here are some videos clips to give you a better picture of the application:
Oct 21, 2007
Updated MegaPost-Resources For All: Interactive Multimedia and Universal Design for Learning
In this post, I've consolidated information and updated links and resources from some previous posts from my TechPsych and Interactive Multimedia Technology blogs. Although many of the resources cited in this post relate to K-12 education, some of the information is useful for instructional designers, school administrators, researchers, college/university educators, parents, and technologists who are interested in developing interactive multimedia applications for children and teens.
UNIVERSAL DESIGN FOR LEARNING (UDL) and PLANNING FOR ALL LEARNERS (PAL)
Universal Design for Learning incorporates concepts such as differentiated instruction, visual and media learning, and providing a climate of academic engagement through the use of digital media technology and strategies such as collaborative project-based learning.
The concepts behind Universal Design for Learning fit with Response to Intervention (RTI), universal prevention/intervention, and team-based instructional consultation/ problem-solving. For those who are considering this approach, plan on spending time exploring this Center for Applied Special Technology (CAST) website.
CAST offers the on-line interactive book, Teaching Every Student in the Digital Age: Universal Design for Learning. This book is a good resource for staff development activities. Plan on spending a good bit of time on this site! The Planning for All Learners section on the CAST website provides resources such as lesson plan examples and planning charts that provide teachers a way of representing the specific needs of each student.
Related:
Karen Janowski's Site: Free Technology Toolkit for Universal Design for Learning
RadTeach is the website of Dr. Judy Willis, a neurologist and middle school teacher.
"Enthusiasm is generated when children are presented with novelty and find creative ways to explore or connect with the new material and are inspired by it. Whenever you can generate this awe and sense of wonder, your children will be pulled into the school lessons they bring home and they will be motivated to connect with the information in a meaningful way." - Judy Willis
Brain-Friendly Strategies for the Inclusion Classroom published May 2007 ASCD
Research-Based Strategies to Ignite Student Learning: Insights from a Neurologist/Classroom Teacher published August 2006 ASCD
Misunderstood Minds is the companion site PBS documentary about learning differences and disabilities. The site provides information and activities that are good for self-study or professional development. Take a look at "Listening to Instructions" a simulation of what it is like to be a first grade student trying to follow directions and concentrate to complete a task, and "Attention", a video clip of Dr. Mel Levine interviewing a boy about his attention difficulties. Take a look at the Misunderstood Minds Resources and Links Page for more information.
Teaching Learners with Multiple Special Needs
This blog has extensive links and resources on a variety of topics related to special needs.
PowerPoint presentation by George Sugai's at the UDL Institute. "An introduction to the defining features of School-Wide Positive Behavior Supports within the context of Universal Design for Learning."
TECHNOLOGY INTEGRATION RESOURCES
EDUTOPIA: The George Lucas Foundation
The Edutopia website provides "..detailed articles, in-depth case studies, research summaries, instructional modules, short documentary segments, expert interviews, and links to hundreds of relevant resources." The Technology integration link. provides resources for teachers, staff development, including video segments that cover technology integration, project based learning, school-to-career, the use of multimedia for emotional intelligence activities, and more.
Bill MacKenty's "How-To" educational technology web-page
FlickSchool: Quick "how-to" videos about digital media production for teens-and teachers of teens
How To: Use Digital Storytelling in Your Classroom
Post: Visual Learning Lab Supports Effective Teaching and Learning
Bill MacKenty's Games and Learning Resources
LESSONS, ACTIVITIES, AND WEBSITES WITH INTERACTIVE CONTENT
National Gallery of Art Kids
PBS Kids
National Geographic Kids
NASA Kids
Discovery Kids
Music Tech Teacher (Includes online games and music learning activities)
Flash Music Games
(some games cam be downloaded)
Exploratorium On-line Microscope Imaging Station
Online interactive whiteboard activities for primary classrooms (from Topmarks Education, UK)
SmartTechnologies lesson activities for the SmartBoard
StarFall is website full of free interactive content designed to support early reading for children who are reading at the K-2 level. The site offers games, music, animation, and video-based activities, plus support for educators and parents. From the website:
"Our scientific, research-based reading materials and activities are modeled on the "Big Five" focus areas recommended by the National Institute of Child Health and Human Development: phonemic awareness, systematic phonics, vocabulary, fluency, and comprehension...Starfall employs the computer to develop feelings of wonderment and play, not rote assessment. The activities, songs and books complement your classroom by creating an atmosphere of fun and enthusiasm that infuses all aspects of learning. Our online and printed activities use positive reinforcement to guide children in making correct responses."
The Moleculareum Project, created by a team from the Rensselaer Polytechnic Institute:
"The Molecularium Project is committed to promote science literacy and awareness for audiences of all ages..Zoom into the molecular level, build molecules from atoms, and explore states of matter in the Nanolab of our interactive Kid’s Site. The experiments, activities and songs included in our Teacher's Resource Guide are free and fun for your home, classroom or museum."
ONLINE LEARNING OBJECTS
According to WISC, learning objects are web-based, self-contained chunks of learning, small enough to be embedded in a learning activity, lesson, unit, or course, are flexible, portable, and adaptable, and can be used in multiple learning environments and across disciplines. They are stored in on-line repositories for use and re-use to teach a variety of concepts. Merlot and WISC provide interactive learning objects geared primarily for higher education.
Learning objects incorporated into instruction provide students with multiple paths to gaining knowledge, consistent with the principles of Universal Design for Learning.
WISC Online Learning Objects
MERLOT: Multimedia Educational Resource for Learning and Online Teaching
"Learning objects are self-directive, so the student can review it over and over" "They grasp it much faster, and they retain in much better."-Instructor.
"Reading something in a book, I may not get the concept. So as I go through it on the screen, I can see how the different things work...for an example , one of the classes I had , that had a CD and a learning object with it, I actually passed the test, got a 100%, because I was able to go over it. The way it was on screen, was completely different than how I had memorized in my head."- Holly Davidson, Student"
"You can read everything in a book that you want, but as soon as you see it in a learning object, you really understand it." - Paul Heidger, Student
Links to Lynn's related blog posts:
Interactive literacy applications and on-line resources
ExploreLearning Gizmos, Interactive Displays, Revisited
(Interactive math and science)
Online Switch-Accessible Games on the BBC Website
RATIONALE FOR THE USE OF DIGITAL MEDIA IN EDUCATION
The Cambridge Handbook of Multimedia Learning, edited by Richard E. Mayer, provides a good framework to support the importance of learning and interacting with multimedia
Visual Literacy and Multimedia Literacy Quotes - Odds and Ends PART ONE
Visual and Multimedia Literacy Quotes-Part Two
Engaged Learning?
Related:
Webcast Video: Technology and Games in Education from the Orange County Educational Technology Department
INTERACTIVE WHITE-BOARDS AND LARGE TOUCH SCREEN DISPLAYS
Interactive whiteboards increase student attention, engagement, participation, and test scores!
(This post contains links to lessons and resources for interactive white-boards and displays.)
Excerpt from the above post:
According to a press release from Smart Technologies, a study conducted in the UK and Europe during the years 2002-2006 found positive results regarding the use of interactive whiteboards in the classroom:
"The following outcomes were among its key findings: interactive whiteboard use results in improved student performance in national tests in English, math and science, compared to student performance without interactive whiteboards; digital content on interactive whiteboards is engaging, motivating, and students pay more attention during lessons; and interactive whiteboard use encourages greater student participation in the classroom."
There are several types of interactive large-screen displays available for use in educational settings, and there are several companies working on large touch-screen displays that could be adapted for use in schools, media centers, and public libraries.
The following is a partial list of companies that make or sell interactive displays and/or whiteboards:
SmartTechnologies
NextWindow
Accenture
NEC
Hitachi Starboard
Promethean
Mimeo
Interwrite
Resources for Interactive Whiteboards and Displays
Online interactive whiteboard activities for primary classrooms (from Topmarks Education, UK)
SmartTechnologies lesson activities for the SmartBoard
Link to Post: EduSim and Croquet: 3D Collaboration Applications
EduSim 3D
Greenbush EduSim
Panorama Archive: Full-screen panoramas from around the world.
PhotoSynth
"Our software takes a large collection of photos of a place or an object, analyzes them for similarities, and displays them in a reconstructed three-dimensional space.
With Photosynth you can:
- Walk or fly through a scene to see photos from any angle.
- Seamlessly zoom in or out of a photo whether it's megapixels or gigapixels in size.
- See where pictures were taken in relation to one another.
- Find similar photos to the one you're currently viewing.
- Send a collection - or a particular view of one - to a friend."
Virtual Earth is like Google Earth. It works with PhotoSynth.
GoogleEarth
"Tilt and rotate the view to see 3D terrain and buildings, or look up to explore the sky"
P3D -The P3D website has good demonstrations of 3D content.
"P3D is a software company focused on virtual reality in education and offers a constantly updated stat-of-the-art technology as an outstanding tool in a classroom environment. The products developed by P3D enable higher interactivity, enhancing student's learning capacity through tri-dimensional images and virtual reality."
Related Information:
Education World article "Speaking of Electronic Whiteboards?"
Large Display Research Overview from Microsoft Research (PDF)
Also see the previous section about lessons, activities, and websites with interactive content.
INTERACTIVE EDUCATIONAL SOFTWARE
INSPIRATION and KIDSPIRATION
"Learning to think. Learning to learn. These are the essential skills for student success. Research in both educational theory and cognitive psychology tells us that visual learning is among the very best methods for teaching students of all ages how to think, and how to learn."
Many school districts have adopted Kidspiration and Inspiration; there are many resources on the website for teacher support for this software. Inspiration and Kidspiration are effective with visual learners. These applications work well on large-screen displays as well as P.C.'s, and hand-held devices. Activities using Inspiration and Kidspiration are good for paired and small group activities. Inspiration now offers a visual data analysis application, InspireData.
I SUPPORT LEARNING
“Our mission is to empower educators in their quest to create and support life-long learners, to make education relevant and engaging for the student through creative software.” I Support Learning provides applications for creative, interactive project-based activities for middle and high school students that integrate the use of technology and build technology skills."
I Support Learning's Personal Experience Curricula:
CARTOON ANIMATION VIDEO
GAME DESIGN
MUSIC VIDEO PRODUCTION
WEB GAME DESIGN
PERSONAL FINANCE AND WEALTH MANAGEMENT
BUILDING GREEN – RESIDENTIAL HOME DESIGN
ARTIFICIAL INTELLIGENCE AND MOBILE ROBOTICS
GREEN INDUSTRY – LANDSCAPE DESIGN
ROBOTS AND INVENTION
HORTICULTURE AND LANDSCAPE DESIGN
Immersive Education
Immersive Education software educational software company. According to information on the website, the software applications..
- encourage creative learning
- adapt to individual teaching styles
- allow for individual learning preferences
- inspire innovation and creativity in the classroom.
"We believe pupils learn best when education is fun - our products are enjoyable to use and encourage learning through exploration."
Links to information about Immersive Education's applications:
- kar2ouche - storyboarding and role-playing software.
- mediastage - 3D virtual production studio.
- krucible - science simulation software.
- missionmaker - game-authoring tool.
Tabula Digita
DimensionM, an interactive multi-player game for algebra.
HAND-HELD AND MOBILE DEVICES FOR LEARNING
There are some classrooms in that do not use textbooks. Textbooks are provided to students electronically on their PDA's. Some versions of electronic textbooks are interactive and include hypertext links to supplemental information or activities. Students who use Wi-Fi enabled PDA's have access to interactive websites that compliment what they are learning in class.
In most classrooms that use this form for instruction, students often work in pairs and groups. Because PDA's run many of the same applications as do PC's, they can be utilized for frequent monitoring and charting of progress. Here are a few resources:
"Our classroom uses Windows CE and Pocket PC based Personal Digital Assistants (we call them PDA's.) We are running a paperless classroom with the PDA's. All homework and reading assignments are done on the PDA's. We do not use copied ditto sheets or heavy textbooks. All of this is accomplished in a 7th and 8th Grade Language Arts class, not in a math or science classroom."This is a good example of how a teacher, a "rookie" during the 1998-99 school year, embraced new and effective ways of using technology to meet the needs of a diverse range of learners in the middle school setting for Language Arts instruction. This website has expanded to include a range of resources for educators interested in learning how to implement paperless classrooms. Research in this area can be found through HiCe and related organizations.
HICE: Center for Highly Interactive Computing in Education
HiCe provides consultation and resources for schools regarding the use of handheld devices in learning environments. The work of the HiCe project at the University of Michigan has been in place for over a decade.."With PDA's, students can access websites created by their teachers to keep track of assignments and upload or e-mail assignments when completed. Software on PDA's such as calendars and an alarm can help students who have organization problems."
A spin-off of the HiCe project is GOKNOW.
CREATE-A-SCAPE:
A create-a-scape, otherwise known as a mediascape, is..." composed of sounds, images and video placed outside in your local area. To see the images and video, and hear the sounds you need a handheld computer (PDA) and a pair of headphones. An optional GPS unit can automatically trigger the images, video and sounds in the right places."
"To create a mediascape, you start with a digital map of your local area. Using special, free software, you can attach digital sounds, pictures and video to places that you choose on the map.By going outside into the area the map covers, you can experience the mediascape. Using the handheld computer and headphones, you can hear the sounds and see the pictures and video in the places the author of the mediascape has put them. All sorts of exciting things can happen as you explore the mediascape."
Quick Demo: How to Make a MediaScape
Create-a-Scape Teacher Resource Site
MOOP:
Moop was developed in
Mattila, P. & Fordell, T. (2005) MOOP- Using m-learning environment in primary schools. http://www.mlearn.org.za/CD/papers/Mattila.pdf
Mattila, P. (2005) Moop - Mobile Learning Environment as Part of Daily School Work
http://www.microlearning.org/micropresentations/micropresentation_friesen_2005.pdf
Quote from the project website:
"…the user group has expanded to encompass people of all ages, from grandparents getting involved in family learning, adults looking to improve their employment prospects, to pregnant teenagers needing health advice and guidance."
OTHER RESOURCES FOR INTEGRATING HAND-HELDS INTO EDUCATION:
Mega-List of Resources and References: Technology, Psychology, Intervention & Prevention
Confronting the Challenges of Participatory Culture: Media Education for the 21st Century", a 72-page white paper by Henry Jenkins, from MIT.
FutureLab's 2006 Teaching With Games Report
John Kirriemuir's blog about games and learning
Oct 9, 2007
3D Collaborative Applications for Interactive Displays and Whiteboards: Croquet, EduSim
According to Julian Lombardi, Duke University's Assistant Vice President of Academic Services and Technology Support and Senior Research Scholar with the Information Science + Information Studies, "Croquet is a powerful new open source software development environment for creating and deploying deeply collaborative multi-user online applications on multiple operating systems and devices. Derived from Squeak, it features a peer-based network architecture that supports communication, collaboration, resource sharing, and synchronous computation between multiple users on multiple devices. Using Croquet, software developers can create and link powerful and highly collaborative cross-platform multi-user 2D and 3D applications and simulations - making possible the distributed deployment of very large scale, richly featured and interlinked virtual environments."
EduSim "How To" Videoclips
EduSim Videos
Croquet Screenshots
Related Information
The Cambridge Handbook of Multimedia Learning, edited by Richard E. Mayer, provides a good framework to support the importance of learning and interacting with multimedia.
According to information from the book, "Multimedia learning is defined as learning from words (e.g., spoken or printed text) and pictures (e.g., illustrations, photos, maps, graphs, animation, or video)... Multimedia environments included on-line instructional presentations, interactive lessons, e-courses, simulation games, virtual reality, and computer-supported in-class presentations...The Cambridge Handbook of Multimedia Learning seeks to establish what works (i.e., to determine which features of a multimedia lesson affect learning), to explain how it works (i.e., to ground research in cognitive theory), and to consider when and where it works (i.e., to explore the implications of research for practice)".