I started this blog in 2006-- and I do plan to get into the routine of blogging more more frequently. I'd love to update the posts that were the most popular in the past, and add links and resources that reflect what is going on with interactive multimedia technology in 2016. So much has evolved since I started this blog! 2006: Virtual Reality & Convergence with Game Technology
Note: Some of the links are broken.
FYI:
I still write blog posts from time to time.
Here are links to the blog post I've written for NUITEQ® about collaborative interactive technologies:
This delightful, uplifting music video from ThePianoGuys playing a rendition of Over the Rainbow and Simple Gifts showed up in my FB feed today. Enjoy.
COMMENT I use uplifting/calming videos in my work with students with complex special needs, including those with autism. Although I create some of the multimedia content I use with students, my time for creativity is limited. I appreciate coming across music videos that deliver much more than I could ever create within the confines of my day-to-day life!
The students I worked with today at Wolfe School really liked the following video, a cover of Coldplay's "Paradise". The video featured guest artist Alex Boye, The piano and musicians were perched on the top of a 1000 ft cliff. They also were fascinated with the "making of" video below.
SOMEWHAT RELATED I wonder if ThePianoGuys could create a music video at the cave setting at this beach in St. Maarten! (My husband did not want to venture very far from this beautiful refuge during a recent vacation.)
A quick search on YouTube reveals thousands of videos created to uplift/relax the spirit. If you need to take a few moments to relax - here is a time-lapse video set to calming music, from LoungeV Studio:
I recently learned about OpenPilot, an open-source project that promotes the development of economical unmanned aerial vehicles, or UAVs. According to information on the website, "OpenPilot is an ideal platform for researchers and hobbyists working on aerial robotics or other mobile vehicular platforms where stabilization and guidance are required. OpenPilot brings the cost down to reasonable prices so people can focus on developing and refining applications rather than paying the extremely high prices of most commercial offerings, or having to do ‘from the ground up’ hardware development." A number of OpenPilot community members have used their UAVs to explore interesting landscapes and at the same time, create engaging video clips. Wouldn't it be fantastic to figure out how to get a 3D or 360 camera in a UAV? Below is an assortment of videos I came across while visiting the OpenPilot website. (I've also included some videos that were created using YellowBird 360 technology, which to my knowledge, has not been attempted with a UAV.)
I think that the UAV concept would be great for an after-school technology club. It similar to robotics, but it also would get the kids outdoors. It would provide a great experience for students who are also interested in photography and videography. RELATED OpenPilot Website OpenPilot Wiki YellowBird
"GML is an extensible markup language used to define gestures that describe interactive object behavior and the relationships between objects in an application. Gesture Markup Language has been designed to enhance the development of multiuser multi-touch and other HCI device driven applications." -Gesture ML Wiki
Photo credit: Ideum RELATED Ideum Blog OpenExhibitsFree multitouch and multiuser software initiative for museums, education, nonprofits, and students GestureWorks Multi-touch authoring for Windows 8 & Windows 7
"Right here, in your web browser, you may contribute to anyone's creation and steer it in any direction you choose, or start a story of your own, by creating a seed animation for a new tree. Our hope is through the collective creativity of the web, an ever-expanding forest of narrative paths will grow." -Chris Milk/Aaron Koblin This Exquisite Forest is a collaborative art project by Chris Milk and Aaron Koblin, creators of the online Johnny Cash collaborative project. The Johnny Cash Project was a crowd-sourced music video for Johnny Cash's final album (see below). What I like about this project is that it provides viewers with the opportunity to create animations that link with those of other people, online. All of the tools to create animations and music are on the website, along with clear instructions and FAQs. You can even curate your own tree. For those who take a less participatory approach to art, the project's website offers something called "The Endless Theater", where viewers can watch the most popular branches of This Exquisite Forest.
The project makes use of Google's App Engine. Google's Cloud Storage handles the accounts and data storage. To access the project, you need Google's Chrome browser, as the project takes advantage of JavaScript and HTML5. RELATED
Richard Mayer has devoted his career to the study of multimedia learning. He is a professor in the Department of Psychological & Brain Sciences at UC Santa Barbara, and the author of Multimedia Learning, 2nd Edition. Although the book was published in 2009, years ago, it is a must-read for anyone interested in this topic.
With the popularity of interactive whiteboards and tablets/iPads in education, it is important for educators, designers and developers to become familiar with the basic principles of multimedia learning. It is also important subject for researchers.
Step into a school and you might notice that older desktop computers have been replaced by an assortment of portable devices. In many cases, students still carry around book bags that contain an assortment of textbooks, binders, worksheets, along with something digital.
So what is that "something digital"? In some cases, it might be a school-issued laptop or net-book. In other cases, it could be an iPad, an e-reader, or another sort of tablet. In other cases, it could be whatever the student brings to school, as part of a "BYOD" (Bring Your Own Device) experiment. No one has yet figured all out, but many people, from a number of disciplines, are trying!
Stantum is an example of a company that has an interest in the use of innovative technology for education. Stantum develops multi-touch technology for mobile devices, including tablets, and follows trends in education closely. I've had the opportunity speak with Guillaume Largillier, Stantum's co-founder and CSO on several occasions, most recently in late October. I'm happy to say that Stantum looks like it will be making some important inroads in education. Why? Stantum is aware that as educational institutions, K-12 and above, leave textbooks behind, there will be an enormous need for devices that will seamlessly support teaching and learning, all around the world. Guillaume spoke of developments in countries such as South Korea, France, Turkey, Thailand, Columbia, Argentina, and China to adopt digital textbooks. Some of these countries have initiated pilot programs using some sort of tablet or mobile devices, or plan to do so in the very near future.
During our conversation, Guilluame pointed out that Stantum is involved in developing durable components and tablets made from materials that can withstand the knocking about that sometimes happens when handled by young hands. Unlike the iPad, the tablets will not need a rugged, more expensive case, so it is likely that the cost to schools will be lower. Since the tablets are open to most operating systems, they are likely to be easier for school IT administrators to deploy, update, and maintain.
A bit about the technology: Stantum's Interpolated Voltage Sensing Matrix (IVSM) technology supports 10 simultaneous touches and can handle touch and stylus input at the same time. It "knows" if it has been touched by a finger and can disregard a palm. This is a good feature to have if a tablet is used young people. Earlier this year, Stantum announced its partnership with NISSHA, a company based in Japan, and unveiled its newest development, Fine Touch Z Technology, powered by IVSM. It can support Windows and Android operating systems. It has a fast scanning engine, high-resolution handwriting input, and does not produce ghost or masking effects. (See video below). One appealing feature of this technology is that it provides has low power consumption. This is a plus when considering the need students to have a device that has an extended battery life.
RELATED A leap towards interactive digital "textbooks":
Although students have been accessing educational content in digital form on computers and through educational television programming for a long time, textbooks and paper-based assignments have been major tools used in school to transmit the curriculum. The tools are changing as we move to a digital, multimedia world of communication, collaboration, knowledge sharing, teaching, learning, and creating. There are many questions to consider.
What sort of digital content will replace traditional textbooks?
Who will create this content?
Will traditional textbook publishers simply transform textbooks into slightly more interactive versions of the ho-hum content students love to hate?
How will digital interaction change the way everyone learns? How will this be measured?
How will teachers and students be provided with opportunities to create new ways of sharing knowledge?
How will usability and accessibility concerns be addressed, for students, teachers - and family members, given that digital content will be accessed both in and outside of school?
What sort of software systems will need to be developed?
What sort of infrastructure will be needed to support this influx of online activity and transfer of large data files - at schools, in homes, and in communities?
How will the technological ecology support learning, given multiple devices, interactive whiteboards and other interactive surfaces?
With change, there is usually confusion as old ways are set aside and people explore new options. Apple is making some inroads with the iPad, providing educators (and students) to create interactive books with the free iBooks Author program. Some schools have 1-1 laptop programs in place, and provide access to educational content through more traditional course management systems or web-based activities that accompany existing textbooks. Schools are signing up for resources such as Google in Education and Microsoft Partners in Learning.
If you are looking for some resources about the rapid increase in interest and adoption of interactive digital textbooks, not just in the U.S., but around the world, take a look at my recent blog post, "Got Interactive (Multimedia) Textbooks Inside Your iPad or Tablet? Lots of Resources!". In the post, I provide a wealth of links to information from the LEAD commission, the FCC's Digital Textbook Playbook, and more. You might also want to take a look at Audrey Watters' article, The Truth About Tablets: Educators are getting iPads and e-readers into students'hands--but it's not easy, which was published online in the School Library Journal earlier this year. For a global perspective, read Alex Wukman's article, World Bank Begins Global Digital Textbook Initiative It wouldn't hurt to learn more about what the textbook publishers have in mind. For that reason, I've put together a sample of resources about interactive digital content from major educational media and textbook publishing organizations: DISCOVERY EDUCATION Techbooks In the following video, students and teachers from an elementary school in Winston-Salem give their reviews of the new Discovery Education Science Techbook. Students can access the Discovery Techbook via the web. In this video, students interact with the content using traditional desktop computers as well as interactive whiteboards. Techbooks are designed to work on tablets or laptops, too. At :39, a first grade teacher explains exactly why she likes the science techbook: "...everything is right here at your hands, it is interesting to the kids, they love to watch the videos, and you can take it further - beyond than that, and everything is done for you, it is just really helpful. It even has the prep, the content review, and it is all aligned to the curriculum." -Laurie Moran
PEARSON Pearson's new interactive textbook for the iPad "It is really going to revolutionize how we think about the classroom experience and what happens in the classroom in the way of learning, and what happens outside the classroom." -Joseph Levine, Author, Miller & Levine Biology
If you live in the US, and haven't yet heard, the U.S. government is encouraging K-12 schools to adopt interactive digital textbooks within the next five years. Are we ready? I don't think so. There is a battle of the tablets going on as I type this post. Traditional, "old-school" textbook publishers appear to be key players in this game, but I am not sure if they have the know-how to create interactive content that is fresh, "touchable", engaging, and meaningful to young people. A boring textbook, tweaked tweaked for a bit of interactivity and multimedia content will not suffice. Feel free to take your time and visit the links I've shared so far. I'm still reflecting on what I've learned so far and will share more of my thoughts in future related posts. Leaders Discuss Transition to Digital Textbooks The LEAD commission
The RSA is the Royal Society for the encouragement of Arts, Manufactures, and Commerce, a charity organization founded in the 1800's, and located in the UK: "At the heart of the RSA's contemporary mission and public debates about the future prospects for the human race is the question: 'Can we go on like this?' Will the ideas and values which transformed our world in the last two centuries be sufficient to find solutions to the challenges we now face or do we need new ways of thinking?" One of the ways the RSA shares information is through events that feature renown speakers. Some of the presentations are transformed into animations, known as RSA Animates, and are some of the most popular videos uploaded by RSA to YouTube, spreading the word to hundreds of thousands across the globe. Here are a few: RSA Animate: The (Honest) Truth About Dishonesty, Dan Ariely RSA Animate: The Power of Networks, Manuel Lima
Manuel Lima is the senior UX design lead at Microsoft Bing.
For more information, see The Power of Networks: Knowledge in an age of infinite interconnections. and the presentation slides.
One of the things I like to do is share updates about the world of multimedia, multi-touch, gesture, screen, surface, and interactive technologies, focusing on off-the-desktop applications and systems. When I started this blog, I had to put forth quite a bit of effort just to FIND interesting things to blog about.
These days, there are so many sources that focus on emerging - and now commonplace- interactive technologies, my main challenge is to filter the noise. Where do I begin?
My archives are vast. I randomly picked the year 2009 and came across one of my previous posts, "News, Videos, and Links about Multitouch and Screen Technologies." The post is long, and contains a number of videos and links that probably will be of value to a future curator of the history of technology.
I welcome comments from readers who might be able to help me update information about various applications and systems I've featured on this blog in the past.
The pictures are screenshots from the results of an image search for "interactivemultimediatechnology". Over the past 6 years, I've posted quite a few!
It just might be the right time for everyone to brush up on 21st century tech skills. iPads and touch-phones are ubiquitous. Touch-enabled interactive whiteboards and displays are in schools and boardrooms. With Microsoft's Windows 8 and the news that the company recently acquired Jeff Han's company, Perspective Pixel, I think that there will be good support - and more opportunities- for designers and developers interested in moving from GUI to NUI.
In the video below, from CES 2012, Jeff Han provides a good overview of where things are moving in the future. We are in a post-WIMP world and there is a lot of catching up to do!
CES 2012 Perceptive Pixel and the Future of Multitouch (IEEE Spectrum YouTube Channel)
During the video clip, Jeff explains how far things have come during the past few years:
"Five and
1/2 years ago I had to explain to everybody what multi-touch was and meant. And
then, frankly, we've seen some great products from folks like Apple, and really
have executed so brilliantly, that everyone really sees what a good
implementation can be, and have come to expect it. I also think though, that the explosion of
NUI is less about just multi-touch, but an awareness that finally
people have that you don't have to use a keyboard and mouse, you can demand
something else beside that. People are
now willing to say, "Oh, this is something I can try, you know, touch is
something I can try as my friendlier interface"."
Who wouldn't want to interact with a friendlier interface? Steve Ballmer doesn't curb his enthusiasm about Windows 8 and Perceptive Pixel. Jeff Han is happy how designs created in Windows 8 scales for use on screens large and small. He explains how Windows 8 can support collaboration. The Story Board application (7:58) on the large touchscreen display looks interesting.
I continue to be frustrated by the poor usability of many web-based and desk-top applications. I like my iPad, but only because so many dedicated souls have given some thought to the user experience when creating their apps. I often meet with disappointment when I encounter interactive displays when I'm out and about during the day. It is 2012, and it seems that there are a lot of application designers and developers who have never read Don Norman's The Design of Everyday Things!
I enjoy making working prototypes and demo apps, but my skill set is stuck in 2008, the last year I took a graduate-level computer course. I was thinking about taking a class next semester, something hands-on, creative, and also practical, to move me forward. I can only do so much when I'm in the DIY mode alone in my "lab" at home. I need to explore new tools, alongside like-minded others.
There ARE many more tools available to designers and developers than there were just four years ago. Some of them are available online, free, or for a modest fee. I was inspired by a link posted by my former HCI professor, Celine Latulipe, to her updated webpage devoted to Rapid Prototyping tools. The resources on her website look like a good place to start for people who are interested in creating applications for the "NUI" era. (Celine has worked many interesting projects that explore how technology can support new and creative interaction, such as Dance.Draw.) Below is her description of her updated HCI resources:
"New HCI resource to share: I have created a few pages on my web site devoted to Rapid Prototyping tools, books, and methods. These pages contain reviews of various digital tools, including 7 different desktop prototyping apps, and including 8 different iPad apps for wireframing/prototyping. I hope it's useful to others. Feel free to share... and please send me comments and suggestions if you find anything inaccurate, or if you think there is stuff that I should be adding. I will be continuing to update this resource." -http://www.celinelatulipe.com (click on the rapid prototyping link at the top)
IDEAS Below are just a few of my ideas that I'd like to implement in some way. I can't claim ownership to these ideas- they are mash-ups of what comes to me in my dreams, usually after reading scholarly publications from ACM or IEEE, or attending tech conferences.
An interactive timeline, (multi-dimensional, multi-modal, multimedia) for off-the-desktop interaction, collaboration, data/info analysis exploration. It might be useful for medical researchers, historians, genealogists, or people who are into the "history of ideas". Big Data folks would love it, too. It would handle data from a variety of sources, including sensor networks. It would be beautiful to use.
A web-based system of delivering seamless interactive, multi-modal, immersive experiences, across devices, displays, and surfaces. The system would support multi-user, collaborative interaction. The system would provide an option for tangible interaction.
A visual/auditory display interface that presents network activity, including potential intrusions, malfunctions, or anything that needs immediate attention that would be likely to be missed under present monitoring methods.
Interactive video tools for creation, collaboration, storytelling. (No bad remote controllers needed.)
A "wearable" that provides new ways for people to express and communicate creatively, through art, music, dance, with wireless capability. (It can interact with wireless sensor networks.)*
An public health application designed to provide information useful in understanding and sepsis prevention efforts.This application would utilize the timeline concept describe at the top of this list. This concept could also be useful in analyzing other medical puzzles, such as autism.
Most of these ideas could translate nicely to educational settings, and the focus on natural user interaction and multi-modal i/o aligns with the principles of Universal Design for Learning, something that is important to consider, given the number of "at-risk" learners and young people who have disabilities.
I welcome comments from readers who are working on similar projects, or who know of similar projects. I also encourage graduate students and researchers who are interested in natural user interfaces to and move forward with an off-the-desktop NUI project. I hope that my efforts can play a part in helping people make the move from GUI to NUI!
Below are a few videos of some interesting projects, along with a list of a few references and links.
Overview One of the primary goals of teaching is to prepare learners for life in the real world. In this ever-changing world of technologies such as mobile interaction, cloud computing, natural user interfaces, and gestural interfaces like the Nintendo Wii and Microsoft Kinect, people have a greater selection of tools for the task at hand. Given the potential of these new interfaces, software, and technologies as learning tools, as well as the ubiquitous application of interactive technology in formal and informal learning environments, there is a growing need to explore how next-generation technologies will impact education in the future.
As a community of Human-Computer Interaction (HCI) and educational researchers, we need to theorize and discuss how new technologies should be integrated into the classrooms and homes of the future. In the last three years, three CHI workshops have provided a forum to discuss key issues of this sort, particularly in the context of next-generation education. The aim of this special issue of Personal and Ubiquitous Computing is to summarize the potential design challenges and perspectives on how the community should handle next-generation technologies in the education domain for both teachers and students.
We invite authors to present position papers about potential design challenges and perspectives on how the community should handle the next generation of HCI in education. Topics of interest include but are not limited to:
Submissions should be prepared according to the Word template located at the bottom of this page. All manuscripts are subject to peer review. Manuscripts must be submitted as a PDF to the easychair submission system. Submissions should be no more than 8000 words in length.
Guest Editors and Contact Information
Syed Ishtiaque Ahmed, Cornell University
Quincy Brown, Bowie State University
Jochen Huber, Technische Universität Darmstadt
Si Jung “Jun” Kim, University of Central Florida
Lynn Marentette, Union County Public Schools, Wolfe School
Yesterday evening I attended a meetup of TechCrunch and Charlotte-area techies, held at the uptown Packard Place entrepreneurial center. It was jammed-packed with people all abuzz with tech start-up fever, fueled by awesome food provided by Zen Fusion. Although my main purpose for attending the TechCrunch meet-up was to learn more about innovative technology start-ups in my region, I also was hoping to capture a few shots of interesting people. I like to keep my eye open for tee-shirt slogans, and one worn by a young gentleman caught my eye, proclaiming that he'd seen the future, and it is in his browser. On the back of his tee-shirt was a bright HTML5 logo, something that is dear to my heart, as I am moving from HTML4 to HTML5. He was polite and agreed to pose for a couple of photographs:
It turned out that the HTML5 guy was at the TechCrunch event with one of his colleagues from T1 Visions, a social touchscreen solutions company that I've featured previously on this blog. They caught me up on the growth of this start-up company, which now has 15 employees and has broadened its reach beyond table-top restaurant applications to the healthcare, education, corporate, retail, and broader hospitality sectors.
What I like about table-top systems is that they provide support for "natural user interaction". It allows for multiple modes of interaction with and presentation of multimedia content. Over the past several years, these systems have been proven to be useful to a wider range of people and settings. Interfaces that support touch and gesture interaction are no longer viewed as novelties, given the pervasiveness of touch-phones and tablets and their ease-of-use for most people.
A useful product from T1 Visions is the T1 Collaboration Table. It supports touch-screen interaction and can also handle up to four simultaneously connected laptops. The table system provides a media viewer that supports sharing of photos across screens, devices, and surfaces. It also contains a web browser, a presentation viewer, and a whiteboard that is compatible with video conferencing. The company provides customized applications for its clients. In the Charlotte area, some of the tables can be found in restaurants, such as the Mellow Mushroom, Cowfish, and Harpers. A few were recently installed in the Atkins library at UNC-Charlotte, to support group-work among students.
To learn more about what T1 Visions has to offer, take a few minutes to view the following videos and follow the links at the end of this post!
Demonstration of how the collaboration table can work within a business environment:
NOTE: TechCrunch is a technology media group founded in 2005 that focuses on innovative technologies. This summer, a group of TechCrunchers are visiting cities in the south that were previously not under their radar, such as Savannah and my home region, Charlotte, N.C. The Charlotte TechCrunch meetup was held on Wednesday, July 11, 2012. I plan to devote a few more blog posts to share what I learned.
