Tom Barrett, a teacher, demonstrates Durham University's SynergyNet Multi-touch Networked Tables

"It is about how children can work together and communicate and how the multi-touch technology can facilitate this."




In this video, Tom Barrett demonstrates multi-touch networked tables, which were designed to be part of an immersive classroom environment. The multi-touch tables are the result of a collaboration between the Technology Enhanced Learning research group at Durham University and colleagues from the Education and Psychology departments.

Tom is a teacher and edublogger. Like me, he has a passion for multi-touch technology. He has been fortunate to have the chance to work with a SMARTTable in his classroom, and also compare the SMARTTable experience with the tables at Durham University, which were not from SMART Technologies.

Tom was one of the first teachers to have the opportunity to try out the SMART Table in his classroom. He was a bit disappointed with the outcome. Even so, he believes that multi-touch technology will be important in education in the future.

From Tom's point of view, there is a need to have more in-depth content for the SMART Table, even at the earlier grades. The following quotes are taken from Tom's "SMART Table in my Classroom- My Conclusions" blog post:

"In my opinion there are three things that contribute to this: poor content; poor creation software and a straight jacketed approach to multi-touch functionality...There seems to be too much residual SMART Notebook thinking and not enough innovative software design. Maybe the product has preceded the necessary thinking behind it all."

"The one shining ray of light that emerges from amidst this all is the Media application. I have posted videos of some of my children working with this program in the past. It remains the only application that offers teachers and children an open environment to learn, and couples it with a unique interface with media. When you use this application you actually feel like you are using something innovative, multi-touch, gestural driven. As a teacher there is the capacity to use rich content of your choice (video) and then layer on top questions that engage the children in a much deeper way"

RELATED

Flickr Group: Multi-touch Interactive Desk: Applications and Gesture Ideas

(Note: I have plenty of ideas for content and software design for multi-touch tables in education, and also cognitive and educational assessment. I'm only missing a table or two!)

What is new at SMART Technologies? 3D-Ready SMART Board 600i, Mixed Reality Document Camera, Scholastic's Story Stage App for the SMARTTable, an IPO, and more!

A lot is happening at one SMART Technologies, one of my favorite "interactive" companies.  Not long ago, SMART Technologies, a company most known for interactive whiteboards found in classrooms and businesses, acquired one of my favorite interactive touch-screen display companies, NextWindow. (SMART Technologies' acquisition of NextWindow: A "smart window" to the world)

SMART Technologies is moving ahead into the world of augmented reality (AR) with a new document camera.  The company's latest interactive whiteboard, the fourth generation 600i,  is ready for stereoscopic 3D content, which is on the way via the Discovery Channel and other media companies.  The SMARTTable looks like it will be provided with more content, at least in the UK, where Scholastic has partnered with SMART Technologies to provide multi-touch and multi-student interaction via the collaborative Story Stage application.

For those with money to invest, SMART Technologies has filed for a proposed initial IPO (Initial Public Offering).


-The SMART Table at Wolfe School 

New SMART Document Camera 330 is mixed reality ready:  Document camera enables exploration of 2D, 3D, animated and audio-enriched content
"Mixed Reality Technology -Manipulate and explore 2D, 3D, text, animation and audio-enriched content on the SMART Board interactive whiteboard by placing a physical object bearing a digital marker under the SMART Document Camera lens"


SMART introduces next-generation SMART Board™ 600i: New projector lowers total cost of ownership and is 3D-ready
"Available in both standard and widescreen formats, the latest generation of the 600i system features an improved and easy-to-use extended control panel (ECP) and a fully integrated short-throw projector that is 3D-ready and has a longer lamp life. The newly designed ECP is mounted to the interactive whiteboard bezel, not to the wall, making it aesthetically pleasing and easier to install."


SMART and Scholastic to develop multi-touch educational content:  Story Stage application for SMART Table to encourage student creativity and collaboration
"Story Stage is an easy-to-use resource designed to facilitate collaborative literacy work. The interactive activities use digital puppets to encourage pairs or groups of children to work together to create their own imaginative retellings of familiar stories."


SMART Classroom Suite 2010 enhances interactive learning: New version offers improved wireless performance and greater support for formative assessment
"The software combines classroom management, assessment, lesson creation and assignment management tools in one offering."


RELATED
SMART Technologies Files for Proposed Initial Public Offering: Stock to be listed on NASDAQ and TSX
Scholastic Story Stage
SOMEWHAT RELATED
Updates: Cognitive "bursts", technology-supported interactive whiteboards, digital storytelling, social skills, and reflections about a new SMARTtable. (long post)
SMART Technologies' acquisition of NextWindow: A "smart window" to the world
InfoComm AV Whitepaper:
Reaching Learners: Immersive Education through Interactive Multimedia (pdf)
By Lynn V. Marentette, Sp.A., NCSP and Anthony Uhrick, VP Sales & Marketing, NextWindow "The value of an immersive approach to education, using multimedia technologies, over traditional classroom environments."

Also posted on the TechPsych blog.

New SMARTBoard Touch Recognition from SMART Technologies: The YouTube Video

Here's the plug:
"SMART's new Touch Recognition feature allows the SMART Board to recognize your touch and switch modes automatically. You can write with a pen, erase with the palm and move objects around with your finger without having to access other tools, buttons or on-screen menus."

Related

Learning Through Touch: The story behind the SMART Table pdf (Heather Ellwood, EdCompass, January 2009)

SMART Table Website

Update, plus iGaze app by Dunedin Multimedia for use during social skills group activities

So what am I up to now?  


I'd like to share with my readers that I've decided to continue in my present position as a school psychologist, while still devoting a portion of my free time to technology. From time-to-time I think deep thoughts about usability, accessibility,  and UX/Interaction related to off-the-desktop interactive multimedia applications running on screens of all sizes.  I'm hoping to create a few multimedia experiments using HTML5 and JavaScript, and explore jQuery if and when I can find the time!  


For the present school year, my main school is a program for students with more significant disabilities, including autism spectrum disorders.  My second school is a magnet high school for technology and the arts,  located on the same campus.  I also consult throughout the district on cases involving students who have suffered traumatic brain injuries, as well as students who have multiple disabilities.  I am thankful that I have a job in a school district that values 21st Century technology.  


I'm looking forward to another technology-rich school year.  I've spent some of the time I usually devote to blogging devoted exploring iPad apps instead.  Since I'm new to the world of iPads, I'm still in discovery mode.  What an adventure!   


There are plenty of educational apps out there, and many of them are suitable for students with special needs.  On the other hand, there is much room for improvement - across all iPad app categories.  Since there is very little research about what makes up a killer app- or suite of apps- for students with special needs, experimenting with  iPad apps is uncharted territory. 

I made the decision to bring my personal iPad2 to work after I discovered a number of apps that I thought would be useful in my work as a school psychologist with students who have special needs, including autism spectrum disorders.  

One of my intervention themes this year focuses on social skills.   This is especially important for students who participate in our schools community-based job training program.  I'm using some content from Unique Learning's transition materials,  as well as on-line activities from Do2Learn's JobTips website, because my aim is to facilitate social skills that will be useful in a variety of job and community settings.   


Although my main technology tool for working with groups is the SMARTBoard,  I've found that using a combination of interactive whiteboard and iPad activities to be especially effective.  I'm paving the way for more role-play activities in the future, and attempting to use technology to my advantage.


This past week, I used the iGaze app, created by Dunedin Multimedia, to help a group of high-school level students practice establishing and maintaining eye gaze, something that is difficult for most of them to demonstrate "in-person".  I was amazed.  Each student was excited to take his or her turn.  Even more amazing?  When each student took a turn, the other students looked at their eyes and faces.  No one rocked or "stimmed".  No one made noises.  I observed several instances of joint attention, much to my delight.   

Below is a video from Dunedin Multimedia's YouTube channel that is similar to what the students viewed during their group activity:

Here is some information from Dunedin Multimedia about the iGaze app:

"Eye contact is important to communication and social development, and yet the impaired ability to make and maintain eye contact is one of the most striking aspects of autism. iGaze is an eye contact simulator that can help to build confidence in using this important means of nonverbal social communication.  The app also contains information on eye contact and eye gaze, with links to relevant research."

During the social skills activity involving the iGaze app,  I used the SMART Board to display a large picture of a boss and a worker standing face to face, making eye contact, engaged in conversation.  The picture served as an anchor to remind the students of pictures and videos they'd previously viewed that illustrated the concept of face-to-face interaction and the importance of establishing eye-contact with others from time-to-time.


I'm hoping I will be able to access the YouTube videos from Dunedin so I can use them on the SMART Board. It will be interesting to see how this plays out!   I'm also planning to take a closer look at Dunedin Multimedia's emotion x app for the iPad.


RELATED 
Screen-shot of iGaze for the iPad Dunedin:

SOMEWHAT RELATED
The SMART Table at my school was updated today - I'm looking forward to using it for some group activities, now that it is back in working order and has new applications loaded up and ready to go!


If you are interested in learning more about technology related to students with special needs, be sure to check out Kate Ahern's blog, Teaching Learners with Multiple Special Needs

Kate's post about the features of Unique Learning Systems.


Upcoming:  more about tablets, interview with folks from Stantum, social-skills game-in-progress.....large displays in public spaces update....

Multi-touch and Flash: Links to resources, revisiting Jeff Han's TED 2006 presentation

Despite the increase in interest in systems that support multi-touch, multi-user multimedia interaction, there is a need for creative, tech-savvy types to develop innovative applications. Why? This technology has the potential to make a powerful impact on how people learn, communicate, solve "big picture" problems, and do their various jobs.

CNN's Magic Wall was one of the first applications to gain the attention of the masses, as it was used as an interactive map during the US presidential election process. Touch-screen interaction gained even more notice after the recent SNL parody by Fred Amisen.

If you think about it, the multi-touch applications you see on the news aren't much different than what you'd get from a "single-touch" program.

Fancy, yes. Truly innovative, no.

Just imagine a 3D multi-touch, multi-user, multimedia version of Google Search. I did. I put my sketches in my idea book and hurt my brain thinking about how it could be coded.

Jeff Han, the man behind Perceptive Pixel and CNN's magic wall, had much more up his sleeve when he demonstrated his work at TED 2006. Even if you've previously seen this video, it is worth looking at again. (I've provided a link to the transcript below.)



Transcript of Jeff Han's TED 2006 Presentation

This video presentation had a transformational effect on me as I watched for the first time. Jeff Han brought to life ideas that were similar to my own as a beginning computer student thinking about collaborative educational games and multimedia applications that could be played on interactive whiteboards.

Here are some selected quotes from the video:

"I really really think this is gonna change- really change the way we interact with the machines from this point on."

"Again, the interface just disappears here. There's no manual. This is exactly what you kind of expect, especially if you haven't interacted with a computer before."

"Now, when you have initiatives like the hundred dollar laptop, I kind of cringe at the idea that we're gonna introduce a whole new generation of people to computing with kind of this standard mouse-and-windows pointer interface. This is something that I think is really the way we should be interacting with the machines from this point on. (applause)"

"Now this is going to be really important as we start getting to things like data visualization. For instance, I think we all really enjoyed Hans Rosling's talk, and he really emphasized the fact that I've been thinking about for a long time too, we have all this great data, but for some reason, it's just sitting there. We're not really accessing it. And one of the reasons why I think that is, is because of things like graphics- will be helped by things like graphics and visualization and inference tools. But I also think a big part of it is gonna be- starting to be able to have better interfaces, to be able to drill down into this kind of data, while still thinking about the big picture here."

So now what?

A recent post by "Alex", on the AFlex World blog discusses a few solutions. Alex had a chance to meet with Harry van der Veen and Pradeep George from the NUI Group, and Georg Kaindl, a multi-touch interaction designer from the Technical University of Vienna. The focus of the discussion was to come up with ideas to encourage Adobe/Flash designers and developers to learn more about multi-touch technology and interaction, and take steps to create innovative applications.

I especially like the following quote from the post:

"...A quick quote from our conversations: “When our children will walk up to a display, they will touch it and expect to do something.”

As a techie and a school psychologist, I see an immediate need for innovative applications. I know that there is a built-in market in the schools, at least for low-cost applications. Despite economic constraints, many school districts continue to invest in interactive whiteboards (IWB's). They are cropping up in preschool and K-12 settings, and teachers are searching for more than what's currently available.

Interactive, collaborative applications are needed in fields such as health care, patient education, finance & economics, urban planning, civil engineering, travel & tourism, museums & exhibitions, special events, entertainment, and more.

Smart Technologies, the company behind SmartBoards, has a new interactive multi-touch, multi-user table designed for K-6 education, the Smart Table. Hewlett Packard has several versions of the TouchSmart PC, which can support at least duo-touch, if not multi-touch, multi-user applications. There are numerous all-in-one large screen displays on the market that support multi-touch and multi-user interaction.

Quotes from Harry van der Veen, of Multitouch NL:

"In 10 years from now when a child walks up to a screen he expects it to be a multi-touch screen with which he can interact with by using gestures."

"...multi-touch screens will be as common as for children is the internet nowadays, as common as mobile phones are for us."


Here is a quote from a conversation I had with Spencer, who blogs at TeacherLED.

"It was interesting this week as I was in a classroom with a teacher who I've not worked with before... he had 2 students using the whiteboard who kept touching it together by mistake. The teacher, exasperated, said to himself, "Why can't they make these things to accept 2 touches without going crazy!"

Proof of the demand! I think you are right when teachers spot the limitations and then see the technology on visits to museums, that might stimulate demand."

Spencer creates cool interactive mini-applications, mostly for math, using Flash, that teachers (and students) love to use on interactive whiteboards. (He's interested in multi-touch, too.)


So what are we waiting for?!

Related:
Natural User Interface Europe AB meets Adobe
Georg's Touche Framework
NUI Group
TeacherLED
Interactive Touch-Screen Technology, Participatory Design, and "Getting It".
Hans Rosling's 2007 TED talk

MICROSOFT: ARE YOU LISTENING? Cool Cat Teacher (Vicki Davis) Tests Out Microsoft's Multi-touch Surface Table

When Microsoft unveiled the Surface in 2007, I was disappointed that there were no immediate plans to market it for use in education. At the time, there were no immediate plans to release the Surface for research and development by university students. I had just completed a course in human-computer interaction, and with my background as a school psychologist, I KNEW this sort of technology would work well in education.

My head was brimming with ideas for this innovative technology for use with the students I work with every day.

The following video shows a demonstration of Microsoft's multi-touch, multi-user Surface table at the 2009 NECC conference, and also provides insightful comments from Vicki Davis, author of the very popular Cool Cat Teacher blog. Vicki discusses the value of surface/tabletop computing in education and shares her views about the need for user involvement in the educational software development process. She also gives great advice about how Microsoft or other developers of tabletop computing systems should proceed.

I agree with Vicky's comments, 100%, as my regular blog readers know!

MICROSOFT, ARE YOU LISTENING?



In the above video, it is apparent that the musical instrument applications do not provide a good touch response on the Surface. Vicki suggests that touch responsiveness is key, and that all Surface applications should be held to the high standard of Apple's iPhone and iPod Touch. Vicky goes on to say that Microsoft should support easy development of applications, and ensure that applications are very user-friendly and easy for school folks to install. If you work in a K-12 setting, you know what I am talking about ; )

Vicky is preaching to the choir when she stresses that Microsoft R&D (and others) should involve users in ongoing development, in a meaningful way, by using REAL people, REAL teachers, people who work with students. If you have ever battled with a "lame" educational application, you know why this is so very important!

Vicky's enthusiasm for the use of tabletop/surface computing in education can not be ignored. She absolutely knows what she is talking about, and she is the instructional technology voice for a multitude of educators around the world.

Here is Vicky's plea:

"I wish Microsoft would listen to these 3 things from a teacher in a classroom (me). I know Microsoft has bigger things to do than watch this video, but, I can dream, can't I?"

(I've paraphrased the following quotes.)

1. Understand the amazing potential for Surface devices in education. Look at three to five years out. You are looking at the future.

(This technology can engage students who have ADHD, etc.)

2. Harness the power of your users! Pull in your users. There are so many people in education would give their thoughts for free!

3. Create virtual and online ways for Microsoft to interact with teachers.


"If Microsoft decides to invest in this, and I do hope that somebody watching this video will understand the importance of integrating the world around us into the learning experiences and the learning environment, as part as how we remake and re-do education."

My sentiments exactly!

A few thoughts:

So where are we now?

Smart Technologies has come out with the SMARTTable, but it was designed for younger students. At this point, there are very few options, especially affordable options, for educators of students in the upper grades to use this technology.

Some members of the NUI-group are involved in creating educational applications for table-top systems, but they are few in number.

A few companies are using this technology for education, but the applications are mostly limited to interactive museum exhibitions.

From my research on this topic, there are very few developers that have the interest or the inclination to create educational applications for table-top computing.

My hope is that this will change soon! Join me in this conversation.

Social Thinking blog - Great post from Michelle Winner's Blog

We're planning an application for a multi-touch SMART Table at one of my schools, and one of the goals is to ensure that it can facilitate social interaction among students with and without disabilties, including autism.  Keeping this in mind, one of my colleages sent me an email a minute ago with a link to a great post by Michelle Winner's blog about social thinking and "inclusion" vs "integration" in education settings.  It is worth a read!

http://www.socialthinking.com/michelles-blog/social-skills-for-an-integrated-setting.html

There is a need for multi-touch/gesture designers/developers!

If you are a talented interactive web designer/developer, game designer/developer, traditional programmer with a creative bent, or someone who who is thinking about working with technology in the future as a programmer or designer,  I urge you to consider thinking about designing/developing multi-touch applications in the near future.

In my opinion, there will be a need for multi-touch web applications as well as for multi-touch education and collaboration applications for the SMART Table, Microsoft's Surface,  multi-touch tablets like the rumored iTablet from Apple, and the multi-touch laptops and all-in-ones (Dell, HP, etc.).

Below are direct links to some of my blog posts related to multi-touch applications and screens. If you are fairly new to multi-touch, I'm sure that looking through some of my blog posts will be helpful.  All of the posts have links to resources, and most have photos and video clips of multi-touch in action.

If you are new to this blog, I have a great deal of information, links, photos, and video clips of various multi-touch screens and applications. The best way to find the stuff is to enter in a keyword in the search box for this blog:  multitouch, touch screen, gesture, multi-touch, etc. on  this blog.

Also do a search on my other blog: The World Is My Interface http://tshwi.blogspot.com

Here are some links:
Do you have an HP TouchSmart, Dell Studio One or NextWindow touch-screen? NUITech's Snowflake Suite upgrade provides a multi-touch plug-in
http://bit.ly/5tdlhc

The following blog post has a video clip that shows someone from Adobe painting with a multi-touch application in development:
More Multi-Touch!: Rumor of the mobile apple iTablet; AdobeXD & Multitouch; 10-finger Mobile Multitouch: http://bit.ly/4S9Upm

Ideum's GestureWorks: http://bit.ly/4C1p7M

Interactive Walls, Interactive Projection Systems, GestureTek's Motion-Based Games: http://bit.ly/6GRGtW

Intuilab's Interfaces: Multi-touch applications/solutions for presentations, collaboration, GIS, and commerce: http://bit.ly/7RK7qN

For software developers:
How to do Multitouch with WPF 4 in Visual Studio 2010: http://bit.ly/7c4YqC

More touch screen "surface" display musings...

I had my first chance to use an interactive touch-screen SmartBoard, by Smart Technologies in 2002-03. Since I work mostly with kids and teens, I wondered why large-display touch screen technology wasn't more widespread, since there are so many free, interactive websites that provide pretty engaging activities for users.

One of the things I learned was that large-display touch-screen technology is in the preschool stage. There are problems with screen responsiveness, screen resolution, durability, and input.

In recent years, the idea of a touch screen has evolved to table-tops and drafting boards, embedded within wireless systems that allow for interoperability with mobile devices and remote applications.

Great technology exists, but no-one has pulled all the components together in a way that can easily scale for the people who would benefit from this sort of technology the most - people who spend most of day time teaching, learning, or both. I had a great experience using a NextWindow Human Touch large-screen display for some of my projects last semester. It was difficult for me to track one down, but once I got my hands on it, I liked it, even though it did not have multi-touch capabilities.

One laptop for each child? That was a good idea for the late 1990's and early 2000's. One high-quality, affordable, large touch-screen display or table for each classroom would be more effective.

One touch-screen display/table for every 4-6 students would be better. Is there anyone out there who is up for the challenge?

Next Post: Updated links to interactive multimedia websites appropriate for large touch screen surfaces.

Michael Haller Discusses Multi-touch, Interactive Surfaces, and Emerging Technologies for Learning

I came across an excellent overview of interactive display technologies that hold promise for education. The link below is a research article written by Michael Haller for BECTA, formally known as the British Educational Communications and Technology Agency.

Emerging Technologies for Learning: Interactive Displays and Next Generation Interfaces(pdf)
Becta Research Report (2008) Michael Haller Volume 3 (2008)

"Multi-touch and interactive surfaces are becoming more interesting, because they allow a natural and intuitive interaction with the computer system.

These more intuitive and natural interfaces could help students to be more actively involved in working together with content and could also help improve whole-class teaching activities. As these technologies develop, the barrier of having to learn and work with traditional computer interfaces may diminish.

It is still unclear how fast these interfaces will become part of our daily life and how long it will take for them to be used in every classroom. However, we strongly believe that the more intuitive the interface is, the faster it will be accepted and used. There is a huge potential in these devices, because they allow us to use digital technologies in a more human way." -Michael Haller

Michael Haller works at the department of Digital Media of the Upper Austria University of Applied Sciences (Hagenberg, Austria), where he is the head of the Media Interaction Lab.

Michael co-organized the Interaction Tomorrow course at SIGGRAPH 2007, along with Chia Shen, of the Mitsubishi Electric Research Laboratories (MERL). Lecturers included Gerald Morrison, of Smart Technologies, Bruce H. Thomas, of the University oof Southern Australia, and Andy Wilson, of Microsoft Research. The course materials from Interaction Tomorrow are available on-line, and include videos, slides, and course notes.

Below is an excerpt from the discription of the Interaction Tomorrow SIGGRAPH 2007 course:

"Conventional metaphors and underlying interface infrastructure for single-user desktop systems have been traditionally geared towards single mouse and keyboard-based WIMP interface design, while people usually meet around a table, facing each other. A table/wall setting provides a large interactive visual surface for groups to interact together. It encourages collaboration, coordination, as well as simultaneous and parallel problem solving among multiple people.

In this course, we will describe particular challenges and solutions for the design of direct-touch tabletop and interactive wall environments. The participants will learn how to design a non-traditional user interface for large horizontal and vertical displays. Topics include physical setups (e.g. output displays), tracking, sensing, input devices, output displays, pen-based interfaces, direct multi-touch interactions, tangible UI, interaction techniques, application domains, current commercial systems, and future research."

It is worth taking the time to look over Haller's other publications. Here is a few that would be good to read:

M. Haller, C. Forlines, C. Koeffel, J. Leitner, and C. Shen, 2009. "Tabletop Games: Platforms, Experimental Games and Design Recommendations." Springer, 2009. in press [bibtex]

A. D. Cheok, M. Haller, O. N. N. Fernando, and J. P. Wijesena, 2009.
"Mixed Reality Entertainment and Art," International Journal of Virtual Reality, vol. X, p. X, 2009. in press [bibtex]

J. Leitner, C. Köffel, and M. Haller, 2009. "Bridging the gap between real and virtual objects for tabletop games," International Journal of Virtual Reality, vol. X, p. X, 2009. in press [bibtex]

M. Haller and M. Billinghurst, 2008.
"Interactive Tables: Requirements, Design Recommendations, and Implementation." IGI Publishing, 2008. [bibtex]

D. Leithinger and M. Haller, 2007. "Improving Menu Interaction for Cluttered Tabletop Setups with User-Drawn Path Menus," Horizontal Interactive Human-Computer Systems, 2007. TABLETOP 07. Second Annual IEEE International Workshop on, pp. 121-128, 2007. [bibtex]

J. Leitner, J. Powell, P. Brandl, T. Seifried, M. Haller, B. Dorray, and P. To, 2009."Flux: a tilting multi-touch and pen based surface," in CHI EA 09: Proceedings of the 27th international conference extended abstracts on Human factors in computing systems, New York, NY, USA, 2009, pp. 3211-3216. [bibtex]




P. Brandl, J. Leitner, T. Seifried, M. Haller, B. Doray, and P. To, 2009. "Occlusion-aware menu design for digital tabletops," in CHI EA 09: Proceedings of the 27th international conference extended abstracts on Human factors in computing systems, New York, NY, USA, 2009, pp. 3223-3228. [bibtex]


References from the BECTA paper:

Elrod, S., Bruce, R., Gold, R., Goldberg, D., Halasz, F., Janssen, W., Lee, D., Mc-Call, K., Pedersen, E., Pier, F., Tang, J., and Welch, B., Liveboard: a large interactive display supporting group meetings, presentations, and remote collaboration, CHI ’92 (New York, NY, USA), ACM Press, 1992, pp. 599–607.

Morrison, G., ‘A Camera-Based Input Device for Large Interactive Displays’, IEEE Computer Graphics and Applications, vol. 25, no. 4, pp. 52-57, Jul/Aug, 2005.

Albert, A. E. The effect of graphic input devices on performance in a cursor positioning task. Proceedings ofthe Human Factors Society 26th Annual Meeting, Santa Monica, CA: Human Factors Society, 1982, pp. 54-58.

Dietz, P.H., Leigh, D.L., DiamondTouch: A Multi-User Touch Technology, ACM Symposium on User Interface Software and Technology (UIST), ISBN: 1-58113-438-X, pp. 219-226, November 2001.

Rekimoto, J., SmartSkin: An Infrastructure for Freehand Manipulation on Interactive Surfaces,
CHI 2002, 2002.

Kakehi, Y., Iida, M., Naemura, T., Shirai, Y., Matsushita, M.,Ohguro, T., ‘Lumisight Table: Interactive View-Dependent Tabletop Display Surrounded by Multiple Users’, In IEEE Computer
Graphics and Applications, vol. 25, no.1, pp 48 – 53, 2005.

Streitz, N., Prante, P., Röcker, C., van Alphen, D., Magerkurth, C.,Stenzel, R., ‘Ambient Displays and Mobile Devices for the Creation of Social Architectural Spaces: Supporting informal communication and social awareness in organizations’ in Public and Situated Displays: Social and Interactional Aspects of Shared Display Technologies, Kluwer Publishers, 2003. pp. 387-409.

Morrison, G., A Camera-Based Input Device for Large Interactive Displays, IEEE Computer Graphics and Applications, vol. 25, no. 4, pp. 52-57, Jul/Aug, 2005.

Ishii, H., Underkoffler, J., Chak, D., Piper, B., Ben-Joseph, E., Yeung, L. and Zahra, K., Augmented Urban Planning Workbench: Overlaying Drawings, Physical Models and Digital Simulation. IEEE and ACM International Symposium on Mixed and Augmented Reality ACM Press, Darmstadt, Germany.

Han, Y., Low-cost multi-touch sensing through frustrated total internal reflection, UIST ’05 (New York), ACM Press, 2005, pp. 115–118.

Hull., J., Erol, B., Graham, J., Ke, Q., Kishi, H., Moraleda, J., Olst, D., Paper-Based Augmented Reality. In Proceedings of the 17th International Conference on Artificial Reality and Telexistence (Esbjerg, Denmark,November 28-30, 2007). ICAT ’07. IEEE, 205-209.

Haller, M., Leithinger, D., Leitner, J., Seifried, T., Brandl, P., Zauner, J., Billinghurst, M., The shared design space. In SIGGRAPH ’06: ACM SIGGRAPH 2006 Emerging technologies, page 29, New York, NY,USA, 2006. ACM Press.

Research email: emtech@becta.org.uk
Main email: becta@becta.org.uk
URL: www.becta.org.uk

(This was also posted on the TechPsych blog.)

A Long Mix of Topics: Updates - Cognitive "bursts", technology-supported interventions, interactive whiteboards, digital storytelling, social skills, and reflections about a new SMARTtable.

UPDATED


In this post, I share some of my reflections about the new multi-touch SMARTTable that arrived last week at one of the programs I serve as a school psychologist.  I decided to update two posts I wrote in 2008, because they provide links and resources that will be useful to my colleagues as we explore this exciting technological tool.
We have a SMARTTable!  

-SMART Technologies

The SMARTTable arrived last Thursday!  

I'll be meeting with some of my colleagues to review our growing library of digital content and figure out what sort of content and activities we can quickly import into the toolkit that came with the SMARTTable.  We also will  refine our ideas about a new application that we hope to enter in a contest sponsored by SMART Technologies.  (I have a few experiments I created using C#, WPF, and Visual Studio 2008 that I'd like to flesh out for the SMARTTable.  I'm counting on one of the teacher assistants, a retired computer programmer who is familiar with C++, to help, and hopefully, a few high school computer students.)


Note: The SMARTTable developer's SKD works on 32-bit Windows computers, according to a representative from SMARTTechnologies.  My computer is 64 bit.  I am investigating ways to work around this problem without having to spend additional money.  

Update:  Should be fine using Windows 7!

In the meantime, I'm thinking of ways the SMARTTable can be used for the students I work with to share their developing "sense of self" with others, focusing on the following questions:

Who am I? (Strengths, skills, preferences, barriers, etc.)

How do I feel? How do I share my feelings with others?
What are the coping strategies I can use when I feel upset?
What is my relationship to the physical world? (Objects, Places)
What is my relationship to the social world?     (Family, Community)
How can I share my sense of self with others?
How do I respond to what others share with me?

I have worked with students of all ages and abilities, usually in pairs or small  groups, on non-digital tables my entire career.  This is the case for many support professionals in the schools, such as school psychologists, school counselors, speech and language therapists, and literacy specialists.  In my case, I have over 20 years of activities related to just about any problem that has come my way.  Coming up with content isn't the problem - the challenge is figuring out how things can be transformed for use on the multi-touch tabletop.

I think the best approach is to focus on the activities I use  when I start a new group. Many of the teamwork and icebreaker activities  I use that help the students learn how  to participate in a group would translate nicely to a multi-touch table. These activities would be useful to counselors and speech and language therapists. They might also be useful to teachers when form new cooperative or collaborative learning groups in the classroom.  

I'm looking forward to learning how all of this can inter-operate:

From what I understand, the computer running the SMARTTable was not intended for accessing internet applications, and does not have anti-virus software.  I mistakenly thought that since the table was Wi-Fi enabled, I could readily access all of the single-touch web applications I use with students.  Maybe things will change.  I imagine that educational websites optimized for multi-touch would be something worth considering for the future, especially if more school schools adopt tabletop technologies.   (It would be nice to have AAC devices, video cameras, laptops, handhelds, and the Internet included in this picture.)

(Below is information from a previous post)


Introduction

Over the past several years, I have worked closely with young people who have severe autism, and during this time, I have taken a variety of computer science, software information systems, and educational technology courses. Over time, I've integrated the use of technology, including digital photography and videography into my work. In some ways, it is still a much-unchartered territory.  Through this work, I've come up with some insights that I'd like to share.

Cognitive Bursts
I've noticed that many young people who are "on-the-spectrum" experience what I call "cognitive bursts", often around puberty, but also during the late teen and early 20's. 

To an untrained eye, these bursts might go unnoticed, or even minimized.  These bursts can't easily be captured through traditional psychological or educational assessments, since these tests were designed for more typically developing students. For example, a young person with ASD might not be able to make a choice in response to a test item by pointing. Another student might not be able to respond to a test item because they do not speak.  An individually-administered cognitive assessment might not generate "IQ" scores that fully reflect significant cognitive gains, especially when the student has delays in language development and working/short term memory deficits.

As a professional, I know that it is not appropriate to provide parents with false hope. I know that the tools we have for assessing cognitive growth among students with autism spectrum disorders are not adequate. For example, two students can have the same "IQ" at age 3, 5, 8- or any age, but function much differently at age 18 or 25.  This is especially true for young people who have attention problems, working memory deficits, and/or delays in language development relative to their non-verbal abilities.

My point is that we must take early cognitive assessment  scores with a grain of salt, and   ensure that there are multiple opportunities for meaningful assessment and significant intervention during other points of a young person's development.

In my opinion, the more severe the situation, the more intensive the intervention!

Special education for students with more severe disabilities has always focused on early identification and early intervention, and for much of my career, "early intervention" was my mantra.   Over the past few years, I have come to the realization that the focus on early intervention is only a small part of the bigger picture.   While some young people make tremendous gains through early intervention, some do not, and this does NOT suggest that they won't have the potential to make significant gains later in their childhood, teens, or early adulthood.   

By focusing primarily on early intervention, we might be missing the boat. We must do more across the young person's development through young adulthood (and of course, beyond.) Each child is different, and each brain's course of development is different. One child may be ripe for growth at 30 months of age, or at age 3 or 4. Another might start talking and initiating interactions at age 14, or begin to make sense of print at age 16! I know one severely autistic youth who was reading at an 8th grade level at age 22, something that probably would not have been predicted by those who worked with him during his early years.

From what I've observed in special education, cognitive bursts are often noticed by a team of perceptive teachers, therapists, and support workers, at which point meet to discuss ways to harness this opportunity to facilitate academic, communication, and at times, social interaction skills development. While this may not be the case for each student and in each school, it really does happen! 

When a student experiences a "burst", no matter how insignificant it might look on the surface, we are given a golden opportunity to fashion an integrated approach to moving the young person forward, and at the same time, help the student develop a more solid sense of self. For students with severe autism, this might be a key to opening up their world.

Technology can help.
Because each young person develops differently, it is important that interventions designed to facilitate this sort of growth be available at all points of development, not limited to the intensive support that is recommended for the youngest of this group.  My mantra now is intervention, intervention, intervention, and INTENSIVE technology-supported intervention during periods of cognitive growth, across the developmental stages, as appropriate.

Here is what I've been doing:
I'm spending a higher percentage of my time observing students in a variety of settings, using video and digital photography to capture my observations. I am using digital content during my assessment process, and I'm using digital content for creating intervention activities that assist in measuring a student's progress over time.  

I am not alone in gathering digital content - at one of my schools, the speech and language therapist, the community-based vocational education teacher, and other teachers have still and video cameras on hand.  Every classroom in this particular school now has an interactive whiteboard (IWB), which has proven to be an effective means of sharing our digital content. The IWB is often used to provide the students with an opportunity to share their own digital content with others. By incorporating this digital content into curricular activities, were finding  that teens with autism can develop a sense of self, which in turn provides an internal  "anchor" that can scaffold their learning of social-interpersonal skills.  

What seems to be working?
I use quite a bit of digital content in my work. Some of it is generated by my colleagues, some of it I do myself, and some of the content is provided by parents.  I often take pictures and video of a student's familiar activities and settings, from the first-person point of view. To do this, I follow the student around in school, home, and/or community setting, and then shoot the various scenes as if I was in the young person's shoes.  In this way, the camera is a window to the student's world, as they see it. I supplement the video with digital photography of the same content, which then can be incorporated into an interactive PowerPoint or slide-show.   I share this content with my colleagues so they can incorporate it in their work.   

This content is used for digital social stories, preparing for community outings and job-trials, student-led IEPs, student-led presentations, and video modeling. 
I also spend some time taking video-clips and pictures of familiar items and objects the student encounters throughout the day, such as teaching materials that the teachers put up on the walls, computer screen shots, video clips of favorite songs and scenes from the television that the student watches, screen shots of educational software that the student uses, and so forth.

I use Kidspiration,  Inspiration, Umajin, and Powerpoint for much of this work. These applications are user-friendly and provide multi-modal output.  In some applications, there is a text-to-speech component that is great for pairing words with visual representations.

How does this work? 
I usually sit beside a student in a comfortable, familiar spot, with my laptop placed where it can be accessed by both the student and myself. We look at the content together. For students who are used to using a switch, I have one available.

I've found that strategies that incorporate digital media provide a means for students to generated more language and communication. This is often initiated by the students!

With students who have autism spectrum disorders, establishing a connection, through digital photography and videography, focusing on familiar things is especially important. Taking the time to capture the student's world, from their perspective, is mandatory, in my opinion. By doing this, we are providing specific information that might help to answer unspoken questions that the young person has, but lacks the skills to formulate or articulate - for example, "Who am I, and what is my relationship to this physical world?"

By taking this approach, the adults - teachers, parents, assistants - who are involved with the student, can work to build a solid scaffold for further learning and interaction. Bit-by-bit, digital content - pictures, video clips, can be built into the process to facilitate social awareness and social-emotional interaction skills. By learning about familiar people, how they "tick", and how one should go about interacting with these people, the student might gain a sense of self within a social context.  We can help them answer the question we all have, at one time or another: "Who am I, and what is my relationship to this social world?" 

Comments:
At the beginning of the 2008-09 school year, I started bringing in my HP TouchSmart PC to use with students multiple needs and those who have autism.   I've used software that takes advantage of the HP TouchSmart's duo-touch capabilities, and found that paired activities using this feature increase joint-attention behaviors, something that is important in the development of social interaction skills among young people with autism. We've used the video camera and Skype to "call" various classrooms, with the image displayed on the classroom's interactive whiteboard.  Now that every classroom has an IWB, there are possibilities for all sorts of communication activities between classrooms at the school level, and with others outside of the school. 

My mantra now is intervention, intervention, intervention, and INTENSIVE technology-supported intervention during periods of cognitive growth, across the developmental stages, as appropriate.

An Interactive Whiteboard in Each Classroom!

SMARTboards were installed in classrooms at Wolfe School during the current school year (2009-10), and have been well-utilized. Most of the teachers caught on very quickly after a short workshop and a little bit of experimentation.   I've noticed that my colleagues are motivated to explore interactive educational resources on the web and play around with the SMARTBoard resources as well.  The reason is that on the large screen, much of the content grabs the students' attention.  This provides the  teachers  a window of time to engage the students in learning and communication activities.   


All of this has taken place in a very short period of time. 

Ubiquitous IWBs have changed the way I conduct assessments of students with multiple or more severe disabilities. I've started using the SMARTboard for informal assessment, especially for students who have limited language abilities and do not point with precision.  For example,  the National Gallery of Art has a website just for young people, the "Kids Zone", and on this site are many activities that are great for IWB interaction.   

PLACES is a panoramic landscape activity that introduces children to the fundamentals of landscape and genre painting while offering a glimpse of life in rural America from the late 18th through the mid-19th century. Music and surprising animations enliven the scene, as children experiment with perspective, composition, color, and scale."   

The student can put all sorts of items on the screen, move them anywhere, and resize things.  It is a quick and fun way to see if they have mastered the concepts of size, number, direction, and so forth.  It also is a way to assess basic receptive vocabulary.

The interactive Jungle application on the SMARTBoard was also useful in eliciting joint attention:

"Create an imaginary landscape online with the NGAkids JUNGLE interactive. Mix and match the colorful characters, control the environment by changing weather and lighting conditions, or construct flowers, trees, and plants using special tools. An "AUTO" button generates random compositions, so you can sample program options and experiment with special effects as a starting point for your own designs."

If you plan to use these activities for informal assessment, make sure you have a checklist of what you plan to assess, and also make sure you've spent some time exploring the applications in depth, using the IWB.  The best thing is to practice before you try this with a student. Better yet, see if you can get the student's teacher to practice with you.


JOINT ATTENTION
If you have an IWB in your classroom and you work with students who have autism spectrum disorders, it is good to think of ways you can harness this technology to encourage joint attention among your students.  Here is some information about joint attention, from one of my previous posts:

Joint Attention:Definition of Joint Attention from UConn:
"Joint Attention is the process of sharing one’s experience of observing an object or event, by following gaze or pointing gestures. It is critical for social development, language acquisition, cognitive development…"


Establishing joint attention is an important step in the development of social interaction skills among young people who have autism spectrum disorders.

Joint Attention Study Has Implication for Understanding Autism Science Daily, 9/29/07
Asperger-Advice: Joint Attention
Autism Games: Joint Attention and Reciprocity
Why is joint attention a pivotal skill in autism?
Tony Charman

Philos Trans R Soc Lond B Biol Sci. 2003 February 28; 358(1430): 315–324. UPDATED POST FROM 9/2008

Resources for the (therapeutic) use of digital and multimedia storytelling and social stories for children and teens...

One of the most popular posts on this blog is  Interactive Multimedia for Social Skills, Understanding Feelings, Relaxation, and Coping Strategies.  If you are interested in this topic, check out the extensive resources, which I'm in the process of updating. Below are some resources and links for those of you who are interested in digital multimedia story telling or digital social stories with young people. Although some of the resources are specific to children or teens with autism spectrum disorders, I've also included information that is appropriate for use in regular classroom settings. 


Digital Storytelling and 21st Century Skills (pdf)
This nine-page primer is useful for anyone interested in learning how to create digital stories or develop digital storytelling activities with young people. The information was provided by David Jakes, an instructional technology coordinator for Community High School District 99 in Downers Grove, IL, provides a good case for digital storytelling and an outline of the process of implementing related activities at the high school level. David Jakes has a website, Jakesonline.org, that contains additional resources about digital storytelling, including strategies for instruction. The website also provide information about collaborative tools and a collection of extensive web resources.
Center for Digital Storytelling
USING MULTIMEDIA SOCIAL STORIES TO INCREASE APPROPRIATE SOCIAL ENGAGEMENT IN YOUNG CHILDREN WITH AUTISM pdf
Selda Ozdemir, Turkish Online Journal of Educational Technology, July 2008, V7(3)
Encouraging Positive Behavior with Social Stories: An Intervention for Children with Autism Spectrum Disorders pdf   Shannon Crozier, Nancy M.Sileo Teaching Exceptional Children, July/August 2005 pp. 26-31 This article provides information that supports a systematic method of implenting social stories that is integrated into a student's Functional Behavioral Assessment and IEP.


Process: 
• Team identifies the need for behavior intervention.
• Functional assessment is completed.
• Social stories included in behavior plan.
• Social story is written.
• Social story is introduced and progress is monitored with data.
• Success is evaluated with data.
An evaluation of the integrated use of a multimedia storytelling system within a psychotherapy intervention for adolescents. (pdf)
"This paper explores the use of multimedia stories in psychotherapy and mental health service delivery with teenagers. It describes a study currently being conducted with adolescents attending the Child and Adolescent Mental Health Service at the Mater Hospital Dublin, Ireland measuring the effectiveness of a therapeutic group work intervention for adolescents experiencing depression, anxiety and other mental health issues. The intervention is essentially a Cognitive Behavioural Therapy (CBT) programme that uses an animated story building system in combination with a series of short movie vignettes to help clients develop their own coping skills, express their experiences creatively and increase their ability to communicate their emotions effectively."


Current Autism Research on Social Stories (Vol 2, Issue 8; August 2007) Positively Autism
Multitimedia Instruction of Social Skills  (CITEd Research Center- Center for Implementing Technology in Education: Multimedia Technologies) This link provides extensive information about on-line resources for programs that simulate social interaction. It also includes information about the use of social stories with students, and resources for putting together multimedia social stories. Included are some summaries of research about multimedia social stories and the use of multimedia for instructional activities. Be sure to explore the rest of the CITEd site when you have the time.


Scott Bellini is a psychologist who focuses on video modeling. He is the director of Access Autism:
Bellini, S., Akullian, J., & Hopf, A. (2007). Increasing social engagement in young children with autism spectrum disorders using video self-modeling. School Psychology Review, 36, 80-90. 
Bellini, S. & Akullian, J. (2007). A meta-analysis of video modeling and video self-modeling interventions for children and adolescents with autism spectrum disorders. Exceptional Children, 73, 261-284.
RELATED
Post: Special issue on Multimedia, Media Convergence, and Digital Storytelling
Digital Stories Targeting Social Skills for Children with Disabilities. Cori More (PRO-ED Journal, 2008)
Digtal/Multimedia Storytelling from A Storied Career: Kathy Hansen's Blog to explore traditional and postmodern forms/uses of storytelling
Digital Storytelling - Katie Christo's Wiki - how-to, resources, tutorials, rubrics, lesson plans, digital storytelling across the curriculum, etc.
The Story-Centered Curriculum - eLearn Magazine
Mind Reading: An Interactive Guide to Reading Emotions
Mind Habits: The Stress Relief Game

SMARTTable Special Report
VITA: Visual Thinking in Autism, Georgia Tech