Wendy Keay-Bright's ReacTickles revised for use on the multi-touch SMARTTable!

Wendy Keay-Bright works at the Cardiff School of Art and Design in the UK, and is part of the Sensory Design Research Group.   She has focused much of her research on interactive technologies and participatory design, working children with autism, educators, and others to create applications such as ReacTickles and ReactColors, originally designed for use on interactive whiteboards.  These applications have been found to be especially effective with young people who have autism spectrum disorders. ReacTickles has been updated for use on the SMARTTable, a multi-touch, multi-user interactive table that supports collaboration, depicted in the video clips below:




(Posted on agent4changenet's YouTube channel.)

RELATED
Tabletop ReacTickles looks like a SMART move
MerlinJohnOnline 4/25/10


Note: Wendy Keay-Bright is involved with the ESRC Technology Enhanced Learning ECHOES project, which stands for "Improving Children's Social Interaction through Exploratory Learning in a Multimodal Environment."

Engaged, Interactive, Collaborative: Microsoft Surface, Finguistics, and Churchend School in the UK - Watch the video!

Finguistics created educational applications for Microsoft Surface. Watch how engaged the students are in this video as they work in groups, and look at the excitement on the faces of the teachers!



The teachers and students have lots of positive things to say about it. Play = Learn

I would like to have the chance to convert my single-touch prototypes to multi-touch for use on the Surface or something similar, but for now, I'm happy with the newly-installed SmartBoards at my favorite school and the two existing Promethean boards, which are used to deliver interactive, rather than passive learning activities for students with multiple special needs, including autism.

RELATED
Under the Hood with Finguistics
Educators get Microsoft Surface!
(MSDN Academic Alliance subscribers now have access to the Surface SDK Workstation Edition)

New SMARTBoard Touch Recognition from SMART Technologies: The YouTube Video

Here's the plug:
"SMART's new Touch Recognition feature allows the SMART Board to recognize your touch and switch modes automatically. You can write with a pen, erase with the palm and move objects around with your finger without having to access other tools, buttons or on-screen menus."

Related

Learning Through Touch: The story behind the SMART Table pdf (Heather Ellwood, EdCompass, January 2009)

SMART Table Website

BETT 09: UK's Annual ICT (Ed Tech) conference - Tabletop Computing and More

BETT is the annual ICT & educational technology conference held in the UK. The UK has the highest number of classrooms in the world with interactive whiteboards, which has been an interesting transformation to follow over the past few years.

I've posted several video clips from BETT '09, which was held this month (January), along with some other resources. Tabletop computing applications for education were demonstrated by Microsoft Surface and Smart Technologies. Take a look!

BETT 2009 Video Overview


Microsoft Surface at BETT 2009

More Surface for Education: User Interface and Paint


Physics and Social Studies


Orientation and Images


Science: Medical and Health Care


SMARTTable at BETT 09
(Note: The two clips below look as if they were taken with a cell phone video camera. I'll post higher-quality videos if I find them.)



SMART Technologies PR video

I think there is a need for more application development in this area!

RELATED
The following two clips are from the visitor's point of view, overwhelmed by it all...


SMARTTechnologies SMARTTable
Microsoft Surface
BETT 2008 Video Magazine
BETT 2008 Teachers TV Report

Need for Multi-touch, Multi-user Interactive Multimedia Applications, and the Miracle Question

Last week I received a few comments on my post, "Multi-touch and Flash: Links to Resources; Revisiting Jeff Han's Presentation". I started to respond to a thoughtful comment by Spencer, of TeacherLED, and I wanted to share it as a post:

Spencer is a teacher and instructional technology consultant who develops web-based interactive applications for use on interactive whiteboards (IWB's). He's interested in multi-touch applications for education and has some good insights into what HCI researchers call the "problem space".

Here are Spencer's comments:

..."I agree that Flash could have a very important role to play here. I chose Flash as my development tool because it allows quick development of ideas and then easy distribution of the product. The importance of this is that it allows people who have a profession other than software developer to create software with the insight of their main role. In my case, as a teacher, I can identify things I wish I had and then make them. Often I find that other teachers had the same wish and they then appreciate the product."

"The unfortunate thing with multi-touch is that it is far from the technology most of us outside the industry/research areas have to work with. An app created in Flash for single touch follows the mouse and pointer method so it can be developed easily. When done it can be easily tested on a standard IWB for the feel (which is often surprisingly different on the IWB compared to using a mouse)."

"The Flash developer community has a very experimental and creative characteristic and I’m sure would be a great driving force for multi-touch but first there needs to come a reason for more people to have some sort of multi-touch display for general use, beyond facilitating experiments. When the various operating systems support it and have the apps to make having a supporting display viable then the experimentation and ideas will really flow."

"In addition, the display makers need to recognize the benefits of Flash and ensure they address them. At the moment it seems to be too often an afterthought if considered at all. SDKs and APIs make no reference to Flash or they remain indefinitely in beta for older versions of Flash only."

"It is a pity that all of this will take time. The more time that passes the more single touch IWBs are bought and installed which will delay the uptake of the eventual multi-touch ones. Meanwhile children continue to have to keep reminding themselves that they can only touch the board in one place when it is clear that every bit of their brain is telling them to interact with the board in a much more natural multi-touch way."

My response to Spencer's comment:

Spencer,

You make good points regarding the barriers to getting the multi-touch approach adopted by the "mainstream". You're right about what the commercial display makers need to do. If they want to market displays that will have more appeal, they must think about the different sorts of applications and programming environments that the displays should support.

Display makers also need to think more about the bigger picture - in what sort of environments will the displays be located? Indoors? Outdoors? Near bright sunlight? What about people with disabilities, children, or the elderly?

I can see that in the future, multi-touch displays and other devices would operate within an embedded systems environment and support mobile computing activities as well. There are existing examples of this concept, of course, but there is much room for creative improvement. An embedded systems approach is complex, and would need to handle input from sensors, support multi-modal signal process, and also provide users with a range of connectivity modes, including RFID. (Data management and storage needs would have to be addressed, along with privacy and security concerns.)

Most importantly, in my opinion, these systems would need to have the flexibility required to support human activities and interactions that have not yet emerged! Certainly this will need to take a multidisciplinary approach.

There are many unanswered questions....How does this fit in with mobile computing and "cloud" computing? What sort of middleware needs to be developed?

Even if we don't have solutions to the bigger problems, there are many smaller problems that I think could be somewhat easily solved.

As you mentioned, many applications that are designed for single-touch screens don't fully support the way people identify, select, and move items around the screen. Although educators access websites every day for use on interactive whiteboards, they are hungry for more. There are not enough websites that are optimized for single-touch interaction, or touch-screen interaction in 3-D "space".

Teachers who are successful users of interactive whiteboards know exactly what we are talking about. They spend quite a bit of time searching for new on-line resources they can use with their students. They know how much the students want to interact with the screen at the same time and would be so excited to have capability at their fingertips!

Optimizing websites for touch-screen applications is possible, but this idea hasn't occurred to web developers. Their jobs don't require it, so there is no incentive. Google is developing FlareBrowser, that can support multi-touch interactions, but according to information on the website, it runs on Mac Leopard 1.5, and nothing else. The present version is bare-bones. I haven't yet tested the FlareBrowser.

I think that another barrier to getting multi-touch off the ground is that the people who might have the knack for multi-touch application development simply don't know it! We've mentioned that Flash developers have the potential to create good multi-touch applications. I also think that game developers and designers could make good contributions to the multi-touch movement. Just think about what thought goes into programming interactions and event handling for 3-D web-based multi-player games!

Yet another barrier is that people who work in lower-tech fields could benefit from collaborative multi-touch applications, but they don't know it, either. The research I've reviewed tells me that multi-touch applications can support a wide range of human endeavors- work, creativity, data analysis, education, collaboration, planning, and so forth.

What is missing is the input of potential end users from a variety of fields. No specific discipline "owns" multi-touch, so it is hard to figure out how we can make this happen.

Could we set up multi-touch technology playgrounds at professional and trade conferences? What about airports and hospital lobbies? Libraries and museums? Shopping centers? Sports events and rock concerts?

This leads me to my next idea, which is jumping ahead a bit:

One of the barriers to the development of multi-touch applications is that it is not easy to gather user requirements when the users are not familiar with the technology.
That is when my "Miracle Question" technique comes into play. I learned this technique when I studied brief solution-focused counseling and found that if modified, can be useful when figuring out user requirements. (The process still needs some fleshing out.)
Why the Miracle Question?
The questions that a developer uses to guide the client during the initial planning stages are very important. Keep in mind that people want to use technology because it meets a need and also solves a problem, which is the similar to the reason a person might seek counseling.
The Miracle Question technique (actually, a series of questions) might help to tease things out. The goal of this type of questioning is to help the client use their own creativity, resources, and problem-solving skills so they can become effective partners throughout the development cycle.
(People with human-computer interaction training might have an easier time understanding how this technique might be modified and applied to different fields.)

FYI
A good example the Miracle Question process, as used in therapy and counseling, can be found on the Network of Social Construction Therapies website in an article written by the late Steve de Shazer:

http://brianmft.talkspot.com/aspx/templates/topmenuclassical.aspx/msgid/366482

There aren't many resources about the use of the Miracle Question in IT or business. Here are a couple:

Solution Focused Management of Unplanned IT Outages (Read pages 132 and the references.)http://conferences.vu.edu.au/web2006/images/CDProceedings06.pdf
Proceedings of 7th International We-B (Working for E-Business) Conference, 2006Katherine O'CallaghanSugumar Mariappandar, Ph.D.School of Business and InformaticsAustralian Catholic University

Miracle Question in Executive Coaching
http://www.1to1coachingschool.com/Coaching_Miracle_Question.htm

Multi-touch and Flash: Links to resources, revisiting Jeff Han's TED 2006 presentation

Despite the increase in interest in systems that support multi-touch, multi-user multimedia interaction, there is a need for creative, tech-savvy types to develop innovative applications. Why? This technology has the potential to make a powerful impact on how people learn, communicate, solve "big picture" problems, and do their various jobs.

CNN's Magic Wall was one of the first applications to gain the attention of the masses, as it was used as an interactive map during the US presidential election process. Touch-screen interaction gained even more notice after the recent SNL parody by Fred Amisen.

If you think about it, the multi-touch applications you see on the news aren't much different than what you'd get from a "single-touch" program.

Fancy, yes. Truly innovative, no.

Just imagine a 3D multi-touch, multi-user, multimedia version of Google Search. I did. I put my sketches in my idea book and hurt my brain thinking about how it could be coded.

Jeff Han, the man behind Perceptive Pixel and CNN's magic wall, had much more up his sleeve when he demonstrated his work at TED 2006. Even if you've previously seen this video, it is worth looking at again. (I've provided a link to the transcript below.)



Transcript of Jeff Han's TED 2006 Presentation

This video presentation had a transformational effect on me as I watched for the first time. Jeff Han brought to life ideas that were similar to my own as a beginning computer student thinking about collaborative educational games and multimedia applications that could be played on interactive whiteboards.

Here are some selected quotes from the video:

"I really really think this is gonna change- really change the way we interact with the machines from this point on."

"Again, the interface just disappears here. There's no manual. This is exactly what you kind of expect, especially if you haven't interacted with a computer before."

"Now, when you have initiatives like the hundred dollar laptop, I kind of cringe at the idea that we're gonna introduce a whole new generation of people to computing with kind of this standard mouse-and-windows pointer interface. This is something that I think is really the way we should be interacting with the machines from this point on. (applause)"

"Now this is going to be really important as we start getting to things like data visualization. For instance, I think we all really enjoyed Hans Rosling's talk, and he really emphasized the fact that I've been thinking about for a long time too, we have all this great data, but for some reason, it's just sitting there. We're not really accessing it. And one of the reasons why I think that is, is because of things like graphics- will be helped by things like graphics and visualization and inference tools. But I also think a big part of it is gonna be- starting to be able to have better interfaces, to be able to drill down into this kind of data, while still thinking about the big picture here."

So now what?

A recent post by "Alex", on the AFlex World blog discusses a few solutions. Alex had a chance to meet with Harry van der Veen and Pradeep George from the NUI Group, and Georg Kaindl, a multi-touch interaction designer from the Technical University of Vienna. The focus of the discussion was to come up with ideas to encourage Adobe/Flash designers and developers to learn more about multi-touch technology and interaction, and take steps to create innovative applications.

I especially like the following quote from the post:

"...A quick quote from our conversations: “When our children will walk up to a display, they will touch it and expect to do something.”

As a techie and a school psychologist, I see an immediate need for innovative applications. I know that there is a built-in market in the schools, at least for low-cost applications. Despite economic constraints, many school districts continue to invest in interactive whiteboards (IWB's). They are cropping up in preschool and K-12 settings, and teachers are searching for more than what's currently available.

Interactive, collaborative applications are needed in fields such as health care, patient education, finance & economics, urban planning, civil engineering, travel & tourism, museums & exhibitions, special events, entertainment, and more.

Smart Technologies, the company behind SmartBoards, has a new interactive multi-touch, multi-user table designed for K-6 education, the Smart Table. Hewlett Packard has several versions of the TouchSmart PC, which can support at least duo-touch, if not multi-touch, multi-user applications. There are numerous all-in-one large screen displays on the market that support multi-touch and multi-user interaction.

Quotes from Harry van der Veen, of Multitouch NL:

"In 10 years from now when a child walks up to a screen he expects it to be a multi-touch screen with which he can interact with by using gestures."

"...multi-touch screens will be as common as for children is the internet nowadays, as common as mobile phones are for us."


Here is a quote from a conversation I had with Spencer, who blogs at TeacherLED.

"It was interesting this week as I was in a classroom with a teacher who I've not worked with before... he had 2 students using the whiteboard who kept touching it together by mistake. The teacher, exasperated, said to himself, "Why can't they make these things to accept 2 touches without going crazy!"

Proof of the demand! I think you are right when teachers spot the limitations and then see the technology on visits to museums, that might stimulate demand."

Spencer creates cool interactive mini-applications, mostly for math, using Flash, that teachers (and students) love to use on interactive whiteboards. (He's interested in multi-touch, too.)


So what are we waiting for?!

Related:
Natural User Interface Europe AB meets Adobe
Georg's Touche Framework
NUI Group
TeacherLED
Interactive Touch-Screen Technology, Participatory Design, and "Getting It".
Hans Rosling's 2007 TED talk