One of my interests is how the power and potential post-WIMP interactive technologies can be harnessed for use for formal and informal education purposes, including life-long collaborative learning.
In May, I had a chance to meet with a number of like-minded people during the CHI 2011 conference at the 2nd Workshop on UI Technologies and Impact on Educational Pedagogy. I was impressed with the depth and breadth of the presentations at the workshop. Since then, I've been looking through other papers and videos from CHI 2011 to find interesting applications that hold potential for use in educational settings.
I've come across a good number of interesting applications and prototypes, so be sure to check back for future posts on this topic. For now, here are a few applications that I'd like to share.
Below are a few videos from Phillip Chi-Wing Fu. (He doesn't know it yet, but I've admitted his videos into the Post-WIMP Explorers' Club.)
Interactive Multi-touch Sketching Interface for Diffusion Curves
"A novel multi-touch sketching interface enabling interactive and practical design with 2D diffusion curves is proposed; featured interaction techniques include simultaneous sketching of multiple diffusion curves and at-the-spot colors tuning."
Distinguishing Multiple Smart-Phone Interactions on a Multi-touch Wall Display using Tilt Correlation
"This paper proposes a novel matching technique, called tilt correlation, which employs the built-in tilt sensor on smart-phones to identify their concurrent contacts on a common multi-touch wall display."
WYSIWYF: Exploring and Annotating Volume Data with a Tangible Handheld Device (CHI 2011)
"Integration of a multi-touch wall display with a tangible handheld device with multi-touch and tilt sensing capabilities to provide intuitive what-you-see-is-what-you-feel visual exploration and annotation of volume data."
The following videos were uploaded by alucero:
Pass-Them-Around: Collaborative Use of Mobile Phones for Photo Sharing (CHI 2011)
"Pass-Them-Around is a phone-based application that allows a small group of collocated people to share photos using the metaphor of passing paper photos around. The prototype encourages people to share their devices and use them interchangeably while discussing photos face-to-face. The prototype supports ad-hoc photo sharing in different contexts by taking into account the spatial arrangement of users around a table, measured with sensors embedded in their mobile phones."
The next video was part of MobileHCI '10:
MindMap: Collaborative Use of Mobile Phones for Brainstorming
Showing posts sorted by date for query post-WIMP. Sort by relevance Show all posts
Showing posts sorted by date for query post-WIMP. Sort by relevance Show all posts
Jul 6, 2011
Revisiting CHI 2011: Videos of Interactive Touch, Gesture, Large Surface, and Mobile Apps with Potential for Use in Education (CHI = Computer Human Interaction)
Labels:
#CHI2011,
CHI 2011,
education,
gesture,
HCI,
interaction,
learning,
mobile,
multitouch,
NUI,
pedagogy,
post-WIMP,
teaching,
UI technology
No comments:
No comments:
Feb 26, 2011
Why bother switching from GUI to NUI? - Asked and Answered by Josh Blake; My 2-cents; Stevie B’s Microsoft Research Video; Marco Silva’s NUI-HCI Presentation (and links)
In Chapter 1 of Natural User Interfaces in .NET, Josh Blake asks and answers a question posed by many people who have been under the spell of keyboard input and GUI/ WIMP interaction:
Why bother switching from GUI to NUI? The answer? Read Chapter 1 (pdf) of the book - the chapter is free.
Here are a few of my personal reasons:
1. I want to buy the next version of the iPad or something like it.
2. I want to buy a new large-screen Internet HD TV.
Moms and dads with kids in tow.
People with special needs and/or health concerns, and the people who care and guide them.
Knowledge sharers and (life-long) learners....
RELATED
Why bother switching from GUI to NUI? The answer? Read Chapter 1 (pdf) of the book - the chapter is free.
Here are a few of my personal reasons:
1. I want to buy the next version of the iPad or something like it.
2. I want to buy a new large-screen Internet HD TV.
3. I want to buy a Kinect.
4. I do NOT want to interact with my new TV with a Sony remote. Too many tiny buttons!
5. I do NOT want to interact with my new TV with a keyboard, because it reminds me of...work.
6. Most importantly:
I want to design apps for the people I care about, and others with similar needs:
My mom. 
My grandson.
People with special needs and/or health concerns, and the people who care and guide them.
Knowledge sharers and (life-long) learners....
RELATED
"Smart" Interactive Display Research
View more presentations from Marco Silva
My YouTube Playlist:
"Natural user interfaces, gesture interaction, multi-touch, natural interaction, post WIMP examples and more... "
Interactive Multimedia Technology Post:
Interactive Touch-Screen Technology, Participatory Design, and "Getting It": Revisited
Interactive Touch-Screen Technology, Participatory Design, and "Getting It": Revisited
RELATED - and somewhat related
Encyclopedia: Human Computer Interaction, Interaction Design, User Experience, Information Architecture, Usability and More (Interaction-Design.org)
"Blake.NUI is a collection of helpful controls, utilities, and samples useful for multi-touch and NUI development with WPF, Surface, and Silverlight."
OpenKinect Community Founder Joshua Blake Comes Up With His Own Version of Win 7 Multitouch on Kinect (Madhav K, KinectHacks 1/14/11)
Kinect SDK Could Open New Worlds for Motion-Controlled Interfaces (Mike Martin, TechNews World, 2/22/11)
(This is not an inclusive list.)
GUI to NUI Post-WIMP Manifesto: TBA
GUI to NUI Post-WIMP Manifesto: TBA
Labels:
design,
HCI,
interaction,
Josh Blake,
kinect,
Marco Silva,
microsoft research,
NUI,
openkinect,
post Wimp,
programming,
seth sandler,
Stevie B,
usability
No comments:
No comments:
Feb 13, 2011
Wii Just Dance2 and Kinect Dance Central: UI and Usability Approaches; Challenges for Developing Accessible Games (revised)
I love to dance- I studied dance through college, and off and on as an adult. I have a DDR (Dance Dance Revolution) game-floor pad somewhere in my attic gathering dust. I'm ready for new challenges.
I'm planning on buying a couple new dance games for the Wii and the Kinect. There is more to this story, given my interest off-the-desktop, post-WIMP HCI (human-computer interaction), interactive multimedia and games, and a career as a school psychologist dedicated to young people with disabilities, I'm excited to see where new technologies, interfaces, and interactions will take us.
So what do the wise men of usability have to say about new ways of interacting with games and other applications?
"Kinect has many great design elements that clearly show that the team (a) knows usability, (b) did user testing, and (c) had management support to prioritize usability improvements, even when they required extra development work." -Jakob Nielsen
Jakob Nielsen, one of the godfathers of usability, shared a few words of wisdom about the Kinect in his 12/27/10 Alertbox post: Kinect Gestural UI: First Impressions. Although he did not review Dance Central, he concludes that the game he reviewed, Kinect Adventures, was fun to play, despite usability problems.
If this is a topic that interests you, I recommend you read Neilsen's post, and also take a look at which are outlined in the post. Also take a look at recent essay Neilsen co-authored with Don Norman, another godfather of usability: Gestural Interfaces: A Step Backwards In Usability
The screen shots below show how the movement icons are used in the game:
I thought it would be useful to learn more about the story behind the making of JustDance2.
At 2:22, Alexia, the project's usability expert, makes her presence known. From what I can tell, she focused on aspects of the game that would make it more usable for non-dancers, including those with "two left feet", to play the game. (I don't know if there was anyone consulted about accessibility concerns for the game.)
Kinect Dance Central
Dance Central uses a different approach when it comes to "teaching" people how to dance along through the game. It would be interesting to test out Dance Central and JustDance 2 with the same set of people to get a better feel for what works and what doesn't. Below is a video that previews, in split-screen, the interaction that takes place in Dance Central:
Dance Central Full Motion Preview
In Dance Central, gamers are provided with information about the moves through icons that cycle up the right hand side of the screen. The level of dance-coordination to keep up with the moves is challenging at times, even for people who are OK at dancing. Players can select dances according to level of difficulty.
Kinect Usability with Regular People
Below are screen shots that provide examples of how the movement icons are displayed in Dance Central:
RELATED
Just Dance 2 Review: Get your body moving. No, really. Give it a shot.
I'm planning on buying a couple new dance games for the Wii and the Kinect. There is more to this story, given my interest off-the-desktop, post-WIMP HCI (human-computer interaction), interactive multimedia and games, and a career as a school psychologist dedicated to young people with disabilities, I'm excited to see where new technologies, interfaces, and interactions will take us.
So what do the wise men of usability have to say about new ways of interacting with games and other applications?
"Kinect has many great design elements that clearly show that the team (a) knows usability, (b) did user testing, and (c) had management support to prioritize usability improvements, even when they required extra development work." -Jakob Nielsen
Jakob Nielsen, one of the godfathers of usability, shared a few words of wisdom about the Kinect in his 12/27/10 Alertbox post: Kinect Gestural UI: First Impressions. Although he did not review Dance Central, he concludes that the game he reviewed, Kinect Adventures, was fun to play, despite usability problems.
If this is a topic that interests you, I recommend you read Neilsen's post, and also take a look at which are outlined in the post. Also take a look at recent essay Neilsen co-authored with Don Norman, another godfather of usability: Gestural Interfaces: A Step Backwards In Usability
Why is this topic important to me?
I have been involved in the Games for Health and Game Accessibility movement for many years. Lately I've been exploring the OpenKinect project with an aim to create ways of making movement-oriented games accessible for young people with more complex disabilities. For example, there is a need to have dance and movement games modified for students (and adults!) who need wheelchairs or walkers. There are students who have milder mobility challenges who love to dance, and the current games don't address their needs. Some of my students have vision or hearing impairments, too. They deserve a chance to play things designed for the Kinect.
"OpenKinect is an open community of people interested in making use of the amazing Xbox Kinect hardware with our PCs and other devices. We are working on free, open source libraries that will enable the Kinect to be used with Windows, Linux, and Mac."
Note: I currently work as a school psychologist with students up to age 22. My main office is adjacent to a large OT and PT room at Wolfe, a program for students who have special needs. We just had a large interactive whiteboard installed in the room that is begging for us to connect it with the school's Wii, and soon (we hope), a Kinect. If we are going to use dance games to help promote healthy activities among our special students, the games need to be accessible for students with cognitive, motor, and other limitations.
FIRST STEPS
Although I can dance, I understand what the world is like through the eyes of many of the young people I work with who have motor coordination and sensory integration problems that interfere with their ability to move and dance, let alone access fast-paced dance games on the Wii or Kinect.
My initial plan is to look at what the new dance games might be like from the view of someone who doesn't know how to dance, and admits that they have "two left feet" - an perhaps, no sense of rhythm. Where would I start?
Wii's Just Dance2 seems to offer some support for learning how to dance through the use of simple movement icons, in the form of outlined figures, that provide information about how to move with the dancer on the screen. As you can see from the video below, the gamer is provided with information about upcoming moves throughout the game.
I decided to take a look at Just Dance2's MIKA "Big Girl" (You Are Beautiful) because some of the adolescent females I work with have weight concerns that interfere with their health. During the teen years, this can become a vicious cycle, resulting in less movement, and less participation with peers in physical activities, such as playing dance games. If a teen has depression as part of this mix, we know that exercise can help, and a fun dance game might be a life-saver, in more ways than one.
Although I can dance, I understand what the world is like through the eyes of many of the young people I work with who have motor coordination and sensory integration problems that interfere with their ability to move and dance, let alone access fast-paced dance games on the Wii or Kinect.
My initial plan is to look at what the new dance games might be like from the view of someone who doesn't know how to dance, and admits that they have "two left feet" - an perhaps, no sense of rhythm. Where would I start?
Wii's Just Dance2 seems to offer some support for learning how to dance through the use of simple movement icons, in the form of outlined figures, that provide information about how to move with the dancer on the screen. As you can see from the video below, the gamer is provided with information about upcoming moves throughout the game.
I decided to take a look at Just Dance2's MIKA "Big Girl" (You Are Beautiful) because some of the adolescent females I work with have weight concerns that interfere with their health. During the teen years, this can become a vicious cycle, resulting in less movement, and less participation with peers in physical activities, such as playing dance games. If a teen has depression as part of this mix, we know that exercise can help, and a fun dance game might be a life-saver, in more ways than one.
The screen shots below show how the movement icons are used in the game:
I thought it would be useful to learn more about the story behind the making of JustDance2.
At 2:22, Alexia, the project's usability expert, makes her presence known. From what I can tell, she focused on aspects of the game that would make it more usable for non-dancers, including those with "two left feet", to play the game. (I don't know if there was anyone consulted about accessibility concerns for the game.)
Kinect Dance Central
Dance Central uses a different approach when it comes to "teaching" people how to dance along through the game. It would be interesting to test out Dance Central and JustDance 2 with the same set of people to get a better feel for what works and what doesn't. Below is a video that previews, in split-screen, the interaction that takes place in Dance Central:
Dance Central Full Motion Preview
In Dance Central, gamers are provided with information about the moves through icons that cycle up the right hand side of the screen. The level of dance-coordination to keep up with the moves is challenging at times, even for people who are OK at dancing. Players can select dances according to level of difficulty.
Kinect Usability with Regular People
Steve Cable (CX Partmers) shared his team's look at usability issues related to the Kinect by testing several games, including Dance Central, with groups of people in his article, "Designing for XBox Kinect - a usability study". The quote below is from the Steve's article:
"We’ve loved playing with the Kinect. There’s no doubt that the game play is lots of fun. In-game menus are a barrier to that fun. Kinect should allow players to move through menus quickly and compensate for inaccuracy.
We felt the Kinect would benefit from some standardised global controls – much like a controller uses the A button to select and the B button to move backwards. We also think it needs a more responsive pause gesture – one that doesn’t interfere with the user’s game play.
Most of our participants found the Dance Central menu to be more effective, more efficient and more satisfying to use. Here are our recommendations for designing a Kinect menu interface:
- Allow users to make selections through positive gestures, rather than timed positions
- Place options on a single axis to make them easier and quicker to select
- Allow users to control menus with the game pad if they prefer
- Use large easy to read text
- Don’t make users scroll through options unnecessarily – it takes too long
- Users will be distracted if used in a social setting – test your menus in a social context to see if they are prone to errors
- Avoid the cursor metaphor, it’s not what gamers are used to seeing in game menus, and makes it harder to implement alternative joypad controls"
Below are screen shots that provide examples of how the movement icons are displayed in Dance Central:
RELATED
Just Dance 2 Review: Get your body moving. No, really. Give it a shot.
Kexa MacDonald, 10/19/10
Labels:
accessibility,
Dance Central,
Games for Health,
Just Dance2,
kinect,
NUI design,
openkinect,
quora,
review,
usability,
video,
Wii
No comments:
No comments:
Jan 28, 2011
"Microsoft is Imagining a NUI Future". You can, too!
Microsoft is Imagining a NUI Future
Steve Clayton, Next at Microsoft Blog, 1/26/11
"Our research shows that the vast majority of people polled in both developed and emerging markets see great potential for NUI applications beyond entertainment. This is especially true in China and India, where 9 out of 10 respondents indicate they are likely to use NUI technology across a range of lifestyle areas – from work, education and healthcare, to social connections, entertainment and the environment. We believe that taking technology to the next billion can be aided by NUI – making technology more accessible and more intuitive to a wider audience". - Steve Clayton, Microsoft
The people at Microsoft don't own the concept! I'm a member of the NUI Group (May, 2007) and SparkOn. Both are on-line communities where you can find people who live and breathe NUI, learn about their work, and even share designs and code. If you are intrigued by NUI - as a designer, developer, or user, please join us.
RELATED
Microsoft Plans a Natural Interface Future Full of Gestures, Touchscreens, and Haptics
Kit Eaton, Fast Company, 1/26/112
Rethinking Computing (video)
Craig Mundie, Microsoft
Interactive Touch-Screen Technology, Participatory Design, and "Getting It" - Revised
Touch Screen Interaction in Public Spaces: Room for Improvement, if "every surface is to be a computer".
Steve Clayton, Next at Microsoft Blog, 1/26/11
"Our research shows that the vast majority of people polled in both developed and emerging markets see great potential for NUI applications beyond entertainment. This is especially true in China and India, where 9 out of 10 respondents indicate they are likely to use NUI technology across a range of lifestyle areas – from work, education and healthcare, to social connections, entertainment and the environment. We believe that taking technology to the next billion can be aided by NUI – making technology more accessible and more intuitive to a wider audience". - Steve Clayton, Microsoft
The people at Microsoft don't own the concept! I'm a member of the NUI Group (May, 2007) and SparkOn. Both are on-line communities where you can find people who live and breathe NUI, learn about their work, and even share designs and code. If you are intrigued by NUI - as a designer, developer, or user, please join us.
Note:
I've been an evangelist and cheerleader for the NUI cause for many years. If you search this blog for "post-WIMP", "NUI", "multi-touch", "gesture", "off-the-desktop", "natural user interaction", "natural user interface", or even "DOOH", you'll be provided with an overwhelming number of posts that include videos, photographs, and links to NUI-related resources, including scholarly articles. There is a small-but-growing number of people from many disciplines, quietly working on NUI-related projects.
I've been an evangelist and cheerleader for the NUI cause for many years. If you search this blog for "post-WIMP", "NUI", "multi-touch", "gesture", "off-the-desktop", "natural user interaction", "natural user interface", or even "DOOH", you'll be provided with an overwhelming number of posts that include videos, photographs, and links to NUI-related resources, including scholarly articles. There is a small-but-growing number of people from many disciplines, quietly working on NUI-related projects.
Microsoft Plans a Natural Interface Future Full of Gestures, Touchscreens, and Haptics
Kit Eaton, Fast Company, 1/26/112
Rethinking Computing (video)
Craig Mundie, Microsoft
Interactive Touch-Screen Technology, Participatory Design, and "Getting It" - Revised
Touch Screen Interaction in Public Spaces: Room for Improvement, if "every surface is to be a computer".
Nov 11, 2010
Interactive Touch-Screen Technology, Participatory Design, and "Getting It", Revisited
I've been planning on updating one of my popular posts, "Interactive Touch-Screen Technology, Participatory Design, and Getting It" for a while.
Here is a compromise - since much still rings true two years since I wrote it, the bulk of this post remains the same. I've updated a few sections with additional video examples of interactive touch-screen applications, good and bad, along with a few links and resources, located at the bottom of the post.
(The missing piece of information? An update about apps for the iPad and similar touch-screen tablets.)
Sit back and enjoy!
-Images: HP; Wired
There's been some discussion over the reasons why so many people don't understand touch screen, or "surface" computing, even though research in this area has been going on for years.
As the new owner of the HP TouchSmart, I know that I get it.
The research I've conducted in this area suggests that people will "get-it" only if there is a strong commitment to develop touch-screen "surface" applications through a user-centered, participatory design process. In my view, this should incorporate principles of ethnography, and ensure that usability studies are conducted outside of the lab.
This approach was taken with Intel's Classmate PC. Intel has about 40 ethnographic researchers, and sent many of them to work with students and teachers in classrooms around the world. (A video regarding ethnographic research and the Intel Classmate project can be found near the end of this post.)
-Images: ClassematePC
Where to start?
K-12 classrooms and media centers. Public libraries. Malls. Hospital lobbies and doctor's offices. Any waiting room. Staff lounges in medical centers, schools, and universities. Community festivities and events. Movie theater lobbies. Museums and other points of interests.
I believe we need to take a "touching is believing" approach.
Here are some thoughts:
When I try to explain my fascination with developing touch-screen interactive multimedia applications, (interactive whiteboards, multi-touch displays and tables, and the like), many of my friends and family members eyes glaze over. This is particularly true for people I know who are forty-ish or over.
Even if you are younger, if you never saw the cool technology demonstrated in the movie Minority Report, or if you have limited experience with video games, or if you haven't came within touching distance of an interactive whiteboard, the concept might be difficult to understand.
The reality?
Even people who have the opportunity to use surface computing technology on large screens do not take full advantage of it. Multi-touch screens are often used as single-touch screens, and interactive whiteboards in classrooms are often serve as expensive projector screens for teacher-controlled PowerPoint presentations.
Most importantly, there are few software developers who understand the surface computing approach, even with the popularity of the iPhone and iPod Touch. Most focus on traditional business-oriented or marketing applications, and have difficulty envisioning scenarios for surface computing. There is a need for a breath of fresh air!
Another factor is that not all people entrusted to market surface or touch screen computing fully understand it.
Despite a cool website showing off the goods, Microsoft's Surface multi-touch table has been slow to take off, limiting hope of bringing down the price tag to a price most families or schools could afford. (The picture above depicts an application for the Surface designed for health care professionals, not K-12 science education.)
Although you can't buy a Surface table for your family room, it is possible to buy a TouchSmart. HP's TouchSmart website is engaging and highlights some examples of touch-screen interaction, but most people don't seem to know about it. (Since this post was last written, there are many more touch and multi-touch options available to the public, such as the Dell, the iPad, etc.)
Unfortunately, you wouldn't have a clue that the HP TouchSmart exists browsing the aisles at Circuit City or Best Buy!
When I was shopping for my new TouchSmart, I noticed that from a distance, the TouchSmart looked just like the other larger flat-screen monitors filling up the aisles. The salespeople at both stores were not well-informed about the system. The only reason I knew bout the new TouchSmart was related to my obsession with interactive multimedia touch-screen applications- designing them, developing them, studying them, reading about them, blogging about them.... ; }
More thoughts:
After studying HCI (Human-Computer Interaction), and relating this knowledge to what I know as a psychologist, my hunch is that the "Window Icon Mouse Pointing-device" (WIMP) and keyboard input mind-set is embedded in our brains, to a certain extent. Like driving a car, it is something automatic and expected. This is true for users AND developers. (Update - See The Post WIMP Explorers' Club: Update of the Updates for a review of a discussion among passionate post WIMP folks)
Think about it.
Suppose one day, you were told that you no longer were allowed to control your car by turning on the ignition, steering the wheel, or using your feet to accelerate, slow down, or stop the car! Instead, you needed to learn a new navigation, integration, and control system that involved waving your hands about and perhaps speaking a few commands.
For new drivers who'd never seen a car before, this new system would be user-friendly and intuitive. Perhaps it would be quite easy for 16-year-old kids to wrap their heads around this concept. For most of us, no. Imagine the disasters we would see on our streets and highways!
When we think about how newer technologies are introduced to people, we should keep this in mind.
In my mind, spreading the word about surface computing is not a "if you build it, they will come" phenomenon, like the iPhone. We can't ignore the broader picture.
From my middle-age woman's vantage point, I believe that it is important that the those involved with studying, developing, or marketing surface computing applications realize that many of us simply have no point of reference other than our experiences with ATMs, airline kiosks, supermarket self-serve lanes, and the like. (The video clip at the very end of this post provides a good example of touch-screen technology gone wrong.) -UPDATE: additional videos were added to this post.
Be aware that there are substantial numbers of people who might benefit from surface computing who prefer to avoid the ATMs, airline kiosks, and self-serve grocery shopping.
Realize that the collective experience with technology, in many cases, has not been too pretty. Many people have had such user-unfriendly experiences with productivity applications, forced upon them by their employers, that any interest or desire to explore emerging technologies has been zapped.
My own exposure to interactive "surface" related technology was somewhat accidental.
A few years ago, a huge box was deposited into the room I worked in a couple of days a week as a school psychologist at a middle school. After a week or so, I became curious, and found out that it was a SmartBoard. Until then (2002!), I did not know that interactive whiteboards existed.
The boxed remained unopened in the room for the entire school year, but no worry. I played with the only other SmartBoard in the school, and found a couple at the high school where I also worked. I hunted for all of the applications and interactive websites that I could find, and tried them out. That is when I was hooked. I could see all kinds of possibilities for interactive, engaging subject area learning activities. I could see the SmartBoards potential for music and art classes. With my own eyes, I saw how the SmartBoard engaged students with special needs in counseling activities. (By the way, if you are working with middle school students, PBS Kid's ItsMyLife website activities work great on an interactive whiteboard.)
A few years have passed, and reflecting on all of my fun experiences with interactive whiteboards, with and without students, I now understand that many teachers still have had limited exposure to this technology.
This school year, many teachers are finding themselves teaching in classrooms recently outfitted with interactive whiteboards, scrambling along with educational technology staff development specialists, to figure out how it works best with various groups of students, and what sort of changes need to be made regarding instructional practice.
For the very first time, interactive whiteboards were installed in two classrooms at one of the schools I work at. One of the teachers I know thanked me for telling her about interactive whiteboards and sharing my resources and links.
If I hadn't let her know about this technology, she wouldn't have volunteered to have one installed in her classroom. It has transformed the way she teaches special needs students.
In the few months that she's used the whiteboard, I can see how much it has transformed the way the students learn. They are attentive, more communicative, and engaged. The students don't spend the whole day with the whiteboard - the interactive learning activities are woven into lessons at various times of the day, representing true technology integration.
Now let's see what happens when all-in-one touch-screen PC's are unleashed in our schools!
UPDATE: Take a look at a post I wrote for Innovative Interactivity just after SMART Technologies acquired NextWindow - the post describe in detail how interactive whiteboards are transforming learning and teaching in a program for students with special needs.
SMARTTechnologies Acquires NextWindow: A "smart window" to the world
There are some interesting changes going on at the intersection of HCI and educational technology research. I participated in a workshop at CHI 2010 last April and was impressed by what is going on in this area, around the globe: Next Generation of HCI and Education
Value of ethnographic research:
Ethnographic Research Informed Intel's Classmate PC
"Intel looked closely at how students collaborate and move around in classroom environments. The new tablet feature was implemented so that the device would be more conducive to what Intel calls “micromobility”. Intel wants students to be able to carry around Classmate PCs in much the same way that they currently carry around paper and pencil." -via Putting People First and Ars Technica
The video below is from Intel's YouTube Channel. Information about Intel's approach to ethnographic research in classrooms during the development of the Classroom PC is highlighted. This approach uses participatory design and allows the set of applications developed for the Classmate PC to reflect the needs of local students and teachers. Schools from many different countries were included in this study.
FYI: TOUCH SCREEN DISPLAYS: NEED FOR IMPROVEMENT!
Touch Screen Coke Machine at the Mall: 90 Seconds to get a Coke
User-Unfriendly Interactive Display in the Rain (Ballantyne Village Shopping Center)
User-Unfriendly Information Kiosk Interactive Map
I encountered this puzzling and frustrating interactive directory/map at the Cleveland Clinic. When I went to visit a relative at the hospital a year or so later, the map was no longer there.
BETTER EXAMPLES OF INTERACTIVE SCREENS:
Samsung's new Omnia SDG i900 was re-created in a much larger size, using lm3lab's Ubiq'window touchless technology.For more about lm3labs, including several videoclips, take a look at one of my previous posts: Lm3Labs, Nicolas Leoillot, and Multimedia Interaction
Midwife Toad App on a Microsoft Surface, Discovery Place Science Center
TellTable: Digital Storytelling on the Surface: Microsoft Research, UK
DECEMBER 31, 2009 -Interactive Soda Machine for Fun
The interactive screen on the Coke machine attracted the attention of this young child. He loved spinning the image of the bottle. So did the dad! He said, "I'd like something like this for my home!". I told him about the HP TouchSmart - both the dad and the mom did not know that there were affordable all-in-one touch screens available, but they knew about SMARTboards, because their children's classrooms had them. Note: No one from this family actually purchased a soft drink. I was hoping to time how long it would take them to do so!
If you have plenty of time, take a look at my Post WIMP Explorers' Club YouTube playlist.
"Natural user interfaces, gesture interaction, multi-touch, natural interaction, post WIMP examples and more..."
Here is a compromise - since much still rings true two years since I wrote it, the bulk of this post remains the same. I've updated a few sections with additional video examples of interactive touch-screen applications, good and bad, along with a few links and resources, located at the bottom of the post.
(The missing piece of information? An update about apps for the iPad and similar touch-screen tablets.)
Sit back and enjoy!
-Images: HP; Wired
There's been some discussion over the reasons why so many people don't understand touch screen, or "surface" computing, even though research in this area has been going on for years.
As the new owner of the HP TouchSmart, I know that I get it.
The research I've conducted in this area suggests that people will "get-it" only if there is a strong commitment to develop touch-screen "surface" applications through a user-centered, participatory design process. In my view, this should incorporate principles of ethnography, and ensure that usability studies are conducted outside of the lab.
This approach was taken with Intel's Classmate PC. Intel has about 40 ethnographic researchers, and sent many of them to work with students and teachers in classrooms around the world. (A video regarding ethnographic research and the Intel Classmate project can be found near the end of this post.)
-Images: ClassematePC
Where to start?
K-12 classrooms and media centers. Public libraries. Malls. Hospital lobbies and doctor's offices. Any waiting room. Staff lounges in medical centers, schools, and universities. Community festivities and events. Movie theater lobbies. Museums and other points of interests.
I believe we need to take a "touching is believing" approach.
Here are some thoughts:
When I try to explain my fascination with developing touch-screen interactive multimedia applications, (interactive whiteboards, multi-touch displays and tables, and the like), many of my friends and family members eyes glaze over. This is particularly true for people I know who are forty-ish or over.
Even if you are younger, if you never saw the cool technology demonstrated in the movie Minority Report, or if you have limited experience with video games, or if you haven't came within touching distance of an interactive whiteboard, the concept might be difficult to understand.
The reality?
Even people who have the opportunity to use surface computing technology on large screens do not take full advantage of it. Multi-touch screens are often used as single-touch screens, and interactive whiteboards in classrooms are often serve as expensive projector screens for teacher-controlled PowerPoint presentations.
Most importantly, there are few software developers who understand the surface computing approach, even with the popularity of the iPhone and iPod Touch. Most focus on traditional business-oriented or marketing applications, and have difficulty envisioning scenarios for surface computing. There is a need for a breath of fresh air!
Another factor is that not all people entrusted to market surface or touch screen computing fully understand it.
Despite a cool website showing off the goods, Microsoft's Surface multi-touch table has been slow to take off, limiting hope of bringing down the price tag to a price most families or schools could afford. (The picture above depicts an application for the Surface designed for health care professionals, not K-12 science education.)
Although you can't buy a Surface table for your family room, it is possible to buy a TouchSmart. HP's TouchSmart website is engaging and highlights some examples of touch-screen interaction, but most people don't seem to know about it. (Since this post was last written, there are many more touch and multi-touch options available to the public, such as the Dell, the iPad, etc.)
Unfortunately, you wouldn't have a clue that the HP TouchSmart exists browsing the aisles at Circuit City or Best Buy!
When I was shopping for my new TouchSmart, I noticed that from a distance, the TouchSmart looked just like the other larger flat-screen monitors filling up the aisles. The salespeople at both stores were not well-informed about the system. The only reason I knew bout the new TouchSmart was related to my obsession with interactive multimedia touch-screen applications- designing them, developing them, studying them, reading about them, blogging about them.... ; }
More thoughts:
After studying HCI (Human-Computer Interaction), and relating this knowledge to what I know as a psychologist, my hunch is that the "Window Icon Mouse Pointing-device" (WIMP) and keyboard input mind-set is embedded in our brains, to a certain extent. Like driving a car, it is something automatic and expected. This is true for users AND developers. (Update - See The Post WIMP Explorers' Club: Update of the Updates for a review of a discussion among passionate post WIMP folks)
Think about it.
Suppose one day, you were told that you no longer were allowed to control your car by turning on the ignition, steering the wheel, or using your feet to accelerate, slow down, or stop the car! Instead, you needed to learn a new navigation, integration, and control system that involved waving your hands about and perhaps speaking a few commands.
For new drivers who'd never seen a car before, this new system would be user-friendly and intuitive. Perhaps it would be quite easy for 16-year-old kids to wrap their heads around this concept. For most of us, no. Imagine the disasters we would see on our streets and highways!
When we think about how newer technologies are introduced to people, we should keep this in mind.
In my mind, spreading the word about surface computing is not a "if you build it, they will come" phenomenon, like the iPhone. We can't ignore the broader picture.
From my middle-age woman's vantage point, I believe that it is important that the those involved with studying, developing, or marketing surface computing applications realize that many of us simply have no point of reference other than our experiences with ATMs, airline kiosks, supermarket self-serve lanes, and the like. (The video clip at the very end of this post provides a good example of touch-screen technology gone wrong.) -UPDATE: additional videos were added to this post.
Be aware that there are substantial numbers of people who might benefit from surface computing who prefer to avoid the ATMs, airline kiosks, and self-serve grocery shopping.
Realize that the collective experience with technology, in many cases, has not been too pretty. Many people have had such user-unfriendly experiences with productivity applications, forced upon them by their employers, that any interest or desire to explore emerging technologies has been zapped.
My own exposure to interactive "surface" related technology was somewhat accidental.
A few years ago, a huge box was deposited into the room I worked in a couple of days a week as a school psychologist at a middle school. After a week or so, I became curious, and found out that it was a SmartBoard. Until then (2002!), I did not know that interactive whiteboards existed.
The boxed remained unopened in the room for the entire school year, but no worry. I played with the only other SmartBoard in the school, and found a couple at the high school where I also worked. I hunted for all of the applications and interactive websites that I could find, and tried them out. That is when I was hooked. I could see all kinds of possibilities for interactive, engaging subject area learning activities. I could see the SmartBoards potential for music and art classes. With my own eyes, I saw how the SmartBoard engaged students with special needs in counseling activities. (By the way, if you are working with middle school students, PBS Kid's ItsMyLife website activities work great on an interactive whiteboard.)
A few years have passed, and reflecting on all of my fun experiences with interactive whiteboards, with and without students, I now understand that many teachers still have had limited exposure to this technology.
This school year, many teachers are finding themselves teaching in classrooms recently outfitted with interactive whiteboards, scrambling along with educational technology staff development specialists, to figure out how it works best with various groups of students, and what sort of changes need to be made regarding instructional practice.
For the very first time, interactive whiteboards were installed in two classrooms at one of the schools I work at. One of the teachers I know thanked me for telling her about interactive whiteboards and sharing my resources and links.
If I hadn't let her know about this technology, she wouldn't have volunteered to have one installed in her classroom. It has transformed the way she teaches special needs students.
In the few months that she's used the whiteboard, I can see how much it has transformed the way the students learn. They are attentive, more communicative, and engaged. The students don't spend the whole day with the whiteboard - the interactive learning activities are woven into lessons at various times of the day, representing true technology integration.
Now let's see what happens when all-in-one touch-screen PC's are unleashed in our schools!
UPDATE: Take a look at a post I wrote for Innovative Interactivity just after SMART Technologies acquired NextWindow - the post describe in detail how interactive whiteboards are transforming learning and teaching in a program for students with special needs.
SMARTTechnologies Acquires NextWindow: A "smart window" to the world
There are some interesting changes going on at the intersection of HCI and educational technology research. I participated in a workshop at CHI 2010 last April and was impressed by what is going on in this area, around the globe: Next Generation of HCI and Education
Value of ethnographic research:
Ethnographic Research Informed Intel's Classmate PC
"Intel looked closely at how students collaborate and move around in classroom environments. The new tablet feature was implemented so that the device would be more conducive to what Intel calls “micromobility”. Intel wants students to be able to carry around Classmate PCs in much the same way that they currently carry around paper and pencil." -via Putting People First and Ars Technica
The video below is from Intel's YouTube Channel. Information about Intel's approach to ethnographic research in classrooms during the development of the Classroom PC is highlighted. This approach uses participatory design and allows the set of applications developed for the Classmate PC to reflect the needs of local students and teachers. Schools from many different countries were included in this study.
FYI: TOUCH SCREEN DISPLAYS: NEED FOR IMPROVEMENT!
Touch Screen Coke Machine at the Mall: 90 Seconds to get a Coke
User-Unfriendly Interactive Display in the Rain (Ballantyne Village Shopping Center)
User-Unfriendly Information Kiosk Interactive Map
BETTER EXAMPLES OF INTERACTIVE SCREENS:
Here are some interesting pictures from lm3labs, which are in my interactive usability hall of fame:
Samsung's new Omnia SDG i900 was re-created in a much larger size, using lm3lab's Ubiq'window touchless technology.For more about lm3labs, including several videoclips, take a look at one of my previous posts: Lm3Labs, Nicolas Leoillot, and Multimedia Interaction
Midwife Toad App on a Microsoft Surface, Discovery Place Science Center
TellTable: Digital Storytelling on the Surface: Microsoft Research, UK
DECEMBER 31, 2009 -Interactive Soda Machine for Fun
The interactive screen on the Coke machine attracted the attention of this young child. He loved spinning the image of the bottle. So did the dad! He said, "I'd like something like this for my home!". I told him about the HP TouchSmart - both the dad and the mom did not know that there were affordable all-in-one touch screens available, but they knew about SMARTboards, because their children's classrooms had them. Note: No one from this family actually purchased a soft drink. I was hoping to time how long it would take them to do so!
Some resources:
Multimedia, Multitouch, Gesture, and Interaction Resources (Mega-list, work in progress)
lm3labs (catchyoo, ubiq'window)
Sparkon (See members links and multi-touch projects)
(More information and resources can be found by doing a "multitouch" or related search on this blog or The World Is My Interactive Interface.)
"Natural user interfaces, gesture interaction, multi-touch, natural interaction, post WIMP examples and more..."
FYI: I visited the Ballantyne Village shopping center a couple of months ago to follow up on the interactive displays, including the one I tried to use while it was raining. The shopping center changed ownership, and the displays were replaced by the old-fashioned kind, pictured below:
Oct 30, 2010
Philipp Geist: Blending the Physical with the Digital; Google TV/Leanback, Vimeo's new Couch Mode, oh...and ViewSonic's 3D (glasses-less) pocket camcorder...
I'm thinking about getting one of the new "internet ready" TVs. I have a serious reason to do this. I'm working on some interactive video projects, and a couple of my projects are geared for teens and young adults who have autism.* My hunch is that many of my students would like to watch- and interact with-content optimized for Google TV and Vimeo's Couch Mode. The content is designed to look good on larger high-resolution flat-screen displays, and I'm sure it would be great on my school's newer SMARTBoards. I need to learn more about developing applications for this purpose.
"With Google Chrome and Adobe Flash Player 10.1, Google TV lets you access everything on the web. Watch your favorite web videos, view photos, play games, check fantasy scores, chat with friends, and do everything else you're accustomed to doing online. Plus, the world's best websites are now being perfected for television -- check out our Spotlight gallery for examples." "The worlds' favorite websites are being tweaked and perfected for the television." -Google TV
Google Leanback
When I visited the Leanback website, I encountered the following screen with a suggestion that I type in what I was looking for. I typed in "lynnvm", the name of my YouTube channel. Apparently Google provides you with a randomly generated featured video that appears in the background that has nothing to do with what you are looking for.
In this screen shot, my YouTube channel offerings are in the foreground. "Maleficent Halloween Tutorial" is what played in the background:
VIMEO INTRODUCES COUCH MODE: Vimeo's version of Google's Leanback is Couch Mode. It is optimized for use on Google TV, so that makes things less complicated in the world of videoviewingland. According to Ryan Hefner's article on the Vimeo staff blog, "Couch Mode is a special new section of Vimeo that allows you to watch collections of videos (such as Staff Picks, your inbox, your videos, etc.) completely uninterrupted like a TV channel."
Couch Mode works on computers, but since it relies on HTML5 and CSS3, without Flash, it only works with Chrome and Safari browsers. For more information, see the video below:
RELATED
Comment: The idea of developing interactive multimedia apps in 3D intrigues me. At this point, the technology is too new for an "armchair technologist" like me to pursue with my incredibly busy work obligations. I don't have the money to buy a 3D video camera. But I might try this out, if it is true that it only costs $238.00!
Viewsonic introduces 3Dv5 3D pocket camcorder, no glasses required
Darren Murph, Engadget, 10/20/10
Film Videos In 3D for Under $250 With Viewsonic's 3DV5
HotHardware, 10/28/10
* About me:
I presently work full time as a school psychologist at a high school and at a program for students with more severe disabilities, including autism. The students I work with have made amazing gains through the use of interactive multimedia applications, and also have responded well to video presented on the large IWB screens.
I went back to school to take computer courses, initially so I could make interactive multimedia applications and games. I continue to blog about interactive multimedia, emerging/ new technologies, and topics related to post-WIMP HCI/UX/ID/IA. Although my "spare time" is limited, I try to keep up my technical skills whenever I can by working on projects that can support the students I work with.
(Currently I use my HP 22-inch TouchSmart PC to view web-based video content, and to evaluate websites that provide "touchable" and interactive content that might work well on interactive whiteboards.)
At any rate, I've been looking for great videos that have the potential for use at work with older students who have autism. I'm also looking for effective ways that the students can use to interact with multimedia and video content. This is important, since the students have minimal verbal communication skills, have limited reading ability- if they can read at all. They learn about their world through visual means, and are capable of learning much more - but not through traditional means.
Since our school is focusing on globalization and learning about the cultures of other countries, I've been on the lookout for some interesting videos that might appeal to our students.
Today I came across a great find- Philipp Geist. Who is Philipp Geist? According to his bio, Philipp works internationally as a light and multi-media artist in the mediums of video, performance, photography and painting. Some of his work focuses on architecture, history, and cultural heritage. A good example of his work is the installation he created for a festival in Thailand in 2009:
Bangkok / Thailand 4D Light & Sound Installation by Philipp Geist (23Min) from Philipp Geist | Videogeist on Vimeo.
"The one-hour show is the central part of the celebrations and will be seen by thousands of visitors. It interprets artistically the king's life and his work dedicated to public welfare. The art installation combines images of the kings and his social projects in the past and present with 3D animations of Thai natural and cultural heritage and abstract painterly passages." (from the Vimeo site)
Phillip Geist's Showreel
HIGH-RES MULTIMEDIA WEB CONTENT ON LARGE PANEL HD TV!
This might boost holiday gift sales and in turn, give a little jolt to the economy. To do my duty for my country, I will continue to research Internet TV as I narrow down my selection for my new Internet-ready TV.... Below is some Information about Google TV, Google Leanback, and Vimeo's Couch mode that I've recently gathered to share with my IMT followers:
GOOGLE TV: "The web is now a channel"
I'm not too excited about the design of the application that transforms your Android phone or iPhone into a remote control. I hate most remote controls. According to Google TV, multiple phones can control the same TV, and you can use your voice to search, which seems like it would be a good thing... I wonder if they tested this out with real families, not just families of Google TV techies .
GOOGLE LEANBACK Video (Integrated into GoogleTV)
Google Leanback
When I visited the Leanback website, I encountered the following screen with a suggestion that I type in what I was looking for. I typed in "lynnvm", the name of my YouTube channel. Apparently Google provides you with a randomly generated featured video that appears in the background that has nothing to do with what you are looking for.
In this screen shot, my YouTube channel offerings are in the foreground. "Maleficent Halloween Tutorial" is what played in the background:
VIMEO INTRODUCES COUCH MODE: Vimeo's version of Google's Leanback is Couch Mode. It is optimized for use on Google TV, so that makes things less complicated in the world of videoviewingland. According to Ryan Hefner's article on the Vimeo staff blog, "Couch Mode is a special new section of Vimeo that allows you to watch collections of videos (such as Staff Picks, your inbox, your videos, etc.) completely uninterrupted like a TV channel."
Couch Mode works on computers, but since it relies on HTML5 and CSS3, without Flash, it only works with Chrome and Safari browsers. For more information, see the video below:
RELATED
"A few of our favorites include Net-A-Porter, which lets you watch runway videos and shop for high fashion; Meegenius, a place where you can read and customize children’s books; TuneIn, a personal radio for your TV; and The Onion which always gives us a good laugh." - Google TV Blog
MeeGenius If you are a teacher, parent, kid, or lover of children's books, visit this interactive website ASAP. It is optimized for Google TV and works nicely on touch-enabled screens and devices.
As I was wrapping up this post, I came across information about Viewsonic's new 3D, glassesless pocket camcorder. I'll update information about this new gadget when I have a chance to learn more about it. !
Viewsonic introduces 3Dv5 3D pocket camcorder, no glasses required
Darren Murph, Engadget, 10/20/10
Film Videos In 3D for Under $250 With Viewsonic's 3DV5
HotHardware, 10/28/10
* About me:
I presently work full time as a school psychologist at a high school and at a program for students with more severe disabilities, including autism. The students I work with have made amazing gains through the use of interactive multimedia applications, and also have responded well to video presented on the large IWB screens.
I went back to school to take computer courses, initially so I could make interactive multimedia applications and games. I continue to blog about interactive multimedia, emerging/ new technologies, and topics related to post-WIMP HCI/UX/ID/IA. Although my "spare time" is limited, I try to keep up my technical skills whenever I can by working on projects that can support the students I work with.
Labels:
3D content,
3Dv5,
autism,
autostereoscopic,
couch mode,
google TV,
HCI,
interactive,
IWB,
multimedia,
NUI,
philipp geist,
postWIMP,
smartboard,
UX,
video,
Viewsonic
No comments:
No comments:
Subscribe to:
Posts (Atom)