Designing for Touch & Gesture: Tips for Apps and the Web (Updated)

In the past, our fingers did the walking, sifting through files, papers, pamphlets, and phonebooks, and then by point-click-clicking with a mouse to interact with images and text, in essence, electronic imitations of the paper-based world. Traditional forms, brochures, ad inserts, and posters informed much of the design. 

How much have things change?   It is 2013, but you'd think it was 1997 from the PowerPoint look and feel of many apps and web sites!   Touch is everywhere, but from what I can tell, not enough designers and developers have stepped up to the plate to think more deeply about ways their applications can support human endeavors though touch and gesture interactions.  

For an overview of this topic, take a look at my 2011 post, written after a number of ugly encounters with user-unfriendly applications:  Why bother switching from GUI to NUI?  

For an in-depth look into the history of multi-touch, the wisdom of Bill Buxton is well-worth absorbing.  He's worked with all sorts of interfaces, and has been curating the history of multi-touch and gesture systems since 2007:


Multi-Touch Systems that I have Known and Loved
Bill Buxton, Microsoft Research, Updated 8/30/12



Even if you are not a designer or developer, I encourage you to explore some of the links below:

Touch Gestures for Application Design
Luke Wroblewski, 10/9/12

Common Misconceptions About Touch
Steven Hoober, 3/18/13

Designing With Tablets in Mind:  Six Tips to Remember
Connor Turnbull, Webdesign tuts+, 9/27/11

Finger-Friendly Design: IDeal Mobile Touchscreen Target Sizes
Anthony T, Smashing Magazine, 2/21/12

Best Practices: Designing Touch Tablet Experiences for Preschoolers (pdf)
Sesame Street Workshop


Are Touch Screens Accessible?
AcessIT, National center on Accessible Information Technology in Education

iOS Human Interface Guidelines
Apple

Android User Interface Guidelines
Using Touch Gestures
Handling Multi-Touch Gestures
Android

Designing for Tablets?  We're Here to Help!
Roman Nurik, Android Developers Blog 11/26/12

Touch interaction design (Windows Store apps)
Microsoft - MSDN

Multi-Touch Systems that I have Known and Loved
Bill Buxton, Microsoft Research, Updated 8/30/12

Sonic Wire Sculptor and Rhonda, a 3D drawing tool

Amit Pitaru and James Paterson, creators of Numby, an app for kids to teach counting skills, have been involved in a number of interesting projects.  Here are two of them:

Sonic Wire Sculpture


The app creates music in 3D, so it is best to use headphones.


Sonic Wire Sculpture is available as an iPhone app.  More information can be found on the Sonic Wire Sculptor website.


Rhonda Forever is a 3D drawing tool that was born around 2003 and has a team of developers who are working to create stable releases of the application so it can be accessed through different platforms.  There is a newer beta version of Rhonda for PC and Mac that can be downloaded from the Rhonda website. The team is working on releases that will support 2-hand interaction.


I'm hoping that Rhonda Forever will be available for the iPad.




RELATED
Everybody Loves Sketch (an user-friendly 3D drawing application that is designed for non-artists, tested with high school students.)
ILoveSketch website

Jennifer Pahlka's TED talk: "Coding a Better Government" - how Code for America fellows and others are harnessing social media, mobile apps, and the web.

FYI: Jennifer Pahlka will be a keynote speaker on Tuesday, March 13 at the SXSW (South by Southwest) conference in Austin, Texas. SXSW events and interactive keynotes will be streaming live at http://sxsw.com/interactive/live 

In the above video, Jennifer Pahlka, the founder of Code for America, discusses how "coding" citizens- app developers, web developers, designers, and others are helping government work more efficiently and effectively, drawing on the strength of the internet, mobile web, and social media.  http://www.ted.com/talks/view/lang///id/1381


Participatory citizenship through coding!


RELATED
Code for America Fellowship Program
The Code for America Fellows work in teams for 11 months, during which they receive mentoring, education about municipal government, and the opportunity to develop a web application to help cities - and citizens- solve problems.  The 2012 Fellows are based in San Francisco, and are provided a $35,000 stipend, travel expenses, and healthcare. Although based in San Francisco, the Fellows are assigned to a host city (Boston, Washington D.C., Philadelphia, or Seatle), and spend some of their time on-site.
Code for America Blog

South by Southwest - SXSW -Austin, Texas
"The South by Southwest Conference & Festivals offer the unique convergence of original music, independent films, and emerging technologies. Fostering creative and professional growth alike, SXSW is the premier destination for discovery" -SXSW 


SXSW YouTube Channel

iBooks Textbooks for iPads, iTunesU App -and links to info about 1:1 iPad initiatives in public schools

There is a lot of buzz about iPads in education. 


I'm convinced that tablet technology will be adopted by schools at a faster rate than anticipated a year or so ago, now that Apple has launched iBooks Textbooks for iPad and the iTunesU app.


Parents, teachers, students, administrators, and school IT specialists, hold on to your hats!

RELATED
Learning Together: The Evolution of a 1:1 iPad Program
Carolyn Foote, Internet@Schools, 1/1/12
The Swiss-Army Knife of Education
Mr. Hooker, The WIFI--Eanes ISD iPad Pilot Project, 10/3/11
iPads in Education
John Evans, Curator
iBooks Textbooks
iTunesU App

iPad for Art: Sketchbook Express, via the Montlieu Academy of Technology, a public K-5 magnet school that adopted a 1:1 iPad initiative in August, 2011.


COMMENT
I have been planning on devoting more posts in the future about the iPad and related interactive technologies for learning- and for fun.  When I have a chance to write my mega-post, you won't be disappointed!

iPads for every student in an elementary school in NC, Smartboards in the classrooms, and one awesome principal

 

The effects of technology at this Title I school resulted in increased attendance, fewer discipline problems, and  college aspirations for the students.   I would like to visit this school soon and see how the teachers integrate iPads and Smartboards.


The story about the principal of this school is inspiring.


RELATED
Buckley Report: Immigrant Principal
Montlieu Academy of Technology

GOOGLE CHROME WEB STORE: All sorts of apps, educational and more!

 I'll have to admit, I no longer consider myself an early adopter.  I put off exploring the Google Chrome Web Store until.... today.  Most of my recent explorations have centered on discovering great apps for my iDevices, but since I'm a cross-platform, cross-device, transmedia type of person, I could not neglect the world of interactive web apps!


Here is a screen shot of the educational web apps from the Google Chrome Web Store:


























After you select a few web apps, you can access them from your Chrome browser by selecting a new tab or new window.  The screen shot below shows the apps I've currently chosen to explore:

I have an HP TouchSmart, and I found that many of the web app worked out OK with touch interaction, even 3D  "rotate and pan".  My hunch is that many of the web apps will work fine on an interactive whiteboard. 


I took a quick look at the MeeGenius! Children's Books interactive web app and was pleasantly surprised.  This web app is a talking book that highlights each word as it is spoken. A human voice is used for the narration.

I'm looking forward to getting into work early on Monday to see what web apps work well on the SMARTboard!

RELATED and SOMEWHAT RELATED
Free Education Apps from Google's New Chrome Web Store
The Yellin Center for Mind, Brain, and Education, 2/9/11
Below is a list of educational Chrome apps recommended in the above linked post:

• Planetarium –an interactive star map for kids (or adults!) interested in astronomy
• Google Books – millions of free e-books in a plethora of subjects
• 3DTin – A realistic 3D model-maker
• MathBoard – a math learning tool appropriate for elementary and kindergarten students
• MeeGenuis! Children’s Books – personalized, “enhanced” web books for younger students
• LucidChart – a collaborative diagramming tool, not unlike Inspiration or MindManager
• Picnik Photo Editor – web-based photo manipulation a la Photoshop
• 20 Things I Learned about Browsers & the Web – a great introduction to how the internet works
• Springpad – note-taking tool that can incorporate assignments, photos, to-do lists, etc.
• Bomomo – an innovative illustration and drawing tool
• A variety of flashcard-based apps which can aid memory, vocabulary, math skills and more

Plinks and Tonecraft - two music tools for Chrome
Danny Nicholson, The Whiteboard Blog  10/11/11
Dinahmoe Labs  (creators of Plink and Tonecraft)
The Whiteboard Blog

Update, plus iGaze app by Dunedin Multimedia for use during social skills group activities

So what am I up to now?  


I'd like to share with my readers that I've decided to continue in my present position as a school psychologist, while still devoting a portion of my free time to technology. From time-to-time I think deep thoughts about usability, accessibility,  and UX/Interaction related to off-the-desktop interactive multimedia applications running on screens of all sizes.  I'm hoping to create a few multimedia experiments using HTML5 and JavaScript, and explore jQuery if and when I can find the time!  


For the present school year, my main school is a program for students with more significant disabilities, including autism spectrum disorders.  My second school is a magnet high school for technology and the arts,  located on the same campus.  I also consult throughout the district on cases involving students who have suffered traumatic brain injuries, as well as students who have multiple disabilities.  I am thankful that I have a job in a school district that values 21st Century technology.  


I'm looking forward to another technology-rich school year.  I've spent some of the time I usually devote to blogging devoted exploring iPad apps instead.  Since I'm new to the world of iPads, I'm still in discovery mode.  What an adventure!   


There are plenty of educational apps out there, and many of them are suitable for students with special needs.  On the other hand, there is much room for improvement - across all iPad app categories.  Since there is very little research about what makes up a killer app- or suite of apps- for students with special needs, experimenting with  iPad apps is uncharted territory. 

I made the decision to bring my personal iPad2 to work after I discovered a number of apps that I thought would be useful in my work as a school psychologist with students who have special needs, including autism spectrum disorders.  

One of my intervention themes this year focuses on social skills.   This is especially important for students who participate in our schools community-based job training program.  I'm using some content from Unique Learning's transition materials,  as well as on-line activities from Do2Learn's JobTips website, because my aim is to facilitate social skills that will be useful in a variety of job and community settings.   


Although my main technology tool for working with groups is the SMARTBoard,  I've found that using a combination of interactive whiteboard and iPad activities to be especially effective.  I'm paving the way for more role-play activities in the future, and attempting to use technology to my advantage.


This past week, I used the iGaze app, created by Dunedin Multimedia, to help a group of high-school level students practice establishing and maintaining eye gaze, something that is difficult for most of them to demonstrate "in-person".  I was amazed.  Each student was excited to take his or her turn.  Even more amazing?  When each student took a turn, the other students looked at their eyes and faces.  No one rocked or "stimmed".  No one made noises.  I observed several instances of joint attention, much to my delight.   

Below is a video from Dunedin Multimedia's YouTube channel that is similar to what the students viewed during their group activity:

Here is some information from Dunedin Multimedia about the iGaze app:

"Eye contact is important to communication and social development, and yet the impaired ability to make and maintain eye contact is one of the most striking aspects of autism. iGaze is an eye contact simulator that can help to build confidence in using this important means of nonverbal social communication.  The app also contains information on eye contact and eye gaze, with links to relevant research."

During the social skills activity involving the iGaze app,  I used the SMART Board to display a large picture of a boss and a worker standing face to face, making eye contact, engaged in conversation.  The picture served as an anchor to remind the students of pictures and videos they'd previously viewed that illustrated the concept of face-to-face interaction and the importance of establishing eye-contact with others from time-to-time.


I'm hoping I will be able to access the YouTube videos from Dunedin so I can use them on the SMART Board. It will be interesting to see how this plays out!   I'm also planning to take a closer look at Dunedin Multimedia's emotion x app for the iPad.


RELATED 
Screen-shot of iGaze for the iPad Dunedin:

SOMEWHAT RELATED
The SMART Table at my school was updated today - I'm looking forward to using it for some group activities, now that it is back in working order and has new applications loaded up and ready to go!


If you are interested in learning more about technology related to students with special needs, be sure to check out Kate Ahern's blog, Teaching Learners with Multiple Special Needs

Kate's post about the features of Unique Learning Systems.


Upcoming:  more about tablets, interview with folks from Stantum, social-skills game-in-progress.....large displays in public spaces update....

NodeBeat HD 1.5 Upgrade, an iOS Music Sequencer: Fun to Play on my iPad!

NodeBeat HD 1.5 - iOS Music Sequencer from AffinityBlue on Vimeo.

Below is information about NodeBeat 1.5 from the Vimeo website:


"This is a brief video showcasing some of the new features in NodeBeat HD 1.5. The same features will be coming to the iPhone/iPod Touch version of NodeBeat in the coming weeks."


"New Features Include:


- Entirely new user interface design
- Drag and drop new nodes
- Drum Generator Node
- Sine, Triangle, Sawtooth, and Square waveform selection
- Tempo and BPM controls
- Compress recording for faster e-mailing of recordings
- Create ringtones from recordings
- Sleep Timer. Now you can fall asleep to NodeBeat
- Shake to Clear Screen
- 5 Finger multitouch reset all"
----------------------------------------------
"NodeBeat is an experimental node-based audio sequencer and generative music application for the iPhone, iPod Touch and iPad."


(Just 99 cents!)


RELATED
NodeBeat website


NodeBeat iTunes Preview

Play and Experiment with Music on your iPad/Pod/Phone with NodeBeat iOS Music Sequencer by Affinity Blue by Seth Sandler and Justin Windle  (previous post)

NodeBeat: Create and View Musical Soundscapes
iPad Creative, 5/31/11

Seth Sandler     Justin Windle

Baby playing NodeBeat 1.0 -using fingers and toes:

Apple's iOS 5 facial recognition feature opens up interactive possibilities

I've been thinking about creating my first iPad app, and as I was searching for information, I came across a few articles related to Apple's new iOS 5 that I found interesting.  


Because my target user group includes young people with autism spectrum disorders (ASD),  I was intrigued by the possibility that the facial recognition APIs might provide a means of assessing mood or emotional states.   Most of us understand that our faces function as mirrors to feelings, and we use our facial expressions to communicate our feelings to others.  Unfortunately, this is a concept that is difficult for young people with ASD to understand.  My hunch is that there is an "app for that".   


I'd love to great to create a little iPad app for young people with ASD for education, intervention, and/or communication activities that incorporates the facial recognition feature!


Apple's iOS facial recognition could lead to Kinect-like interaction
Darrell Etherington, GigaOm/Reuters, 7/27/11
Here is a quote from the above article:
"You could create apps that track a user’s eye movement and dynamically change content accordingly, for instance. App developers might even be able to use data gathered from facial recognition APIs to identify so-called “hotspots,” providing insight about where a user is looking most within an app and arranging content accordingly. In time, an iPhone app might even be able to assess the emotional state of the user, based on whether they’re frowning or smiling, and address the user in a manner appropriate to their mood. It might also be able to tell how engaged users are with mobile ads and content, which might be useful for iAd customers, among others."

Face Recognition Coming to iOS5

Stan Schroeder, Mashable, 7/26/11

Below is a video from from the above post which demonstrates an app developed by Polar Rose, a company that was purchased by Apple.

RELATED
Apple plans native panorama functionality in iOS5
Seth Weintraub, 9TO5Mac, 7/8/11

iOS 5's final release may include "Assistant" speech-to-text feature
Chris Rawson, TUAW, 7/23/11

Eric Sailer's Lists of iPad, iPod, and iPhone Apps for (special) education

Update: Here is a link to Eric Sailer's resources on Scribd:
http://www.scribd.com/doc/24470331/iPhone-iPad-and-iPod-touch-Apps-for-Special-Education



Update:  I think Eric Sailer's blog has moved.  For now, here is a link to his website:
http://ericsailers.com/iresources.html

Eric Sailers is a speech/language pathologist who has an extensive amount of experience using "apps" in educational settings.  He's compiled a great list of apps and related resources on his blog:  http://slpsharing.com/app-resources/

Eric spends some of his time giving presentations about the use of Apple's mobile devices in education. He's co-created a few apps, too!

April 25th Release Date: Play and Experiment with Music on your iPad/Phone/Pod with NodeBeat iOS Music Sequencer by Affinity Blue, by Seth Sandler and Justin Windle

If you love music and have an iPad, iPhone, or iPod touch, you'll have fun exploring and experimenting with NodeBeat. As soon as I get my iPad, I plan to get this app! 


"NodeBeat is an experimental node-based audio sequencer and generative music application for the iPhone, iPod Touch and iPad. NodeBeat is designed to let you create a variety of rhythmic sequences and ambient melodies in an intuitive fashion. It is scheduled for release on April 25th." -Affinity Blue


NodeBeat - iOS Music Sequencer from AffinityBlue on Vimeo.

NodeBeat is a collaboration between Seth Sandler and Justin Windle. It was developed using openFrameworks, with PureData for audio synthesis.  According to the NodeBeat website, "Octaves and Notes make up the two types of nodes. Octaves pulse and send messages to Notes within proximity. Each Octave is assigned a random octave and each Note, a random note; therefore, a Note will play in several octaves depending on the Octave it's connected to. Pause nodes to create your own beats or let them roam free to have them generate their own." Features include adjustable node physics, node add/remove, pitch shift, adjustable pulse rate and beat sync, adjustable echo, attack, decay, and release for creative sound sculpting, and for the iPad, audio waveform display and landscape/portrait views.

RELATED
NodeBeat website
Seth Sandler's posts about music
Seth's posts and videos about AudioTouch, and interactive multi-touch interfaces for computer music exploration and collaboration.
Soulwire (Justin Windle's blog)


Cross-posted on The World Is My Interactive Interface and TechPsych blogs.

Apps and Gadgets to Soothe the Baby: New York Times Article (video, links) -a little off-topic

I came across a link to a recent NY Time's article, "The Pacificer Gets and Upgrade:  Apps and Gadgets to Soothe the Baby" on Nat Torkington's "Four short links" O'Reilly Radar post, and thought it was worth sharing, especially for my readers who are new parents or new grandparents! Here are a few of the links from the NY Times article:


iPhone Apps
White Noise   by TMSOFT  (Rated positive by new dad and NY Times writer Farhad Manjoo)
"White Noise provides ambient sounds of the environment to help you relax or sleep.  Includes high quality looping noises such as ocean waves, crickets chirping at night, and the soothing sound of rain fall."


MamaRoo "It moves like you do"

The following two apps/gadgets didn't seem to work as advertised, according to Farhad Manjoo:
Cry Translator "The iPhone App to Understand Baby Crying"
Why Cry Baby Analyzer


Baby Monitors
Philips Advent Basic with DECT Technology
Angelcare Movement and Sound Monitor
"Besides relaying your baby’s cries, it has a sensor pad, placed under the crib or bassinet mattress, that alerts parents when a baby has not moved for 20 seconds. It does not go off incessantly (the machine is sensitive enough to detect breathing) and is well suited to parents like me who cannot shake the paranoia that can come with caring for a new human being." -Farhad Manjoo


Farhad discussed monitors that feature web connectivity,   such as the MobiCam Digital Wireless Video Monitor and the Home Digital Video Monitor. These systems enable parents to watch the monitor's video of the baby via a smartphone.  To make this work, the baby monitor must be connected to a laptop or computer, which in turn must be turned on continuously, something that might not bee too convenient for busy parents.  He suggested that new parents take a look at the Lorex Wireless Easy Connect Network Camera, which has a built-in Wi-Fi chip that enables it to  connect to a wireless network without a computer. The Lorex isn't a baby monitor, but since it doesn't require a computer to work, it might be useful to anxious parents on-the-go who might want to see if their baby is being treated appropriately by a caregiver in their absence.


OTHER
Voice Activated Crib Light with Womb Sounds (just $14.99!)
Itzben Baby Care Timer ($24.00 -keeps track of diaper changes, feedings, and sleep patterns)


Farhad Manjoo's comment about babies and gadgets is worth heeding:
"... but babies are a fickle, ever-changing lot, and what works one time seems hopeless the next. As a gadget fiend, I’m used to evaluating technology in controlled settings, but there is nothing controlled about life with baby."


RELATED
ThinkGeek:  GeekKids Newborn and Infant 
The Pacificer Gets and Upgrade:  Apps and Gadgets to Soothe the Baby
Farhad Manjoo, New York Times, 11/24/10


Cross-posted on the TechPsych blog

Interactive iPad Apps for Kids with Autism: Could some of these be transformed for multi-touch tabletop activities?

I came across a great post about interactive iPad apps for special needs:

Ten Apple iPad Apps to Help Children with Autism
Joanne Carter, MacCreate 11/5/10


In this article,  Joanne Carter shares screen shots and detailed descriptions of a variety of iPad apps that support learning and communication skills of young people with autism.  You can find additional information about the apps discussed in the article by visiting the following links: 

Proloquo2go, Story_Builder, "Off we go" book series, Soundtastic, Visual Impact, Living Safely,  Tapspeak Sequence for iPad, iCommunicate for iPad, Autoverbal Talking Soundboard Pro, Is that Gluten Free?, and I Dress for Weather. 


I think that some of these apps have the potential to be transformed and tweaked for use on multi-touch, multi-user  tables such as the SMARTTable or Microsoft's Surface.  The aim would be to encourage paired and group communication and social skills among children with special needs.  I'll share my thoughts on this topic in a future post.

Virtual Glee App for your iPad, from Smule: Sing with yourself, or other Gleeks around the world, in perfect harmony!

Sing with yourself and Gleeks around the world in perfect harmony: 

Demo of the Glee iPhone + iPad App by Smule - "I'll Stand By You"
"Glee, for iPhone + iPad, by Smule. This impromptu, one-take performance by Smule engineer Nick late one night at HQ demonstrates how the app transforms his voice into a group of singers harmonizing with him."

This is exactly why I NEED an iPad!

Tutorial Part 1
"How To"



Tutorial Part II:
The Glee GlobeShows how you can share your Glee songs and sing with others around the globe!





RELATED 
Cool iPad Digital Piano App

Apple iSlate, iTablet , MacBook Touch: Will it support gesture interaction & haptic feedback?

Soldier Knows Best produces great tech-oriented videos. Here's his spin on all of the rumors about the possibility of the Apple iSlate.


I just inherited a 10 month-old Mac Book, installed Snow Leopard and upgraded to iLife 2009. I'm so used to touching the screen on my HP TouchSmart PC that I found myself touching my Mac Book screen from time to time, especially when I was editing video clips in iMovie. I think the latest version of iMovie was designed with touch/gesture interaction in mind!

From what I can tell, Snow Leopard and iLife 2009 will be able to support a range of touch interactions, if not gesture input as well.

Here are some rumors that have been conjured up and distributed on the web:

The Exhaustive Guide to Applet Tablet Rumors (Matt Buchanan, Gizmodo, 12/26/09)
Apple Expects to Sell 10 Million Tablets in First Year (Pete Cashmore, Mashable, 1/1/10)
iGuide Emerges as Another Potential Apple Tablet Name (Adam Ostrow, Mashable, 12/29/09)
The Tablet (John Gruber, Daring Fireball, 12/31/09)
"And so in answer to my central question, regarding why buy The Tablet if you already have an iPhone and a MacBook, my best guess is that ultimately, The Tablet is something you’ll buy instead of a MacBook."
Apple Owns iSlate.com Domain: The Mystery Deepens (Dan Nosowitz, Gismodo, 12/25/09)
What is the Ultimate Role of the Apple Tablet? (Arnold Kim, MacRumors, 12/31/09)

• 153+ comments to the above MacRumor post

iPad, iTablet, iSlate, or MacTab  (Cruz Miranda, 8/31/09)

Why am I excited about this?

I want to see if the iSlate would be good for collaborative educational games, assisted technology, augmentative communication, and alternative assessment for students who have multiple/severe disabilities.

That is a huge goal, so I'm going to start simple.  I am not giving up on Windows 7 multi-touch programming. I just have an urge to find out for myself what works, what doesn't, and what platform works best for specific "personas" and "scenarios".

I plan to make a little app for the iPhone/iPod Touch, based on a game I made several years ago, "Shoes Your Battles" for a game class. I think I'd like to make this game for the Apple iTablet!

The first version of Shoes Your Battles created with Game Maker, and the second version was in Flash, back in the days of ActionScript 2.0.  I started on third version, one that could be used as an advergame for people to play while shopping for shoes during shoe sales, but it never got past the planning stage.  

The idea for the third version came to me when I my elderly aunt came to visit from out-of-town and just had to go shoe shopping on the day after Thanksgiving.  It was extremely difficult to figure out what was on sale, how much it cost, after taking off the previous mark-downs and what was on sale that had a price that was not yet marked down.  

Adding to the confusion was the fact that there were few salespeople and herds of women.   It was madness.  There were pairs of shoes in the wrong boxes, boxes of shoes and no way to quickly find out the true prices!   We were in the shoe department for hours, and it wasn't as fun as you'd think. If you've been in a crowded women's shoe department to buy that special pair of shoes during a fantastic shoe sale, you'll know what I mean.

At any rate, I wanted my little "Shoes Your Battles" game to help with this dreadful scenario, by somehow incorporating a shoe shopping advisor and a means to figure out the REAL sales prices of those awesome, to-die-for shoes. Unfortunately, the technology wasn't where it needed to be at the time- I am always dreaming up things that are too d--- futuristic!

4 years later, we have iPhones and SmartPhones and 3G internet and RFID and ubiquitous WiFi and the Wii and more women who like to play games and...and... The time is ripe.

Apple better come up with the iSlate!

SOMEWHAT RELATED


Thinking about post-WIMP HCI
It is always important to re-visit wisdom from the past when thinking about new interfaces and means of technology-supported human interaction.  Here are a few resources from the field of Human-Computer Interaction found on the HCI Vistas website:
The Prism of User Experience  -A nice graphic metaphor to help the conceptualization process. (Denish Katre, 2007)
Journal of HCI Vistas: Multi-disciplinary Perspective of Usability and HCI
Personas as part of a user-centered innovation process Lene Nielsen, 1/08 HCI Vistas Vol-IV
10 Steps to Personas (Lene Nielsen, 7/07, HCI Vistas Vol-III)

Social Music On-the-Go - Stanford Mobile Phone Orchestra; Ge Wang's SMULE apps for the iPhone

In the Charlotte Observer today, I came across an article about the Stanford Mobile Phone Orchestra- via the New York Times:

Musicians push edge of computer music with iPhone
Stanford orchestra turns popular songs into elaborate electronic renditions
I remembered that I'd posted about this previously nearly a year ago:  Play a flute by blowing on your iPhone!  In that post, I discussed an iPhone app by Smule called "Ocarina".  (If you follow the link to the post, you can view a video of some folks playing a version of Stairway to Heaven on their iPhones.)

Stanford Mobile Phone Orchestra (MoPhO)

A bit of background information:
Smule is company started by Ge Wang, an assistant professor of music at Stanford who is the director of the Stanford Mobile Phone Orchestra.  He also directs the Stanford Laptop Orchestra (SLOrk), and previously was involved with the Princeton Laptop Orchestra, known as PLOrk.  (Perry Cook, the author of Real Sound Synthesis for Interactive Applications, is one of the directors of PLOrk.)

Ge Wang's Ph.D. work was on ChucK, an audio programming language for "real-time synthesis, composition, performance, and analysis, supported on Mac0S X, Windows, and Linux.  To celebrate Ge Wang's thesis, Perry Cook wrote Everybody Hack ChucK Tonight (.mp3), to the tune of "Everybody Wang Chung Tonight".  The lyrics explain what the ChucK program is all about.

Ge Wang is teaching Mobile Music (Music, Computing, and Design II) at Stanford during the Spring 2010 semester. His department is affiliated with CCRMA, the Center for Computer Research in Music and Acoustics. (CCRMA offers non-credit week-long workshops each summer, open to the public. Maybe there will be a workshop on mobile music next summer!)

Enjoy the videos below:

Stanford Researcher Uses Cell Phones to Make Music

Stanford Report, March 2009

Someone posted this on YouTube for the holidays last December:

Yelp has an Augmented Reality iPhone App for the iPhone 3Gs, "The Monocle"!

(Cross posted from tshwi)

Thanks to Dan Saffer for the link from the Read Write Web!

Here is my partial "reblog":

"Social review service Yelp has snuck the first Augmented Reality (AR) iPhone app specifically for the US into the iTunes App Store. The undisclosed new feature allows iPhone 3Gs owners to shake their phones three times to turn on a view called "the Monocle." This view uses the phone's GPS and compass to display markers for restaurants, bars and other nearby businesses on top of the camera's view...Blogger Robert Scoble discovered the hidden feature and posted about it on FriendFeed today. "

Video (in French, but easy to understand by the demonstration)


"Both GPS and a compass are used to determine location and direction being pointed at."

Screen Shots:




I want an iPhone. Verizon, can you hear me now?!