One of the things I like to do is share updates about the world of multimedia, multi-touch, gesture, screen, surface, and interactive technologies, focusing on off-the-desktop applications and systems. When I started this blog, I had to put forth quite a bit of effort just to FIND interesting things to blog about.
These days, there are so many sources that focus on emerging - and now commonplace- interactive technologies, my main challenge is to filter the noise. Where do I begin?
My archives are vast. I randomly picked the year 2009 and came across one of my previous posts, "News, Videos, and Links about Multitouch and Screen Technologies." The post is long, and contains a number of videos and links that probably will be of value to a future curator of the history of technology.
I welcome comments from readers who might be able to help me update information about various applications and systems I've featured on this blog in the past.
The pictures are screenshots from the results of an image search for "interactivemultimediatechnology". Over the past 6 years, I've posted quite a few!
I was browsing at a Barnes & Noble in Charlotte, N.C. this weekend and was surprised when I looked up to see that I was in the "Men's Interest" section. Is this the message the bookseller wants to give to half of its potential customers?
I don't think it was intentional. It was just one of many subtle- and not-so-subtle - reminders that our society has a long way to go to encourage females, of any age, to consider computing and related technical fields of work and study.
Although women have made inroads in fields such as law and medicine, this is not true for many technical fields. According to a thoughtful article written by Rane Johnson-Stempson, of Microsoft Research, a study by the Computing Research Association indicated that female students were only 14% of computer science graduates in the U.S. in 2011. The under-representation of women in computer science and related technical fields in 2012 continues to be a problem, on many levels. RELATED Unlocking the Clubhouse: Women in Computingwas published about ten years ago. The book was an outgrowth of the research of Allan Fisher and Jane Margolis, of the School of Computer Science at Carnegie Mellon University, and is still worth reading.
Getting more women into computer science Connie Glasser, Atlanta Business Journal, 5/11/12 "Just when we need every available gifted mind to help business recover from the Great Recession, we can't afford to overlook the contributions that women could make if CS environments were more inclusive."
Marissa Mayer on Women in Tech: CNET's CES 2012 Panel (January, 2012) "Right now it is a good time for women to be in tech, but there are not a lot of women in tech..."
It just might be the right time for everyone to brush up on 21st century tech skills. iPads and touch-phones are ubiquitous. Touch-enabled interactive whiteboards and displays are in schools and boardrooms. With Microsoft's Windows 8 and the news that the company recently acquired Jeff Han's company, Perspective Pixel, I think that there will be good support - and more opportunities- for designers and developers interested in moving from GUI to NUI.
In the video below, from CES 2012, Jeff Han provides a good overview of where things are moving in the future. We are in a post-WIMP world and there is a lot of catching up to do!
CES 2012 Perceptive Pixel and the Future of Multitouch (IEEE Spectrum YouTube Channel)
During the video clip, Jeff explains how far things have come during the past few years:
"Five and
1/2 years ago I had to explain to everybody what multi-touch was and meant. And
then, frankly, we've seen some great products from folks like Apple, and really
have executed so brilliantly, that everyone really sees what a good
implementation can be, and have come to expect it. I also think though, that the explosion of
NUI is less about just multi-touch, but an awareness that finally
people have that you don't have to use a keyboard and mouse, you can demand
something else beside that. People are
now willing to say, "Oh, this is something I can try, you know, touch is
something I can try as my friendlier interface"."
Who wouldn't want to interact with a friendlier interface? Steve Ballmer doesn't curb his enthusiasm about Windows 8 and Perceptive Pixel. Jeff Han is happy how designs created in Windows 8 scales for use on screens large and small. He explains how Windows 8 can support collaboration. The Story Board application (7:58) on the large touchscreen display looks interesting.
I continue to be frustrated by the poor usability of many web-based and desk-top applications. I like my iPad, but only because so many dedicated souls have given some thought to the user experience when creating their apps. I often meet with disappointment when I encounter interactive displays when I'm out and about during the day. It is 2012, and it seems that there are a lot of application designers and developers who have never read Don Norman's The Design of Everyday Things!
I enjoy making working prototypes and demo apps, but my skill set is stuck in 2008, the last year I took a graduate-level computer course. I was thinking about taking a class next semester, something hands-on, creative, and also practical, to move me forward. I can only do so much when I'm in the DIY mode alone in my "lab" at home. I need to explore new tools, alongside like-minded others.
There ARE many more tools available to designers and developers than there were just four years ago. Some of them are available online, free, or for a modest fee. I was inspired by a link posted by my former HCI professor, Celine Latulipe, to her updated webpage devoted to Rapid Prototyping tools. The resources on her website look like a good place to start for people who are interested in creating applications for the "NUI" era. (Celine has worked many interesting projects that explore how technology can support new and creative interaction, such as Dance.Draw.) Below is her description of her updated HCI resources:
"New HCI resource to share: I have created a few pages on my web site devoted to Rapid Prototyping tools, books, and methods. These pages contain reviews of various digital tools, including 7 different desktop prototyping apps, and including 8 different iPad apps for wireframing/prototyping. I hope it's useful to others. Feel free to share... and please send me comments and suggestions if you find anything inaccurate, or if you think there is stuff that I should be adding. I will be continuing to update this resource." -http://www.celinelatulipe.com (click on the rapid prototyping link at the top)
IDEAS Below are just a few of my ideas that I'd like to implement in some way. I can't claim ownership to these ideas- they are mash-ups of what comes to me in my dreams, usually after reading scholarly publications from ACM or IEEE, or attending tech conferences.
An interactive timeline, (multi-dimensional, multi-modal, multimedia) for off-the-desktop interaction, collaboration, data/info analysis exploration. It might be useful for medical researchers, historians, genealogists, or people who are into the "history of ideas". Big Data folks would love it, too. It would handle data from a variety of sources, including sensor networks. It would be beautiful to use.
A web-based system of delivering seamless interactive, multi-modal, immersive experiences, across devices, displays, and surfaces. The system would support multi-user, collaborative interaction. The system would provide an option for tangible interaction.
A visual/auditory display interface that presents network activity, including potential intrusions, malfunctions, or anything that needs immediate attention that would be likely to be missed under present monitoring methods.
Interactive video tools for creation, collaboration, storytelling. (No bad remote controllers needed.)
A "wearable" that provides new ways for people to express and communicate creatively, through art, music, dance, with wireless capability. (It can interact with wireless sensor networks.)*
An public health application designed to provide information useful in understanding and sepsis prevention efforts.This application would utilize the timeline concept describe at the top of this list. This concept could also be useful in analyzing other medical puzzles, such as autism.
Most of these ideas could translate nicely to educational settings, and the focus on natural user interaction and multi-modal i/o aligns with the principles of Universal Design for Learning, something that is important to consider, given the number of "at-risk" learners and young people who have disabilities.
I welcome comments from readers who are working on similar projects, or who know of similar projects. I also encourage graduate students and researchers who are interested in natural user interfaces to and move forward with an off-the-desktop NUI project. I hope that my efforts can play a part in helping people make the move from GUI to NUI!
Below are a few videos of some interesting projects, along with a list of a few references and links.
For years, the tech community in Charlotte was dominated by people who worked for the major financial institutions. Things are changing. Facilitated by social media and local meet-ups, groups such as Charlotte UX, Packard-Place, and Charlotte Hackers and Founders have created an energetic synergy that reflects the "can-do", positive attitude of the Charlotte region.
Last week I attended a meetup of TechCrunchand Charlotte-area techies, held at the uptown Packard Place entrepreneurial center. My main purpose for attending the TechCrunch meet-up was to learn more about innovative technology start-ups in my region. Because the place was crowded and there were very few booths, I changed my strategy and wandered about, and stopped if anything interesting - like a T-shirt or a winning smile- caught my eye. (See TechCrunch Charlotte Highlights: T1 Vision, inTouch Software)
Cards I Collected, Info I gathered
One group of enthusiastic techies had a working demonstration of their system, NexTable, the "next generation reservation, table management system and marketing solutions for restaurants".Since I worked as a waitress in my college days, and married a guy in the restaurant business (he's now a recruiter), I know how crazy it is to make sure the reservation process goes smoothly. From what I could see from the iPad interface, the reservation component would be very easy for busy restaurant staff to implement.
The NexTable system looks like it would be helpful to wait-staff, customers, and restaurant owners. It aggregates online reservations from any source, which is great for busy, hungry people. It provides a text platform for customer reservations, wait-lists, confirmations, and notifications when tables are ready. Data generated from this system can be used to customize marketing campaigns.
Queen Associates: Frances M. Queen Queen Associates is an IT consulting group that provides talent for clients across multiple industries. They promote the hiring of veterans and minority candidates. "It takes many different voices, many different viewpoints and an environment of respect and inclusiveness to provide the best service for our clients." -Queen Associates website
fritter: Derek G. Dahl "By eliminating order errors and the hassle of phone ordering, our cutting edge online ordering platform streamlines the food ordering and delivery process. Fritter.co creates ordering solutions for restaurants and hungry customers, providing quality cuisine at an affordable price. Support local restaurants while enjoying great deals and promotions."-Fritter website
AddShoppers: Jon West AddShoppers is a social commerce analytics company. It helps internet-based retailers figure out how their social media efforts reach customers. The service is free. "AddShoppers is the only social sharing platform built for online retailers. We help retailers increase sharing while at the same time providing unparalleled analytics".
MailVu : Addy Kapur MailVU provides a way to communicate through video email. It can be embedded into websites or even learning management systems such as Blackboard and Moodle. For businesses, it can be embellished with a logo.
KYCK"The Global Soccer/Futbol Experience" "KYCK is a digital media platform that discovers, curates, and delivers personalized and highly relevant soccer content to our users" I wasn't able to catch the guy who was wearing this poster on his back. I don't follow soccer, but it seams that the platform could be used by followers of other special interests, such as tracking weather disasters around the world, favorite music groups and events, and so forth.
Traditional remote controls for televisions and home media centers are particularly frustrating to use, in my opinion. There are too many buttons, the buttons are too small, and it is easy to mess it all up. (I've touched on this topic in a number of blog posts, such asUX of ITV: The User Experience and Interactive TV (or Let's Stamp Out Bad Remote Controls)and others, linked at the end of this post.)
There is hope for the future! I'm happy to share a video about a method of remote control that might make interacting with my TV less annoying...and possibly pleasant!
Researchers at the Technical University of Darmstadt have been working on a way for people to control their TV without a remote or hand-held device. I think they are on the right path. The video shows how UI elements, mapped to the hand, can be touched, with accuracy, to interact eyes-free with a large-screen TV. This work won the Best Paper award at EuroITV'12.
Niloofar Dezfuli, Mohammadreza Khalilbeigi, Jochen Huber, Florian Müller, and Max Mühlhäuser. 2012. Leveraging the palm surface as an eyes-free tv remote control. In Proceedings of the 2012 ACM annual conference extended abstracts on Human Factors in Computing Systems Extended Abstracts (CHI EA '12). ACM, New York, NY, USA, 2483-2488. http://doi.acm.org/10.1145/2223656.2223823
I am brave. I am one of the few pioneers who have tried to scan QR codes, no matter what. I have squatted in front of a display to carefully align my phone with a code. I have disregarded many passers-by who had no choice than to walk around me. I have blocked the path to the restrooms at a crowded airport, just to capture that special shot.
I was so intent on capturing one QR code that I stood on my tip-toes in front of a large refrigerated drink section of an airport convenience store full of thirsty people. I didn't care if I blocking the aisle to the drinks! I just had to watch the ads cycle on a dynamic display for the right moment to scan and snap the QR code.
My last QR encounter was at a Harris Teeter supermarket in Charlotte, NC. The purpose of the QR code was to link customers to the store's mobile app. It turned out to be a laborious process. It had little to do with the QR code or my iPhone's scanning software. It had nothing to do with my ability to steady my iPhone and quickly capture a QR code. It had everything to do with a poorly-planned spot for this transaction to take place. Even though QR codes have been around for a while, it seems that little thought has been given to the user/customer experience at the "point of scan".
This QR code made me look especially silly. I got a few strange looks from other shoppers entering and exiting the store. Imagine what I looked like as I approached the automatic sliding door! After a sashaying around for a few minutes, I captured the code. I guess I was suffering from QR Code Fatigue by that point, because after I downloaded the mobile app, I left the store and forgot all about it.
App store and mobile app screen shots:
There was another option for scanning the Harris Teeter QR code. At the checkout counter, there was one located on the screen on top of the cash register. It was not possible to reach over everything in order to scan the code. Even if I could access the QR code while checking out, it wouldn't be wise. I'd be holding up the line, and no-one wants to be "that guy".
A couple of weeks ago, I decided to relax a bit and do some gardening. I noticed that every single plant had a tag, and every single tag had a QR code. Instead of taking a break to sip some iced tea, like other neighborhood ladies, I got my iPhone, wiped a plant tag clean (you don't want to scan a QR code obscured with garden soil), found a flat surface, and scanned. I spent too much time learning more than I ever cared to know about vincas and begonias.
The following section is from one of my previous posts on this topic, reviewing my second encounter with a QR code at the Orlando airport. This particular QR code was on a display in a very crowded convenience store.
I spotted another QR tag! There were a few problems with this set-up. First, the display was located on a wall close to a ceiling, making it very inconvenient to scan with my phone. The page with the QR tag didn't stay up long enough for me to attempt to scan it, although waited around until it cycled back in view- three times. I gave up and took a picture instead, standing on my tip-toes with my arms extended over my head, the same position required to scan the QR tag, if it would stay put!
Another problem was that the display was perched above a high-traffic area, right above the refrigerated drinks. The only place to scan the QR tag was in a narrow aisle, behind the throngs of thirsty travelers making a bee-line to the cold drink section of the store. I was in the way of a number of people in this bee-line.
What were they thinking when they decided that the best place this large display featuring a QR tag should be several feet above eye-level in this particular location? The displays in front of the store at the airport were located on either side of the doorway, at "body" level. This is a logical place to put a display. Potential customers can view the displays without getting in the way of other travelers. Here's a picture of one display, found at a garden center, that provided an adequate user-experience. Instead of a QR code, the display had a Microsoft Smart Tag. The display was located in a spot where I didn't feel silly. The tag was easy to scan, and it linked me to a website that had the same basic content as the touch screen below. I could "scan and go" if I was in a hurry, scan and look at the screen of my iPhone if I wanted to walk around the store, or stand in front of the display and watch the infomercials. The following picture is a screen shot of a site someone put up as a joke, no pictures, of course, because there are so few that do!
Update 3/13 I was going through the drive-through pharmacy at CVS and spotted this QR code on the window. I asked the pharmacy technician what she knew about the QR code, and she said, "What QR code". From our discussion, not much was known about this QR code, and no one to her knowledge had ever tried to scan it. I tried, and failed to succeed.
My archives are vast. I randomly picked the year 2009 and came across one of my previous posts, "News, Videos, and Links about Multitouch and Screen Technologies." The post is long, and contains a number of videos and links that probably will be of value to a future curator of the history of technology.
