I like this demonstration of Adam Somers AirHarp music application for use with the Leap Motion 3D controller:
AirHarp is being developed in C++ using Adam Somer's audio processing toolkit, MusKit. This looks interesting! Things have changes since I last took a computer music technology course (back in 2003). Adam Somers is a senior software engineer at Universal Audio. He has a graduate degree in music technology from Stanford, and a background in computer science, electronics, human-computer interaction, and signal processing. Leap Motion is a motion-control software and hardware start-up company located in San Francisco, California. According to promotional information from the website, the company's first product, the Leap Motion controller, is 200 times more sensitive than existing technologies. It will be interesting to see how this plays out. (I'm still waiting for my pre-order.) RELATED AirHarp (links to GitHub) Leap FAQs Leap Motion Website Leap Motion Developer Portal Leap Motion Leadership Team Leap Motion goes retail: Motion controller sold exclusively at Best Buy Michael Gorman, engadget, 1/16/13
The Affinity+ concept has the potential to be useful in educational settings such as schools, museums, and libraries. Although it was designed to support collaborative activities among software designers/developers, it could support a wide range of collaborative project-based learning activities. The clearly narrated video below was produced by a team from the Pacific Northwest National Laboratory.
"Affinity diagraming is a powerful method for encouraging and capturing lateral thinking in a group environment. The Affinity+ Concept was designed to improve the collaborative brainstorm process through the use of large display surfaces in conjunction with mobile devices like smart phones and tablets. The system works by capturing the ideas digitally and allowing users to sort and group them on a large touch screen manually. Additionally, Affinity+ incorporates theme detection, topic clustering, and other processing algorithms that help bring structured analytic techniques to the process without requiring explicit leadership roles and other overhead typically involved in these activities." -PNNL RELATED Affinity+ Semi-Structured Brainstorming on Large Displays Russ Burtner, Richard May, Randy Scarberry, Ryan LaMothe, Alex Endert Pacific Northwest National Laboratory
Here's an interesting use of technology for health - the Xbox Kinect in the OR!
Thanks to Harry van der Veen for the link! RELATED Kinect sensor poised to leap into everyday life Niall Firth, NewScientist, 1/17/13 For the tech-curious: PrimeSense(Company that developed the 3D depth sensor that powers the Kinect, the sensor in Ava, a healthcare robot by iRobot, and more.) OpenNI (Framework for the development of 3D sensing middleware libraries and applications.) NiTE: Natural Interface Technology for End User (Perception algorithms layer for 3D computer vision, allows for hand locating, tracking, analyzing scenes, and tracking skeleton joints.)
"GML is an extensible markup language used to define gestures that describe interactive object behavior and the relationships between objects in an application. Gesture Markup Language has been designed to enhance the development of multiuser multi-touch and other HCI device driven applications." -Gesture ML Wiki
Photo credit: Ideum RELATED Ideum Blog OpenExhibitsFree multitouch and multiuser software initiative for museums, education, nonprofits, and students GestureWorks Multi-touch authoring for Windows 8 & Windows 7
I've been looking for a relatively short video about human-computer interaction and related fields to include in a presentation I'm planning for high school students. The presentation is my small part to promote Computer Science in Education Week (CSEd). One of the goals of CSEd Week is to spread the word that computer science education is much more than learning how to program one. Technical and computational thinking skills are important to develop, but young people also need to know what sort of things they can do with these skills as they become adults in our technological society. As stated on the CSEd website, "Computing professionals work on creative teams to develop cutting-edge products and solutions that save lives, solve health problems, improve the environment, and keep us connected." Coincidentally, I was pleasantly surprised by a tweet I received today that linked to Connecting, a well-produced 18-minute video about interaction and user experience design. This video would be great to share with high school students.
The video features a number of well-spoken, creative professionals who are passionate about their work, people, and the future. Although the video is a bit techno-centric, it depicts people who live and breathe technology in a favorable light. It also inspires some degree of thought and reflection on the part of the viewer. Although much of what is discussed in Connecting is futuristic, the seeds were planted years ago. If you are new to the HCI/UX/ID/UCD world, it might help to read Mark Weiser's 1991 article, The Computer for the 21st Century, published in Scientific American in 1991, before viewing the video. After viewing the video, I encourage you to take the time to read some of the comments on the Vimeo website. Also read Marc Rettig's comments, posted on the IxDA website: "A film about interaction design: what it says about us".
Near the end of the video, there is a discussion about where we might be headed, as interconnected, technically enhanced, augmented humans. Hopefully we will not create, and then be assimilated into a Borg-like collective, or live out our days in a Matrix-like disembodied state. In the wrong hands, what might happen? Is resistance futile?!
Younghee Jung: "... you can not necessarily foresee the consequences when people adopt what you designed..to see something completely different from what you created. .it is like throwing a stone in the water, and you don't know what it will cause."
Blaise Aguera y Arcas: "....these are all augmentations of abilities as humans. And when the augmentation really works, then that extension of yourself feels natural, and beautiful and does what you want, and doesn't get in the way....The use of voice, and the use of natural gestures... you are removing the extraneous, you are removing the artificial."
Massimo Banzi: "...Something that can do it's own thing, disappearing in the background, is correct" (nod to Weiser)
Jennifer Bove: "...it is really important to look at what the consequences are of putting these products into the world when we think about things like the phone...the way it has changed our behavior, it can be enabling, and also disrupting...for these things to change our lives for the better, or enable for them to let us do things we couldn't do before.. they have to feel natural, and feels like a conversation." Robert Murdock: "How you actually design and enact a living system in UX is something that is quite challenging...you have to think about patterns of desired outcomes and behaviors you want to achieve, instead of moving a user through one flow in an experience."
Jonas Lowgren: "...back in the day.. it was one user, one task, one computer, its all gone now, its is much more like you are setting the stage, really, for other people to perform, but you can never tell them what to do." Eric Rodenbeck: "....the map is like a living thing, that is being made up of everything we got. The idea that it is different in the morning than what it was in the evening, is a really good idea to stay connected to the idea that the world is changing." Helen Walters: "What we need is for designers to be embedded in the topics that are really, really important right now, so there can be a better synergy between design, and business design, and social change design, and entrepreneurship." Andrei Herasimchuck "That is where the future lies with us. There will be software in everything..You can take all of those (digital) pieces, and you can design all kind of things around it. People are now actually entering their lives and what is going around them, into a digital format, and so we will start do things with that in the future, and I think it will be exciting."
Robert Fabricant: "The network is sampling the world, and knowing what is cropping up where, being able to match and find patterns...and anticipate outbreaks of diseases. .. We are trying now to collect from the periphery a much richer set of what is going on the world so we can learn as a society and optimize and evolve the right systems and services". SOMEWHAT RELATED IxDA Experientia: Putting People First What's the Difference- IXD, IA, UXD, HCI, UCD, UX (Jon Karpoff)
CFP for Special Issue of Personal and Ubiquitous Computing on Educational Interfaces, Software, and Technology (EIST) -Extended Deadline: December 9, 2012 Overview One of the primary goals of teaching is to prepare learners for life in the real world. In this ever-changing world of technologies such as mobile interaction, cloud computing, natural user interfaces, and gestural interfaces like the Nintendo Wii and Microsoft Kinect, people have a greater selection of tools for the task at hand. Given the potential of these new interfaces, software, and technologies as learning tools, as well as the ubiquitous application of interactive technology in formal and informal learning environments, there is a growing need to explore how next-generation technologies will impact education in the future. As a community of Human-Computer Interaction (HCI) and educational researchers, we need to theorize and discuss how new technologies should be integrated into the classrooms and homes of the future. In the last three years, three CHI workshops have provided a forum to discuss key issues of this sort, particularly in the context of next-generation education. The aim of this special issue of Personal and Ubiquitous Computing is to summarize the potential design challenges and perspectives on how the community should handle next-generation technologies in the education domain for both teachers and students. We invite authors to present position papers about potential design challenges and perspectives on how the community should handle the next generation of HCI in education. Topics of interest include but are not limited to:
Submission Guidelines Submissions should be prepared according to the Word template located at the bottom of this page. All manuscripts are subject to peer review. Manuscripts must be submitted as a PDF to the easychair submission system. Submissions should be no more than 8000 words in length. Guest Editors and Contact Information
Syed Ishtiaque Ahmed, Cornell University
Quincy Brown, Bowie State University
Jochen Huber, Technische Universität Darmstadt
Si Jung “Jun” Kim, University of Central Florida
Lynn Marentette, Union County Public Schools, Wolfe School
Max Mühlhäuser, Technische Universität Darmstadt
It just might be the right time for everyone to brush up on 21st century tech skills. iPads and touch-phones are ubiquitous. Touch-enabled interactive whiteboards and displays are in schools and boardrooms. With Microsoft's Windows 8 and the news that the company recently acquired Jeff Han's company, Perspective Pixel, I think that there will be good support - and more opportunities- for designers and developers interested in moving from GUI to NUI.
In the video below, from CES 2012, Jeff Han provides a good overview of where things are moving in the future. We are in a post-WIMP world and there is a lot of catching up to do!
CES 2012 Perceptive Pixel and the Future of Multitouch (IEEE Spectrum YouTube Channel)
During the video clip, Jeff explains how far things have come during the past few years:
"Five and
1/2 years ago I had to explain to everybody what multi-touch was and meant. And
then, frankly, we've seen some great products from folks like Apple, and really
have executed so brilliantly, that everyone really sees what a good
implementation can be, and have come to expect it. I also think though, that the explosion of
NUI is less about just multi-touch, but an awareness that finally
people have that you don't have to use a keyboard and mouse, you can demand
something else beside that. People are
now willing to say, "Oh, this is something I can try, you know, touch is
something I can try as my friendlier interface"."
Who wouldn't want to interact with a friendlier interface? Steve Ballmer doesn't curb his enthusiasm about Windows 8 and Perceptive Pixel. Jeff Han is happy how designs created in Windows 8 scales for use on screens large and small. He explains how Windows 8 can support collaboration. The Story Board application (7:58) on the large touchscreen display looks interesting.
I continue to be frustrated by the poor usability of many web-based and desk-top applications. I like my iPad, but only because so many dedicated souls have given some thought to the user experience when creating their apps. I often meet with disappointment when I encounter interactive displays when I'm out and about during the day. It is 2012, and it seems that there are a lot of application designers and developers who have never read Don Norman's The Design of Everyday Things!
I enjoy making working prototypes and demo apps, but my skill set is stuck in 2008, the last year I took a graduate-level computer course. I was thinking about taking a class next semester, something hands-on, creative, and also practical, to move me forward. I can only do so much when I'm in the DIY mode alone in my "lab" at home. I need to explore new tools, alongside like-minded others.
There ARE many more tools available to designers and developers than there were just four years ago. Some of them are available online, free, or for a modest fee. I was inspired by a link posted by my former HCI professor, Celine Latulipe, to her updated webpage devoted to Rapid Prototyping tools. The resources on her website look like a good place to start for people who are interested in creating applications for the "NUI" era. (Celine has worked many interesting projects that explore how technology can support new and creative interaction, such as Dance.Draw.) Below is her description of her updated HCI resources:
"New HCI resource to share: I have created a few pages on my web site devoted to Rapid Prototyping tools, books, and methods. These pages contain reviews of various digital tools, including 7 different desktop prototyping apps, and including 8 different iPad apps for wireframing/prototyping. I hope it's useful to others. Feel free to share... and please send me comments and suggestions if you find anything inaccurate, or if you think there is stuff that I should be adding. I will be continuing to update this resource." -http://www.celinelatulipe.com (click on the rapid prototyping link at the top)
IDEAS Below are just a few of my ideas that I'd like to implement in some way. I can't claim ownership to these ideas- they are mash-ups of what comes to me in my dreams, usually after reading scholarly publications from ACM or IEEE, or attending tech conferences.
An interactive timeline, (multi-dimensional, multi-modal, multimedia) for off-the-desktop interaction, collaboration, data/info analysis exploration. It might be useful for medical researchers, historians, genealogists, or people who are into the "history of ideas". Big Data folks would love it, too. It would handle data from a variety of sources, including sensor networks. It would be beautiful to use.
A web-based system of delivering seamless interactive, multi-modal, immersive experiences, across devices, displays, and surfaces. The system would support multi-user, collaborative interaction. The system would provide an option for tangible interaction.
A visual/auditory display interface that presents network activity, including potential intrusions, malfunctions, or anything that needs immediate attention that would be likely to be missed under present monitoring methods.
Interactive video tools for creation, collaboration, storytelling. (No bad remote controllers needed.)
A "wearable" that provides new ways for people to express and communicate creatively, through art, music, dance, with wireless capability. (It can interact with wireless sensor networks.)*
An public health application designed to provide information useful in understanding and sepsis prevention efforts.This application would utilize the timeline concept describe at the top of this list. This concept could also be useful in analyzing other medical puzzles, such as autism.
Most of these ideas could translate nicely to educational settings, and the focus on natural user interaction and multi-modal i/o aligns with the principles of Universal Design for Learning, something that is important to consider, given the number of "at-risk" learners and young people who have disabilities.
I welcome comments from readers who are working on similar projects, or who know of similar projects. I also encourage graduate students and researchers who are interested in natural user interfaces to and move forward with an off-the-desktop NUI project. I hope that my efforts can play a part in helping people make the move from GUI to NUI!
Below are a few videos of some interesting projects, along with a list of a few references and links.
Overview One of the primary goals of teaching is to prepare learners for life in the real world. In this ever-changing world of technologies such as mobile interaction, cloud computing, natural user interfaces, and gestural interfaces like the Nintendo Wii and Microsoft Kinect, people have a greater selection of tools for the task at hand. Given the potential of these new interfaces, software, and technologies as learning tools, as well as the ubiquitous application of interactive technology in formal and informal learning environments, there is a growing need to explore how next-generation technologies will impact education in the future.
As a community of Human-Computer Interaction (HCI) and educational researchers, we need to theorize and discuss how new technologies should be integrated into the classrooms and homes of the future. In the last three years, three CHI workshops have provided a forum to discuss key issues of this sort, particularly in the context of next-generation education. The aim of this special issue of Personal and Ubiquitous Computing is to summarize the potential design challenges and perspectives on how the community should handle next-generation technologies in the education domain for both teachers and students.
We invite authors to present position papers about potential design challenges and perspectives on how the community should handle the next generation of HCI in education. Topics of interest include but are not limited to:
Submissions should be prepared according to the Word template located at the bottom of this page. All manuscripts are subject to peer review. Manuscripts must be submitted as a PDF to the easychair submission system. Submissions should be no more than 8000 words in length.
Guest Editors and Contact Information
Syed Ishtiaque Ahmed, Cornell University
Quincy Brown, Bowie State University
Jochen Huber, Technische Universität Darmstadt
Si Jung “Jun” Kim, University of Central Florida
Lynn Marentette, Union County Public Schools, Wolfe School
Max Mühlhäuser, Technische Universität Darmstadt
Yesterday evening I attended a meetup of TechCrunch and Charlotte-area techies, held at the uptown Packard Place entrepreneurial center. It was jammed-packed with people all abuzz with tech start-up fever, fueled by awesome food provided by Zen Fusion. Although my main purpose for attending the TechCrunch meet-up was to learn more about innovative technology start-ups in my region, I also was hoping to capture a few shots of interesting people. I like to keep my eye open for tee-shirt slogans, and one worn by a young gentleman caught my eye, proclaiming that he'd seen the future, and it is in his browser. On the back of his tee-shirt was a bright HTML5 logo, something that is dear to my heart, as I am moving from HTML4 to HTML5. He was polite and agreed to pose for a couple of photographs:
It turned out that the HTML5 guy was at the TechCrunch event with one of his colleagues from T1 Visions, a social touchscreen solutions company that I've featured previously on this blog. They caught me up on the growth of this start-up company, which now has 15 employees and has broadened its reach beyond table-top restaurant applications to the healthcare, education, corporate, retail, and broader hospitality sectors.
What I like about table-top systems is that they provide support for "natural user interaction". It allows for multiple modes of interaction with and presentation of multimedia content. Over the past several years, these systems have been proven to be useful to a wider range of people and settings. Interfaces that support touch and gesture interaction are no longer viewed as novelties, given the pervasiveness of touch-phones and tablets and their ease-of-use for most people.
A useful product from T1 Visions is the T1 Collaboration Table. It supports touch-screen interaction and can also handle up to four simultaneously connected laptops. The table system provides a media viewer that supports sharing of photos across screens, devices, and surfaces. It also contains a web browser, a presentation viewer, and a whiteboard that is compatible with video conferencing. The company provides customized applications for its clients. In the Charlotte area, some of the tables can be found in restaurants, such as the Mellow Mushroom, Cowfish, and Harpers. A few were recently installed in the Atkins library at UNC-Charlotte, to support group-work among students.
To learn more about what T1 Visions has to offer, take a few minutes to view the following videos and follow the links at the end of this post!
Demonstration of how the collaboration table can work within a business environment:
NOTE: TechCrunch is a technology media group founded in 2005 that focuses on innovative technologies. This summer, a group of TechCrunchers are visiting cities in the south that were previously not under their radar, such as Savannah and my home region, Charlotte, N.C. The Charlotte TechCrunch meetup was held on Wednesday, July 11, 2012. I plan to devote a few more blog posts to share what I learned.
I've been following a number of people that have been working in the area of natural user interfaces and interaction for many years. An example of this work is NUITEQ, a company lead by Harry van deer Veen. Below is NUITEQ's most recent show reel of Snowflake Suite, an off-the-shelf multitouch SDK.
Here is the description of the software from the naturaluserinterface YouTube channel:
"NUITEQ's award-winning multitouch software product Snowflake Suite comes off the shelf with 30+ apps, a free SDK to develop your own multitouch software apps and its content is easy to customize. The solution is offers high performance, stability, quality and comes with dedicated support. Apps includes presentation, productivity and creativity tools as well as games. The software can be used in different scenarios such as corporate presentations, exhibitions, entertainment, education, public spaces, consumer electronics, retail and digital signage."
FYI: Tutorials about the user of Snowflake Suite can be found on the naturaluserinterface YouTube channel.
Harry van der Veen has been sharing his NUI journey journey since 2007 on his Multitouch blog.
I came across the following video about nsquare's presenter application. It looks like it has potential. It supports "seamless" collaboration between people, multimedia content, interactive tables, interactive touch screens, and tablet devices.
This will be our third annual workshop in conjunction with CHI 2012.
One of the primary goals of teaching is to prepare learners for life in the real world. In this ever changing world of technologies such as mobile interaction, cloud computing, natural user interfaces, and gestural interfaces like the Nintendo Wii and Microsoft Kinect, people have a greater selection of tools for the task at hand. Teachers and students can leverage these tools to improve learning outcomes. Educational interfaces and software are needed to ensure that new technologies serve a clear purpose in the classrooms and homes of the future.
Since teachers are always looking for creative ways to engage 21st century learners, there needs to be an academic venue for researchers to discuss novel educational tools and their role in improving learning outcomes. This workshop aims at filling this void: combining the pedagogical expertise of the cooperative learning, and learning sciences communities with the technical creativity of the CHI, UIST and interactive surface communities. The objective of this workshop is to become a conference within two years
We invite authors to present position papers about potential design challenges and perspectives on how the community should handle the next generation of HCI in education.
Submission: The deadline for workshop paper submissions is Dec 20, 2011. Interested researchers should submit a 4-page position paper in the ACM CHI adjunct proceedings style to the workshop management system. Acceptance notifications will be sent out February 20, 2012. The workshop will be held May 5-6, 2012 in Austin, Texas. Please note that at least one author of an accepted position paper must register for the workshop and for one or more days of the CHI 2012 conference.
