I like this demonstration of Adam Somers AirHarp music application for use with the Leap Motion 3D controller:
AirHarp is being developed in C++ using Adam Somer's audio processing toolkit, MusKit. This looks interesting! Things have changes since I last took a computer music technology course (back in 2003). Adam Somers is a senior software engineer at Universal Audio. He has a graduate degree in music technology from Stanford, and a background in computer science, electronics, human-computer interaction, and signal processing. Leap Motion is a motion-control software and hardware start-up company located in San Francisco, California. According to promotional information from the website, the company's first product, the Leap Motion controller, is 200 times more sensitive than existing technologies. It will be interesting to see how this plays out. (I'm still waiting for my pre-order.) RELATED AirHarp (links to GitHub) Leap FAQs Leap Motion Website Leap Motion Developer Portal Leap Motion Leadership Team Leap Motion goes retail: Motion controller sold exclusively at Best Buy Michael Gorman, engadget, 1/16/13
Here's an interesting use of technology for health - the Xbox Kinect in the OR!
Thanks to Harry van der Veen for the link! RELATED Kinect sensor poised to leap into everyday life Niall Firth, NewScientist, 1/17/13 For the tech-curious: PrimeSense(Company that developed the 3D depth sensor that powers the Kinect, the sensor in Ava, a healthcare robot by iRobot, and more.) OpenNI (Framework for the development of 3D sensing middleware libraries and applications.) NiTE: Natural Interface Technology for End User (Perception algorithms layer for 3D computer vision, allows for hand locating, tracking, analyzing scenes, and tracking skeleton joints.)
One of the things I like to do is share updates about the world of multimedia, multi-touch, gesture, screen, surface, and interactive technologies, focusing on off-the-desktop applications and systems. When I started this blog, I had to put forth quite a bit of effort just to FIND interesting things to blog about.
These days, there are so many sources that focus on emerging - and now commonplace- interactive technologies, my main challenge is to filter the noise. Where do I begin?
My archives are vast. I randomly picked the year 2009 and came across one of my previous posts, "News, Videos, and Links about Multitouch and Screen Technologies." The post is long, and contains a number of videos and links that probably will be of value to a future curator of the history of technology.
I welcome comments from readers who might be able to help me update information about various applications and systems I've featured on this blog in the past.
The pictures are screenshots from the results of an image search for "interactivemultimediatechnology". Over the past 6 years, I've posted quite a few!
It just might be the right time for everyone to brush up on 21st century tech skills. iPads and touch-phones are ubiquitous. Touch-enabled interactive whiteboards and displays are in schools and boardrooms. With Microsoft's Windows 8 and the news that the company recently acquired Jeff Han's company, Perspective Pixel, I think that there will be good support - and more opportunities- for designers and developers interested in moving from GUI to NUI.
In the video below, from CES 2012, Jeff Han provides a good overview of where things are moving in the future. We are in a post-WIMP world and there is a lot of catching up to do!
CES 2012 Perceptive Pixel and the Future of Multitouch (IEEE Spectrum YouTube Channel)
During the video clip, Jeff explains how far things have come during the past few years:
"Five and
1/2 years ago I had to explain to everybody what multi-touch was and meant. And
then, frankly, we've seen some great products from folks like Apple, and really
have executed so brilliantly, that everyone really sees what a good
implementation can be, and have come to expect it. I also think though, that the explosion of
NUI is less about just multi-touch, but an awareness that finally
people have that you don't have to use a keyboard and mouse, you can demand
something else beside that. People are
now willing to say, "Oh, this is something I can try, you know, touch is
something I can try as my friendlier interface"."
Who wouldn't want to interact with a friendlier interface? Steve Ballmer doesn't curb his enthusiasm about Windows 8 and Perceptive Pixel. Jeff Han is happy how designs created in Windows 8 scales for use on screens large and small. He explains how Windows 8 can support collaboration. The Story Board application (7:58) on the large touchscreen display looks interesting.
I continue to be frustrated by the poor usability of many web-based and desk-top applications. I like my iPad, but only because so many dedicated souls have given some thought to the user experience when creating their apps. I often meet with disappointment when I encounter interactive displays when I'm out and about during the day. It is 2012, and it seems that there are a lot of application designers and developers who have never read Don Norman's The Design of Everyday Things!
I enjoy making working prototypes and demo apps, but my skill set is stuck in 2008, the last year I took a graduate-level computer course. I was thinking about taking a class next semester, something hands-on, creative, and also practical, to move me forward. I can only do so much when I'm in the DIY mode alone in my "lab" at home. I need to explore new tools, alongside like-minded others.
There ARE many more tools available to designers and developers than there were just four years ago. Some of them are available online, free, or for a modest fee. I was inspired by a link posted by my former HCI professor, Celine Latulipe, to her updated webpage devoted to Rapid Prototyping tools. The resources on her website look like a good place to start for people who are interested in creating applications for the "NUI" era. (Celine has worked many interesting projects that explore how technology can support new and creative interaction, such as Dance.Draw.) Below is her description of her updated HCI resources:
"New HCI resource to share: I have created a few pages on my web site devoted to Rapid Prototyping tools, books, and methods. These pages contain reviews of various digital tools, including 7 different desktop prototyping apps, and including 8 different iPad apps for wireframing/prototyping. I hope it's useful to others. Feel free to share... and please send me comments and suggestions if you find anything inaccurate, or if you think there is stuff that I should be adding. I will be continuing to update this resource." -http://www.celinelatulipe.com (click on the rapid prototyping link at the top)
IDEAS Below are just a few of my ideas that I'd like to implement in some way. I can't claim ownership to these ideas- they are mash-ups of what comes to me in my dreams, usually after reading scholarly publications from ACM or IEEE, or attending tech conferences.
An interactive timeline, (multi-dimensional, multi-modal, multimedia) for off-the-desktop interaction, collaboration, data/info analysis exploration. It might be useful for medical researchers, historians, genealogists, or people who are into the "history of ideas". Big Data folks would love it, too. It would handle data from a variety of sources, including sensor networks. It would be beautiful to use.
A web-based system of delivering seamless interactive, multi-modal, immersive experiences, across devices, displays, and surfaces. The system would support multi-user, collaborative interaction. The system would provide an option for tangible interaction.
A visual/auditory display interface that presents network activity, including potential intrusions, malfunctions, or anything that needs immediate attention that would be likely to be missed under present monitoring methods.
Interactive video tools for creation, collaboration, storytelling. (No bad remote controllers needed.)
A "wearable" that provides new ways for people to express and communicate creatively, through art, music, dance, with wireless capability. (It can interact with wireless sensor networks.)*
An public health application designed to provide information useful in understanding and sepsis prevention efforts.This application would utilize the timeline concept describe at the top of this list. This concept could also be useful in analyzing other medical puzzles, such as autism.
Most of these ideas could translate nicely to educational settings, and the focus on natural user interaction and multi-modal i/o aligns with the principles of Universal Design for Learning, something that is important to consider, given the number of "at-risk" learners and young people who have disabilities.
I welcome comments from readers who are working on similar projects, or who know of similar projects. I also encourage graduate students and researchers who are interested in natural user interfaces to and move forward with an off-the-desktop NUI project. I hope that my efforts can play a part in helping people make the move from GUI to NUI!
Below are a few videos of some interesting projects, along with a list of a few references and links.
Yesterday evening I attended a meetup of TechCrunch and Charlotte-area techies, held at the uptown Packard Place entrepreneurial center. It was jammed-packed with people all abuzz with tech start-up fever, fueled by awesome food provided by Zen Fusion. Although my main purpose for attending the TechCrunch meet-up was to learn more about innovative technology start-ups in my region, I also was hoping to capture a few shots of interesting people. I like to keep my eye open for tee-shirt slogans, and one worn by a young gentleman caught my eye, proclaiming that he'd seen the future, and it is in his browser. On the back of his tee-shirt was a bright HTML5 logo, something that is dear to my heart, as I am moving from HTML4 to HTML5. He was polite and agreed to pose for a couple of photographs:
It turned out that the HTML5 guy was at the TechCrunch event with one of his colleagues from T1 Visions, a social touchscreen solutions company that I've featured previously on this blog. They caught me up on the growth of this start-up company, which now has 15 employees and has broadened its reach beyond table-top restaurant applications to the healthcare, education, corporate, retail, and broader hospitality sectors.
What I like about table-top systems is that they provide support for "natural user interaction". It allows for multiple modes of interaction with and presentation of multimedia content. Over the past several years, these systems have been proven to be useful to a wider range of people and settings. Interfaces that support touch and gesture interaction are no longer viewed as novelties, given the pervasiveness of touch-phones and tablets and their ease-of-use for most people.
A useful product from T1 Visions is the T1 Collaboration Table. It supports touch-screen interaction and can also handle up to four simultaneously connected laptops. The table system provides a media viewer that supports sharing of photos across screens, devices, and surfaces. It also contains a web browser, a presentation viewer, and a whiteboard that is compatible with video conferencing. The company provides customized applications for its clients. In the Charlotte area, some of the tables can be found in restaurants, such as the Mellow Mushroom, Cowfish, and Harpers. A few were recently installed in the Atkins library at UNC-Charlotte, to support group-work among students.
To learn more about what T1 Visions has to offer, take a few minutes to view the following videos and follow the links at the end of this post!
Demonstration of how the collaboration table can work within a business environment:
NOTE: TechCrunch is a technology media group founded in 2005 that focuses on innovative technologies. This summer, a group of TechCrunchers are visiting cities in the south that were previously not under their radar, such as Savannah and my home region, Charlotte, N.C. The Charlotte TechCrunch meetup was held on Wednesday, July 11, 2012. I plan to devote a few more blog posts to share what I learned.
I recently learned more PO-MO, a relatively new start-up tech company based in Winnipeg, Canada. According to the company's information, PO-MO "specializes in interactive digital display solutions, including gesture and motion based interactivity, interactive display content creation and management, and large interactive display and projection services for advertisers, educators, and events."
Po-Motion was afinalist in an elevator pitch video contest last fall. It has several advantages over potential competitors. The system is easy to use, and priced within the range that is affordable for schools, museums, and other cost-conscious groups who would like to provide technology-supported immersive interactive experiences for people of all ages. The PO-MOtion software designed for interactive floors and walls starts at $39.99, and works on any computer, using any USB web camera and a projector. Other applications make use of Kinect sensors.
I especially like one of PO-MO's recent projects, the Impossible Animals Museum Exhibit, created using Unity 3-D, for the Manitoba Children's Museum. How does it work? Children create a colored egg using crayons and paper, which is then scanned into the exhibit and digitally embedded into the system, which includes an interactive wall and floor. When the egg is touched, it is activated to hatch, and then becomes a motion reactive animal. The environment includes things like water, landscapes, and even a spaceship. The system has a "reset world" button for museum staff to use when needed.
The following video explains how the PO-MO system works:
PO-MO is also involved in promotional projects, assisting retailers, ad agencies, and brand managers with creative ways to engage customers and clients:
Other products and services provided by PO include mobile app development. I especially like the augmented reality business card depicted in the following video clip:
Imagine if your local shopping centers, museums, libraries, or even schools offered this level of immersive interaction on a regular basis!
RELATED The PO-MOtion system has a wide range of uses. It is currently used in an educational setting in a sensory room for students with special needs, something that I'd like to try out in the near future with students at Wolfe School. I plan to share more about this in another post.
"Meghan has been a professional designer and animator since graduating from Red River College in 1997. After graduation, she moved to Calgary, where she spent almost two years directing projects at Aurenya Studios, a start-up animation company. In 2001, Meghan was engaged by Community Connections to support community-based IT development projects in rural Manitoba and in Winnipeg’s inner city. In 2008, Meghan joined Manlab, developing educational interactive games and resources for Immigrate Manitoba. She also launched Meghan PO-MO Project, a sole proprietorship which provided sound reactive visuals for DJs and venues across Canada. In 2009, Meghan was contracted as the User Experience Designer at Tipping Canoe, a multinational internet marketing company.
In 2010, Meghan formed PO-MO Inc. in partnership with Curtis Wachs. She began working exclusively for the company in December, 2010. Today, Meghan is the driving force behind PO-MO Inc." Curtis Wachs – Technical Director/COO, PO-MO Inc. "Curtis graduated from Assiniboine Community College in 2003 where he studied object oriented programming. Directly upon graduating, Curtis was hired by Assiniboine Community College to help design and develop software for online classes. Curtis relocated to Winnipeg in 2006 to create interactive training material for sales staff at E.H. Price. During the course of his work, Curt was apprenticed in 3D modelling and animation by Liem Ngyuen, a former Frantic Films resident. In 2008, Curtis joined Manlab, where he created online educational games for Travel Manitoba, Immigrate Manitoba, and other clients. In 2010, Curtis formally joined PO-MO Project, and the company became a partnership. In June 2010, PO-MO Inc. was founded.
Curtis is currently the technical director at PO-MO Inc., overseeing the project management and workflow of contracted and R&D development projects."
The above video is an overview of the "5D" connected retail experience platform by Razorfish Emerging Experience. This concept looks like it was designed for me - someone who loves tech, has a high need for hassle-free shopping. Someday I hope I will have the ultimate technology-supported shopping experience : )
FlatFrog Multitouch is a company based in Sweden. It was founded by Ola Wassvic and Christer Fåhraeus. The technologies support 20+ simultaneous touches, and recognize object size, a useful feature. FlatFrog screens can be optimized for a wide range of light conditions FlatFrog's multi-touch and gesture interaction is featured in the short video clips below.
FlatFrog is gearing up for commercial release. According to the FAQ's on the website, "all sizes are possible, from 5" to 100" and upward. Promethean is one of the company's investors. There is a volume manufacturing agreement with Kortek Corporation, known for industrial and gaming displays.
Thanks Touch User Interface for sharing this information!
(Touch User Interface is the blog for Sensible UI, known for the ArduMT, aka the Arduino Multi-touch Development Kit)
If you are a long-time reader of this blog, you probably know that I sometimes have some unusual dreams about technology. I don't blog about my dreams very often, but last night, I had another technology dream, a continuation of a dream I had one night last week. Voronoi Diagram (Wikipedia) I'm pretty sure that the last two dreams were sparked by playing an online interactive demonstration of a Voronoi application before going to sleep one night, and also reading an article about "extracting ordered patterns from a triangular mesh surface" in the Novemeber/December IEEE Potentials magazine before turning in last night.
The dance probably was influenced by my recent viewing of the North Carolina Dance Theater's performance of Innovative Works with my mom, someone who encouraged my study of music, art, and dance at an early age.
Some of my tech dreams are sort of...practical. For example, in one recurring dream, I find myself coding for a flexible mesh/grid application. Sometimes the mesh/grid has something to do with wireless sensor networks on curved terrain, perhaps related to something like the Smart Grid, and sometimes I find myself working on an application that analyzes streaming data from a variety of sources, for security prediction purposes. At other times, I'm coding for something more artistic, my preference.
Last night, my dream focused on creating a flexible mesh fabricthat used in a multimedia dance/graphic arts/music performance. I was coding for this performance using a Voronoi-like algorithm.
This is the best I can do to explain this: The fabric is carried by the dancers, and is both reactive and generative. In essence, the fabric is intertwined/embedded in the dance, the music, and the graphics. In my dream, everything looked/sounded/felt awesome and otherworldly, and the music that merged and morphed during the dance was so beautiful, not only the melodies, but the sounds. (In a previous dream, the mesh contained a "nanotechnology" component, but I'll save that quest for the future.)
I thought I'd look at some of my web book marks and search a bit more for information related to this topic. For now, here is the "brain dump". I have more to add to this post, and plan to port it to a reference page for this blog in the future. I hope that this post will be useful to some of my art/music/dance/tech readers!
RELATED AND SOMEWHAT RELATED
Update: Right after uploaded this post, I came across a link to a WebGL demo for a 3D music video of pop singer Ellie Goulding's song, 'Lights', by HelloJoy. Visitors to the webpage can click to interact with the environment. If you keep the button pressed, you fly faster. If you tweet the link, you'll see your name crop up as you fly around in the soundspace. For more information about the making of 'Lights' - take a look at Behind the scenes of 'Lights": the latest WebGL sensation! (Carlos Ulloa, 11/9/11)
Patterns in the Noise (Nathan Nifong's site - FYI, Nathan worked with Celine Latulipe with the Dance.Draw project while completing his bachelor degree in computer science at UNC-Charlotte) Voronoi Dance (Christian Gross, using OpenFramework) Voronoi art: Slow Trip (Oktalist/Mat)
The above video, by Mat/Oktal, was inspired by his viewing of Thomas Ruff's Substrat images. Scott Snibbe Studio (Intearctive art, music, and animation for iPhone, iPad, iPod, and Mac)
Interactive Voronoi Diagram Generator with WebGL (Alex Beutel) The above video was found in Alex Beutel's blog post, "Interactive Voronoi Diagrams with WebGL"
Jim Spadaccini is the director of Ideum and is responsible for the Ideum blog, which is well worth reading if you are interested in creative use of multi-touch, gesture, and interactive multimedia applications. Jim also serves as the principal investigator of the Open Exhibits project, a non-profit open source, multi-touch, multiuser software initiative, which includes a free software development kit (SDK) for museums, students, nonprofits, and educational use.
More about IDEUM Ideum will be involved in an upcoming day-long conference at the Tech Museum on Tuesday, November 15th (2011), Building Interfaces for the New Decade ConferenceSan Jose, California, 11/15/11
Ideum is working on interconnectivity between devices, as shown below, where people are connecting their iPhones to a multi-touch table:
Here is a video about Ideum's Tiny Drifters exhibit, 7 foot multi-touch wall at the Monterey Bay Aquarium:
Here is a promotional video about Ideum's rugged MT-55 multi-touch table. It holds up to the interaction of thousands of museum visitors.
I'd like to share the on-line demo of MindHabit's suite of serious games that I've found useful in my work with teens and young adults who need support in the area of social-emotional skills.
What I like about the online demo is that it adjusts to the player's responses. This feature made it fun to use during the last few social skills groups I facilitated at work, since it could be played by students with a range of cognitive abilities. I had students take turns playing the game using a SMARTboard, and found that all of the students paid attention to what was going on. In my opinion, using the interactive whiteboard supported "off-the-shoulder" learning among the students who were not at the board.
MindGames is available for Windows and Macs, and the full version is just $19.99 and provides 100 game levels. The full version tracks progress and includes four games.
Here's some information from the company's website: "Based on social intelligence research conducted at McGill University, these stress busting, confidence boosting games use simple, fun-to-play exercises designed to help players develop and maintain a more positive state of mind." "Based on the principles of social intelligence: Inhibition - uses game mechanics to promote positive habits; Association - connects personal info to positive feedback; Activation - uses personal references"
You are playing the MindHabits Trainer online demo. Your progress will not be logged beyond this session.
Interview with Guillaume Largillier about Stantum's multi-touch tablet.
Collaboration with Stevens Institute of Technology, focusing on a serious game project to support learning of "on-the-job" social skills for teens and young adults with autism spectrum disorders and related challenges.
More news about large interactive displays, multi-touch, and gesture applications/installations.
I had the pleasure of visiting the Levine Museum of the New South with my daughter, a history buff, and my 7 month old grandbaby. It was his first trip to a museum, and he loved it - the pictures, the hands-on exhibits, and of course, the multimedia technology. Future historian? Museum curator? Digital archivist? We'll see...
We had a chance to explore the Cotton Fields to Skyscrapers exhibit and Whirlwind of Growth, updated to take a look at the past decade and how recent events, such as rapid population growth in the region (69% between 1990, the year I moved to Charlotte, and 2010), recession and financial crisis, have impacted the Charlotte region and the Carolinas.
I especially liked the New South TalkBack part of the exhibit. I think the TalkBack feature was originally part of the Changing Places exhibit, a multi-part project that focuses on how people in the Charlotte region deal with the growing cultural diversity and steady stream of newcomers who come from just about everywhere, and how newcomers experience their transition to the area. The exhibit was unveiled in February of 2009, and due to positive feedback, was extended well past the planned ending date. The exhibit's impact was extended to the virtual world through the Changing Places Project website, as well as the museum's YouTube channel. Both are great places to explore.
I'd like to share a bit more about the Levine Museum of the New South "experience" from content related to the Changing Places: From Black and White to Technicolor® exhibit:
This overview video of the Changing Places exhibit at the Levine Museum provides a good dose of civic pride. I've lived in the Charlotte area since 1990 and have come to appreciate the rich tapestry that makes up our community, reflected in the short stories (and songs!) shared in the visitors' video clips.
Changing Places "Our History":
(I especially liked the uplifting background music in this video.)
Changing Places: "The Out of Towners" is a mashup of clips created by visitors to the Levine Museum who come from other places in the US and around the globe:
For more videos from Levine Museum visitors, take a look at the Levine Museum's YouTube channel. The videos are much better than what I took using my cell phone video cam, below:
New South Talkback, Levine Museum of the New South
What I didn't like so much...
Below is a video of a multi-touch picture/info display, part of the exhibit. I didn't like it very much, as the screen was very cluttered and the content was "jumpy" at the slightest touch. Given my interest in large displays, multi-touch, and gesture interaction, my expectations were pretty high:
REMIX HISTORY .
Perhaps it is my civic duty to volunteer with a few other multi-touch techies to create a "redo" of this application!
RELATED The following links are to some of the organizations/people who work or have worked in partnership with the Levine Museum of the New South: Darcie Fohrman, Exhibit Developer and Designer, Changing Places Irene Morris , Graphic Design, Changing Places Dr. Pamela Grundy, curator, Changing Places Dr. Tom Hanchett, assistant curator, Changing Places Sarah Bryan, researcher, Changing Places New Granada Productions, edited video footage from the Talkback Booth at the Levine Museum, spanning the course of 14 months. Studio Displays, Inc. (Changing Places) Brad Larson Media "Using technology to encourage family learning in museums, zoos, and other public places" (Changing Places) Community Building Initiative Emulsion Arts, Film production, Changing Places Luquire George Andrews, Changing Places (PR, brand strategy, media planning, digital solutions, located in Charlotte, NC) Nancy Pierce, Photographer, Changing Places Michael Daul, Kaplan & Associates Cultural Resources, Online Curator, Website Design and Development, Changing Places
Video preview of the new section of the exhibit, including some shots of "Remix History":
