I came across the picture below today, and discovered more about it in a CNET article:
"Gesture-recognition technology, such as that seen in Project Natal, has changed the face of gaming, but Microsoft's Craig Mundie (pictured) believes it will also transform the office." (Ina Fried, CNET)
This approach looks like it incorporates visual thinking and conceptualization in a nice and fluid way.
According to Fried, author of the the linked CNET article, Craig Mundie, at the Microsoft Research Faculty Summit this past July demonstrated the set-up in the above picture. The demonstration "...included hologram-like videoconferencing, a virtual digital assistant, and multiple surface computers, along with voice-, touch- and gesture-recognition technology. The desk in the demo was a multitouch surface computer, and the office's walls were also a display that could easily switch from being a virtual window or collection of digital photos to a corkboard of sticky notes or various workspaces. In one case, Mundie also used Project Natal-like depth cameras to put himself in the middle of an architectural demo, essentially putting himself inside a building that was not yet built."
The author of the above article was also at the Microsoft Research Faculty Summit and touched on the concept of natural user interfaces:
“All the things we talk about as natural user interfaces have been largely used one at a time as enhancements to [graphical user interfaces],” Mundie said. Gesture recognition, expressive responses, immersive 3-D virtual environments, and understanding of context—these advances in computing algorithms will lead to software that is “better at anticipating what you might want.”
When I find a video of the Project Natal/Surface/Natural User Interface Office in action, I'll be sure to post it!
Why this is important:
Natural user interfaces might have the potential to support more efficient and effective collaboration among people in the workplace, and the visual display of information may also support better decision-making and problem solving.
As I set out to revisit some of my early posts on this blog, I came across a link to a review the Flower game, created for the Playstation 3 by Jenova Chen, Kellee Santiago, and other members of thatgamecompany.It is good to know that this group continues to work towards the goal of creating innovative games that provide gaming experiences that are calming, pleasant to use, and are unlike most game genres.
Flower (Credit: IGN)
"Supported HD Output: 720p, 1080i, 1080p- From the same developers that brought you the serene flOw comes Flower, whose gameplay has players guiding a flower petal across gorgeously rendered fields and plains, replete with individually modeled leaves of grass that blow and sway in the wind. Colorful, peaceful, and exhillerating, Flower is a difficult game to describe, and simply must be experienced in all it’s HD beauty to truly be appreciated. A real gem."
(Flower is played with the Sixaxis wireless game controller that can sense motions and provide precision in interactive game play. The Sixaxis was recently replaced by Duoalshock 3, which provides motion sensing and rumble.I'm not sure if the new controller works with Flower.)
The following video review by Pete Dorr won't give enough details about the game to spoil it for you. Go ahead and watch the review!
Since I don't have a PS3 (yet), I'm sharing a couple of reviews of Flower that I found on the IGN website:
"thatgamecompany's mellow, beautiful adventure through the dreams of plants starts out so disarmingly pleasant that one would be forgiven for assuming it's just a game about flying around and picking up flower petals. It's not until one experiences the couple-hour experience from beginning to end (preferably with 5.1 audio, the lights off and no human distractions) that the game really gets its emotional hooks into you. Without ever introducing a single line of dialogue, any characters (at least in the traditional sense) or clear goals, Flower takes you on a ride that's over far too soon, yet somehow seems just right for what it's trying to do: suck you into a world without the usual video game trappings while playing with your emotions in a way no amount of manufactured tear-jerk dialogue or surprise deaths ever could. It really is unlike anything else you'll ever play and stands as one of the best (if not the best) examples of SixAxis motion controls on the PS3."
"thatgamecompany has depicted something that I never once imagined: what would a flower's dream look like if we could see it?Flower, a PS3 downloadable that comes as a spiritual successor to flOw, is one of the most beautiful games that I've ever played. Not just because the visuals are entirely breathtaking, but also because the experience of playing it offers more enjoyment, emotion and enlightenment than any game I've tried in years."
"Before you read on, keep in mind that Flower isn't quite for everyone. There will be those that experiment with it but won't see the point, as it strays so far from the traditional spectrum of gameplay systems, but it's truly worth playing...Flower is a poem, where you as the player participate in the dreams of flowers confined to the ever-changing landscape of a colorless city. Each dream, or stage, revolves around a different theme and gameplay objective, but your method of control remains the same: tilt the controller to direct a series of flower petals through beautiful environments. Press any button to stir up a wind that will coax them along. As you begin with one flower petal, you'll soon touch other flowers and gather more petals, generally influencing the environment in a way that pleases the dreamer. This setup works amazingly well to create varying tasks for you to enjoy..." Flower was listed as one one of the best games for the Playstation 3 for HDTV, along with a flOw, a 2006 game from the same company:
flOw "Supported HD Output: 720p, 1080i, 1080p- Calm, serene, and beautiful in it’s simplicity, flOw was one of the first PSN titles released when the Playstation 3 launched in November 2006, and it’s still a gorgeous work of art. From the calming music, to the simple gameplay, flOw proves that you don’t need high resolution textures and thousands of animations to produce a beautiful image."
"FlOw is a game about piloting an aquatic organism through a surreal biosphere where players consume other organisms, evolve, and advance their organisms to the abyss."
I came across Cloud about four years ago, when I was looking for non-violent games that could support relaxation and stress-reduction. Cloud was created by some of the members of thatgamecompany when they were students at UCS.
I later posted about how I used the game in in my work as a school psychologist:
Cloud is a relaxing, nonviolent game, created by graduate students at USC. The character, a child, flies around, collecting clouds and putting them into puzzles in the sky. The music is very soothing. The game is free and can be downloaded through the website. This game could be used to help relax students with anxiety disorders, Asperger syndrome, etc., develop coping strategies. Here is the link to the Cloud video trailer.
(I use the Cloud Game as a form of crisis intervention with students undergoing high level of agitation and stress at the middle and high school level.)
Below is a video of the Cloud game, which was designed for use on a computer. It still appears to be available for free from the USC website.
"being completely involved in an activity for its own sake. The ego falls away. Time flies. Every action, movement, and thought follows inevitably from the previous one, like playing jazz. Your whole being is involved, and you're using your skills to the utmost."
The game went on to become an award-winning PS3 game. The early web-based version of flOw is free, and some of the gameplay can be done using your finger on a touch screen. The music is relaxing, almost hypnotic.
Why this is important:
There is a need for games that can be used for relaxation, as most commercial games focus on action and thrills, often delivered through violence.
(Pictures and videos are from the IGN website, YouTube, and related websites.)
I took a look at an on-line community, Dance-Tech.NET which focuses on the "interdisciplinary explorations on the performance of motion". I was happy to find that two people I know are members of this vibrant group of people:
Celine Latulipe
Dr. Celine Latulipe is a Human-Computer Interaction researcher at the University of North Carolina-Charlotte. She is the lead in the Dance.Draw project, a collaboration between the Software and Information Systems department, the Department of Dance, and a digital artist.
As Celine puts it, "You are more than your eyes and one hand. Why should you have to be less than you are when confronted with a digital device? Exquisite interaction is a collection of research projects...that aim to enrich your expressive creativity in the digital realm by allowing you and your collaborators to use more of your body in that digital interaction"
Doug Foxcreated and maintains theGreat Danceand Kinetic Interface blogs. When he was 42, he decided to study dance - modern, ballet, and jazz. He is interested in the intersection of dance and technology, and more recently became involved in the study of animation related to dance and movement.
The following is from Doug's "About" page - the videos and links are worth exploring:
"As a starting point, I'd like to encourage readers to visit the Movement Is at the Heart of Scientific and Technological Change background page. Here you will find 16 videos (plus links to more videos) that show in very concrete terms how new computer interfaces and digital devices being used in a range of fields are, in essence, body-centric and movement-centric..."
Contest winners were paired with choreographers, who created a new dance work based on a peer-reviewed article in a scientific journal. (The Science Dance Match-Up Challenge)
I just had to post the videos here - you can read detailed descriptions about each video on YouTube:
The role of Vitamin D in beta-cell function
Cerebral activation patterns induced by inflection of regular and irregular verbs with positron emission tomography: A comparison between single subject and group analysis
A Molecular Dance in the Blood, Observed
Popular Choice: Physics Tango "Single Molecule Measurements of Protelomerase TelK-DNA Complexes."
The four videos generated by the professional choreographers can be accessed on Vimeo.
Links to the scientific articles, the bios of the choreographers and scientists, and videos of the choreographer's renditions of the scientist's work can be found on the Science/AAAS website:
The Gonzo Scientist: The Science Dance Match-Up Challenge John Bohannon, Science, 4/17/09 SOMEWHAT RELATED Usually I reserve this section for links and information from external sources, but this time, I thought I'd share a few of my opinions that are somewhat related to this topic.
Why do I think interdisciplinary pursuits are important?
In my opinion, to move forward, the arts and other disciplines need to embrace the interdisciplinary way of thinking. There is much that is mixing and converging as I type these words. There is less emphasis for young people to pick one little corner of a field of study and make it their life's obsession. I have always had an "interdisciplinary" approach to life, ever since I can remember. I attribute this to my parents, who nurtured me as a musical, dancing, artistic child to pursue my talents at a young age, and when faced with choosing a college major, to go for a double major.
My interdisciplinary nature has fueled my journey into the world of technology, and my early background in the arts probably explains why I'm excited about interactive multimedia, extending into the realms of emmersive games, multi-touch and gesture interaction, and technology-supported interaction that takes place in larger public spaces.
Note:
As many of my readers know, I work as a school psychologist in my "day job". I DO miss the time when I was working part-time and taking graduate classes at UNC-Charlotte, but when the economy went downhill, It was necessary for me to return to work full-time.
The upcoming academic year will be busy! With the recent budget cuts to school districts in the in the state of North Carolina, I will have another school added to my schedule. I'm excited that it is a high school for technology and the arts, and that the school has a strong dance program!
I've posted quite a bit recently, since I have plenty to share. Soon I will only have time to post about 3 times a week.
"It is about how children can work together and communicate and how the multi-touch technology can facilitate this."
In this video, Tom Barrett demonstrates multi-touch networked tables, which were designed to be part of an immersive classroom environment. The multi-touch tables are the result of a collaboration between the Technology Enhanced Learning research group at Durham University and colleagues from the Education and Psychology departments.
Tom is a teacher and edublogger. Like me, he has a passion for multi-touch technology. He has been fortunate to have the chance to work with a SMARTTable in his classroom, and also compare the SMARTTable experience with the tables at Durham University, which were not from SMART Technologies. Tom was one of the first teachers to have the opportunity to try out the SMART Table in his classroom. He was a bit disappointed with the outcome. Even so, he believes that multi-touch technology will be important in education in the future.
From Tom's point of view, there is a need to have more in-depth content for the SMART Table, even at the earlier grades. The following quotes are taken from Tom's "SMART Table in my Classroom- My Conclusions" blog post:
"In my opinion there are three things that contribute to this: poor content; poor creation software and a straight jacketed approach to multi-touch functionality...There seems to be too much residual SMART Notebook thinking and not enough innovative software design. Maybe the product has preceded the necessary thinking behind it all."
"The one shining ray of light that emerges from amidst this all is the Media application. I have posted videos of some of my children working with this program in the past. It remains the only application that offers teachers and children an open environment to learn, and couples it with a unique interface with media. When you use this application you actually feel like you are using something innovative, multi-touch, gestural driven. As a teacher there is the capacity to use rich content of your choice (video) and then layer on top questions that engage the children in a much deeper way"
RELATED
Flickr Group: Multi-touch Interactive Desk: Applications and Gesture Ideas (Note: I have plenty of ideas for content and software design for multi-touch tables in education, and also cognitive and educational assessment. I'm only missing a table or two!)
The best part of this video is when the man is able to play a games (tic-tac-toe & rock, paper, scissors) with is daughter as he uses the vision device:
(The father depicted in the video is Erik Weihenmayer, who is a professional mountaineer and motivational speaker. He's the only blind person to have climbed the "Seven Summits", the highest points on each continent.) Photo courtesy of engadget
The system uses a head-mounted video camera which takes in visual information and translates it into electric patterns on a tab placed on the tongue in such a was that 3D representation is interpreted by the brain.
It should be noted that the BrainPort vision device is investigative and not yet available for sale.
So what happens when engineers and musicians get a chance to mix it up?
The Humanthesizer project is the result of a collaboration between musician Calvin Harris, and masters students from the Royal College of Art Industrial Design Engineering program.What makes this project innovative is that a special conductive ink, called Bare Ink, was used to support the creation of the music.
From the Bare Conductive website:
"Bare is a conductive ink that is applied directly onto the skin allowing the creation of custom electronic circuitry. This innovative material allows users to interact with electronics through gesture, movement, and touch. Bare can be applied with a brush, stamp or spray and is non-toxic and temporary. Application areas include dance, music, computer interfaces, communication and medical devices. Bare is an intuitive and non-invasive technology which will allow users to bridge the gap between electronics and the body." (Please overlook the bikini-clad women in the videos below, because the innovative technology supporting the Humanthesizer is worthy of the watch. Parents be forewarned!)
The Making of the Humanthezizer
Close to the finished version:
Information from YouTube regarding the Humanthesizer project:
"Calvin performs a version of his new single Ready For The Weekend on a unique human synthesiser. The instrument employs 15 bikini clad models painted with Bare Conductive, a new skin safe ink which conducts electricity. When the performers touch the connection completes a circuit, triggering a sound....The instrument consists of 34 pads on the floor which have been painted with the conductive ink and connected to a computer via some clever custom electronics. The performers stand on the pads, and touch hands to complete a circuit and trigger a sound. Different combinations of pads trigger the different sounds needed to play the track....Bare Conductive was developed by RCA students Bibi Nelson, Becky Pilditch, Isabel Lizardi and Matt Johnson. The custom electronics and software for the project were created by Matt Johnson, employing two Arduinos and the graphical audio programming tool Max MSP. The performers, floor pads, Arduinos and Max MSP combine to create a giant MIDI controller which is used to create music which is sequenced and quantized with Ableton Live." (Special thanks to: RCA / Bare Conductive - Bibi Nelson, Becky Pilditch, Isabel Lizardi, Matt Johnson - http://www.bareconductive.com)
(The Ready for the Weekend music video is available on YouTube. Parents: the music video has dancing girls throughout, in one-piece bathing suits.)
