Updated: Coding for Multimedia Interaction On ( and Around) Screens of All Sizes - with some interesting links

The last time I created an application for a mobile device was about four years ago.  It was in the olden days when most of us knew nothing about multi-touch interaction on any size or sort of display.  Soon after that, I focused my work on large displays, and tried not to get too distracted by the iPhone when it was introduced.

I can't ignore the iPad, as I think it holds a ton of potential for education and supporting people with special needs.

I can't ignore Android, since I have an Incredible. (I'm tethered to a never-ending family contract with Verizon.)

I have to figure things out during my summer break, right?  Brushing up on my "old" coding skills won't move me forward. Soooooo.......

On top of everything else* I'm trying to pack in,  I plan to take a peek at Sencha Touch  http://www.sencha.com/products/touch/, an HTML 5 mobile application framework that allows for Apple iOS and Google Android touchscreen devices.

I am searching for the magic solution that will allow the development of applications that allow for smooth interaction across screens of all sizes, from smartphones to huge displays.

Summer is a great time for dreaming and searching, but it is so short.  Time-saving advice from my readers would be greatly appreciated!  I'm also looking for good information about programming for stereoscopic 3D games for my series of posts about 3D innovations.

*What I'm trying to pack in:
Volunteering for the Society for Information Display, working on a SMARTTable app, exploring ideas for a multi-dimensional  interactive timeline, etc.

RELATED
Parallel Timelines


SOMEWHAT RELATED
FYI:  Recent posts on The World Is My Interface blog
WSJ's Interactive Graphics:  China's Housing Markets, via Innovative Interactivity
The First Self-replicating Cell with a Computer for a Parent:  Synthetic Life
More TED Talks:  Chip Conley's discussion about measuring what makes life worthwhile
Fractals in our world:  "I'm a mathemetician and I'd like to stand on your roof" - Ron Eglash on African Fractals

Coding for Multimedia Interaction On (and Around) Screens of All Sizes

Here is the updated version of post:

http://interactivemultimediatechnology.blogspot.com/2010/06/updated-coding-for-multimedia.html

Link - Women in Tech: Building Confidence and Visibility, via the Glass Hammer, by Melissa J. Anderson

 
Women in Tech: Building Confidence and Visibility
Melissa J. Anderson, the Glass Hammer, 6/15/10


The Glass Hammer is an on-line community "designed for women executives in financial services, law, and business."   I recently came across the Glass Hammer website when I followed a link to an interesting article written by Melissa J. Anderson, about women in technology.  Melissa Anderson, the author of the article, is the associate editor of the Glass Hammer, and is completing a Master's degree at NYU in Media, Culture, and Communication.

Nicki Gilmour, Publisher and Editor-in-Chief, is the woman behind this organization, with a background in the media industry. She is also the CEO of Evolved People Media.

You can follow Glass Hammer on Twitter, Mixx, Digg , Delicious, become a Glass Hammer friend on Facebook (http://www.facebook.com/theglasshammer)  or join groups on Facebook and LinkedIn.

Contributing writers at the Glass Hammer at the time of this post:

- Pamela Weinsaft (New York City)
- Elizabeth Harrin (London)
- Andrea Newell (Grand Rapids, Michigan)
- Liz O’Donnell (Boston)
- Natalie Sabia (New York City)
- Caroline Shannon (Dayton, Ohio)
- Jessica Titlebaum (Chicago)
- Tina Vasquez (Los Angeles)
- Gigi DeVault (Munich)

RELATED
Women in IT: The Facts
(Downloadable from the National Center for Women and Information Technology website)

Kinect Sensor for Xbox 360 Offers Full-Body and Gesture Interaction: No controllers or remotes!

Project Natal was the code name for the Kinect Sensor for Xbox 360. For $149.99 you can pre-order your very-own system from the Microsoft Store that will allow you to interact with video games with your body alone. No need for controllers or 'motes!

Presentation about the fitness benefits of the Kinect Sensor for Xbox 360:



This video is a preview of a dance game for the Xbox using the Kinect Sensor:


It would be great if I could do my Zumba moves with Kinect Sensor system and a great Xbox application!

Here's another video that explains the system in more detail, with brief interviews of innovators from Microsoft:


Here is a copy of my previous post about Project Natal:

How It Works: Microsoft's Project Natal for the Xbox 360 video from Scientific American


Microsoft gathered a wealth of biometric data to recognize the range of human movement in order to develop an algorithm for the next generation of controller-less gaming. "Natal will consist of a depth sensor that uses infrared signals to create a digital 3-D model of a player's body as it moves, a video camera that can pick up fine details such as facial expressions, and a microphone that can identify and locate individual voices."


The technology behind Natal has the potential for a range of uses beyond gaming.

Scientific American article:
Binary Body Double:  Microsoft Reveals the Science Behind Project Natal for Xbox 360

iRiS: iPhone Remote Interaction System, Newly Updated

"We used Touch Projector to paint on a multimedia facade (here: Ars Electronica Center, Linz, Austria). Users simply aim their device at the facade causing the viewfinder to show the facade locally. In this local image, users can now draw with the selected color. The building then changes its color accordingly as if users directly painted on it."


iRiS Team:

Sebastian Boring (lead) (LINK:http://www.medien.ifi.lmu.de/team/sebastian.boring/) 
Sven Gehring

Alexander Wiethoff  http://www.medien.ifi.lmu.de/team/alexander.wiethoff/
Magdalena Blöckner
Johannes Schöning http://www.johannesschoening.de




Info about the iRiS team members - (will be updated)
Johannes Schöning works at the Innovative Retail Laboratory of the German Research Centre for Artificial Intelligence (DFKI) in Saarbruecken.  His research interests include new methods and interfaces to intuitively navigate through spatial information, and ways new intelligent interfaces can help people to solve daily tasks more effectively.  Johannes recently completed his Ph.D. at the Saarland University.  He runs the multi-touch Google group. (LINK: http://groups.google.de/group/multi-touch) Johannes is involved in a range of interesting professional activities.  He's currently looking for new opportunities.


Sebastian Boring started his Ph.D. thesis in April 200 6 at Ludwig-Maximilians-Universität München and expectes to graduate July 2010.  He's a member of the Fluidum. His research interests include user interfaces for ubiquitous computing and instrumented environments, especially in interaction techniques for multiple devices and displays. He is also interested in 3D computer graphics, 3D modeling and animation, and computer vision.  (Sebastian has worked on projects to benefit assisting young people with autism).



Alexander Wiethoff is a passionate interaction designer committed to developing and implementing visionary strategies that support architectural, industrial and interaction design objectives. He works as a researcher in the field of Human-Computer Interaction at the University of Munich. Currently his key research focus is the analysis of design processes in the area of pervasive computing, the creation of work process tools that support communication of multidisciplinary teams and the design of usability testing methods in this domain.

FYI: RELATED PUBLICATIONS


SOMEWHAT RELATED
Urban Screens Conference and  Exhibition:  Calls for Interactive Media and Video Art (2010) 
More Urban Screens and 3D Media Facades
Book:  Media Facades:  History, Technology, and Content (M. Hank Haeusler)
3D Projections on Buildings: A distinctive way of communicating
Communicating Through Architecture:  Media Facades and the Digital Infrastructure  The Rathous (Contains an assortment of videos and pictures)
Art and Commerce Meet on Building's Interactive Media Facades
Kelsey Keith, Fast Company, 10/2/2009

3D TV Post, Are we there yet?

Here is the link to my post, part of a series about 3D TV, on the Innovative Interactivity (II) blog:

3D TV: Are we there yet? 

Quick Post: iPad and Mobile Learning - mLearnCon June 15-17, San Diego

Sorry for the delay in posting this.  I've been more than busy lately!

Here is the plug for the mLearnCon conference, currently taking place, June 15-17 2010 in San Diego, CA:


"Join the mLearning vanguard as we explore the potentials (and pitfalls) for mobile learning!
mLearnCon is where every aspect of mobile learning including management strategies, platforms (SmartPhones, PDAs, iPods, Tablets, etc), operating systems (Android, BlackBerry, iPhone OS X, Palm, Symbian, Windows Mobile, etc.), authoring tools and technologies, content design & development, and best practices will be explored. Whether you are working in an academic, corporate, government, or military setting ... mLearnCon will give you the ideas, information, and community you and your organization need to succeed."

Link:  http://www.elearningguild.com/mLearnCon/content/1603/


RELATED
Marc Rosenberg's article provides a good overview of the game-changing issues surrounding e-learning, m-learning, and recent technological advances such as the iPad:

Marc My Words:  Thinking About Mobile Learning in the Age of iPad

3D TV on YouTube -- and the World Cup in 3D, too!

3D Update:  3D TV: Are we there yet?   (My post on the Innovative Interactivity (II) blog)

3D TV has been available on YouTube since mid-2009. For a brief overview, watch the following video from Epic Fu:

-jetsetshow

Grab your 3D glasses!

Peter Bradshaw's overview of YouTube in 3D: Go out there and make some 3d!

Nearly a year later, this short video clip has over 1,440,000 views. A sign of a meaningful trend?

YES!


UPDATE:  Links to information about the World Cup (Soccer) broadcasts in 3D:



-3dvision blog



-Graphic from Box Office Football:  Sky 3D-Will you watch the world cup in 3D Vote


AT &T U-Verse TV Launches ESPN 3D Channel for 2010 FIFA World CUP(TM)
(Auto-Mobi.Info)


ESPN 3D officially launches with FIFA World Cup coverage:  Live from Bristol
(Ben Drawbaugh, 6/11/10)
 "As for the tech side of things, ESPN was taking in the 1080i50 side by side 3D signal and converting it to the US-friendly 1080i60 side by side, and since DLP sets expect the checkerboard 3D format, a RealD POD was used to make the link between the cable set-top-box and the TV. The glasses were supplied by XpanD, but not the same universal shutter glasses that we expect to be available at retail soon."

Engdget Interview:  Chuck Pagano talks ESPN 3D, mobile streaming
(Darren Murph, 6/12/10)
"Chuck stated that ESPN first started taking the whole 3D idea seriously around four years ago, and while the FIFA World Cup feed isn't being shot / directed by ESPN, it's hard to turn down 25 live events in 3D that the entire world will take interest in. He noted that while DirecTV, Comcast and AT&T are signed on at launch, deals are in the works with "all available options" in the US; outside of America, he confessed that the company "isn't looking at anything internationally right now."'

RELATED
I'm Sold on 3D TVs...And I Kind of Hate Myself For It

(Mark Wilson, 1/11/10 Gizmodo)

"I wouldn't buy anything but a BIG 3D TV.

Without fail, the bigger the 3D, the better the illusion. Anything under 50 inches is basically a joke, unless it's your computer monitor or something. And I will say, even though Vizio's XVT Pro television wasn't my favorite experience (it's an LCD and thereby less smooth), the fact that it was 72-inches meant that a plane's wing almost hit me in the nose."

What's next on your 3D TV?  Get ready for 3D videogames!
(John Gaudiosi, Reuters/Christian Science Monitor 6/10/10)

-Newscom/Christian Science Monitor

PlayStation 3 to Soon Get Stereoscopic 3D Support in Firmware 3.3
(3D Vision Blog: A Normal User's Look Into the World of 3D Stereo Technologies)

Note:  I'll share more about 3D TV and related technologies in a post on the Innovative Interactivity blog this week.  There is a lot to learn!
 
 -XpanDX103 3D glasses universally compatible with 3DTVs, but not all wallets
(Richard Lawler, Engadget, 3/17/10)

WONG - Multi-touch Pong Rerun - Link to Updated Game and Code (via Seth Sandler)

In 2007-2008 there were a handful of multi-touch concept demo videos created, accompanied by upbeat music. The song in this video would be great for the Glee iPad/iPhone app!

Wong was created by Seth Sandler, who recently cleaned up the code.  The latest version of Wong, along with the cleaned-up code,  can be found on the TechSparked website: http://techsparked.com/application/wall-pong

YouTube Video of WONG by Falcon4ever:


Here is a plug for Seth Sandler:

More Applications by Seth

Musical Squares

Audioshape Sequencer

Multitouch Gestures

Bloom

Multi

Virtual Glee App for your iPad, from Smule: Sing with yourself, or other Gleeks around the world, in perfect harmony!

Sing with yourself and Gleeks around the world in perfect harmony: 

Demo of the Glee iPhone + iPad App by Smule - "I'll Stand By You"
"Glee, for iPhone + iPad, by Smule. This impromptu, one-take performance by Smule engineer Nick late one night at HQ demonstrates how the app transforms his voice into a group of singers harmonizing with him."

This is exactly why I NEED an iPad!

Tutorial Part 1
"How To"



Tutorial Part II:
The Glee GlobeShows how you can share your Glee songs and sing with others around the globe!





RELATED 
Cool iPad Digital Piano App

Interactive Data + Health Apps: (article by Alex Howard about the Community Health Data Forum, with video about HHS.gov/open and info about games and health data!)

For a wealth of information and links related to this topic, please read Alex Howard's recent O'Reilly Radar post, "Here come the healthcare apps:  A look at the applications--and the big opportunities--showcased at the Community Health Data Forum."

Information from  HHS.gov/Open: 




"... working with a growing array of technology companies, researchers, health advocates, employers, media, consumer advocates, marketers, providers, etc., we are seeking to identify the uses of this data that would do the most to raise awareness of health performance, help motivate civic leaders and citizens to improve performance, and help improvers do the improving.  

Potential examples include:  

• Interactive health maps on the web that allow citizens to understand health performance in their area vs. others with tremendous ease and clarity
• “Dashboards” that enable mayors and other civic leaders to track and publicize local health performance and issues
• Social networking applications that allow health improvement leaders to connect with each other, compare performance, share best practices, and challenge each other
• Competitions regarding how communities can innovate to improve health performance
• Viral online games that help educate people about community health
• Utilization of community health data to help improve the usefulness of results delivered by web search engines when people do health-related searches and further raise awareness of community health performance 
• Integration of community health-related data into new venues, such as real estate websites, which could be highly effective disseminators of such information
• Etc."

Harnessing the Power of Information to Improve Health
Event: Community Health Data Initiative Launched (06/02/2010) (Long!)


RELATED
Making community health information as useful as weather data
Alex Howard, O'Reilly Radar 6/2/10


HEALTH DATA AND GAMES FOR HEALTH!
My favorite section of Alex Howard's post:


Game mechanics and health data
"Community Clash isn't the only game that's using community health data: SCVNGRlocation-based technology that has become familiar to many through Foursquare and Gowalla with specific challenges to earn points. SCVNGR provides a platform for organizations to build games upon. To date, more than 550 institutions in 44 states and 20 countries have taken them up on the opportunity as clients, including museums, conferences, universities and cities. combines the
"John Valentine, SCVNGR's conference and events manager, says that SCVNGR now has more than 20 million locations in its system and is being downloaded thousands of times daily from the iTunes and Android app stores. In D.C., SCVNGR will be a part of the upcoming Digital Capital Week."

Designing for TV Screen Interaction: Interesting IxDA Thread

The IxDA (Interaction Design Association) forum has an thread about designing for TV interaction.  Several IxDA members have shared some resources on this topic.  I've just uncovered the links and resources and thought I'd share them in this post, with a detailed follow-up in a week or so, as I'm also researching topics related to 3D TV.

Maurice Carty recommends the PVR (Personal Video Recorder) report from teehan+lax:
http://teehanlax.com/work/pvr-report/  (you can download the pdf version of the report from the link)

From bauruag, who is a fan of Boxee:

Boxee : http://method.com/#/productui/index/detail/CaseStudy/43 [1]
Time Warner - http://method.com/#/productui/index/detail/CaseStudy/35 [2]
Microsoft Online Media - http://method.com/#/productui/index/detail/CaseStudy/30 [3]

RELATED
Definition of Interaction Design

Definition of IxD

"Interaction Design (IxD) defines the structure and behavior of interactive systems. Interaction Designers strive to create meaningful relationships between people and the products and services that they use, from computers to mobile devices to appliances and beyond. Our practices are evolving with the world; join the conversation." -IxDA

Quick Multi-touch News Links & Pics: GestureTek's 42-inch Multi-touch GestTable; NUITEQ News; 3M's 20 Touch Multi-touch, Visual Planet's Adjustable Multi-touch Surface....

MORE MULTI-TOUCH
GestureTek News

- GestureTek intros 42-inch multitouch GestTable, your HDTV suddenly turns jealous. (Darren Murph, Engadget, 6/9/10)
"This 42-inch beast of a table contains a multitouch LCD with a lovely 1080p resolution, which means that you could theoretically watch the next installation of the Batman series on the same surface that's holding your cup of joe."   (A 70-inch version will be coming soon...)   GestureTek (Thanks to Seth Sandler for the link!)
NUITEQ News
NUITEQ wins prestigious Red Herring 100 Europe award
N-trig to show NUITEQ's Snowflake Suite with stylus support at SID in Seattle on May 23-28
NUITEQ releases version 1.9.1 of its award-winning multi-touch software product Snowflake Suite





3M News  3M Touch Systems introduces a 20-Finger Multi-Touch Display (BusinessWire 5/25/10)   The 3M touch screen pictured below uses projected capacitive technology.

VISUAL PLANET News

Design and Technology Student creates Interactive Coffee table 5/5/10

Visual Planet’s ViP Interactive Foil has been innovatively used by a student from King’s School Sixth Form in Rochester, UK to create an aesthetically pleasing and extremely creative coffee table inviting interaction from the family.

NUITEQ’s Snowflake Suite now compatible with Visual Planet’s Dual Touch Zoom and Rotate Foils
4/2010 - aNUITEQ®, an award winning technology company, announced today that its multi-touch software product Snowflake Suite is compatible with the worlds number #1 touch foil from Visual Planet.

I like the adjustable multi-touch table from Visual Planet.   Details later!

Beautiful Data: The Stories Behind Elegant Data Solutions (and a side trip of the color strata)

This is one of the books on my summer reading list.  I planned on buying this book last summer.  I got around to ordering it a few days ago, and today, it arrived in the mail.  


Nathan Yau,  the author of Flowing Data, was responsible for the first chapter, "Seeing Your Life in Data".   I'm a fan of the Flowing Data blog. Nathan shares quite a bit of interesting- and beautiful- information on his blog.  For example, his recent post, Strata of common and not so common colors, 





"The Color Strata includes the 200 most common color names (excluding black-white-grayish tones), organized by hue horizontally and relative usage vertically, stacked by overall popularity, shaded representatively, and labeled where possible. Besides filtering spam, ignoring cruft, normalizing grey to gray, and correcting the most egregious misspellings (here’s looking at you, fuchsia), the results are otherwise unadulterated."  - via Stephen, of Weather Sealed


After looking at all of the colors,  I followed Nathan's links and the links from his links (I love colorful rabbit holes!)... and found this graphic about the differences between girls and boys, from Doghouse Diaries, on the xkcd blog:

John Underkolffler Demonstrates G-Speak-collaborative, multi-display interaction (TED Talk Video by John Underkloffer, Minority Report science advisor)

John Underkloffer Points to the Future of UI (User Interface)


"Minority Report science adviser and inventor John Underkoffler demos g-speak -- the real-life version of the film's eye-popping, tai chi-meets-cyberspace computer interface. Is this how tomorrow's computers will be controlled?"

In this video, technologies that have been around for 10-15 years are demonstrated, along with newer user interface interaction, navigation, manipulation, and analysis techniques. Includes 3D interaction as well as collaborative, multi-display interaction.


"Media should be accessible, in fine grained form."

New iPhone, 3G, NOT 4G, June 21st: Front-facing camera with HD video and editing capability. Available through AT &T. Post includes video.

Fast Facts:
3G, NOT 4G :[
The square rim is the antennae.
960 by 640 resolution.
3.5-inch screen.
Multiple applications at the same time.
Stronger battery life.
Video recording in high definition: Shoot and edit capabilities!
Front-facing video camera for video calls and conferences.
5 mega pixel camera.
AT&T. No Verizon : [


-Wall Street Journal

Apple iPhone 4 Website


-Wall Street Journal Online

Egregious Citations Issued to BP, via Nathan Yau of Flowing Data (cross-post)

Interesting post with related data visualizations and a few comments:
Egregious Citations Issued to BP
Nathan Yau 6/6/10

SOMEWHAT RELATED
Flowing Data's Beginner's Guide -A Guided Tour

Cross-posted on The World Is My Interface

Interactive 360-degree video demo-reel from Yellowbird

A while ago I posted about the Yellowbird 6-lens 360 degree video camera, and promised I'd share some a Yellowbird video when I had a chance. As you watch the video, you can click and drag anywhere in the scene to get another view.

Enjoy!

Yellowbird Interactive 3D Showcase


RELATED
About Yellowbird
Yellowbird Newsletter
Yellowbird created a 360-degree interactive video tour for the Dutch Batavia Stad website, a fashion and style company. Some of the navigation is in "3-D", and at various points, the viewer can click on an icon to view a regular video-clip to obtain additional information.
Batavia Stad 360 Tour
Choose and Move Your Own POV (Point of View): Interactive 360-degree video of on-line performances from the MTV-U Woodie Awards

UPDATED iRiS: iPhone Remote Interaction System: Paint a multi-media facade on your favorite building! via Johannes Schoning (updated with links)

"We used Touch Projector to paint on a multimedia facade (here: Ars Electronica Center, Linz, Austria). Users simply aim their device at the facade causing the viewfinder to show the facade locally. In this local image, users can now draw with the selected color. The building then changes its color accordingly as if users directly painted on it."


NOTE:
Currenty Johannes Schöning works at the Innovative Retail Laboratory of the German Research Centre for Artificial Intelligence (DFKI) in Saarbruecken.  His research interests include new methods and interfaces to intuitively navigate through spatial information, and ways new intelligent interfaces can help people to solve daily tasks more effectively.  Johannes recently completed his Ph.D. at the Saarland University.  He runs the multi-touch Google group. (LINK: http://groups.google.de/group/multi-touch) Johannes is involved in a range of interesting professional activities.  He's currently looking for new opportunities.


SOMEWHAT RELATED
Urban Screens Conference & Exhibition:  Calls for Interactive Media and Video Art (2010) 
More Urban Screens and 3D Media Facades
Book:  Media Facades:  History, Technology, and Content (M. Hank Haeusler)
3D Projections on Buildings: A distinctive way of communicating
Communicating Through Architecture:  Media Facades and the Digital Infrastructure  The Rathous (Contains an assortment of videos and pictures)
Art and Commerce Meet on Building's Interactive Media Facades
Kelsey Keith, Fast Company, 10/2/2009

Pics and Video Interview: Plastic Nine-Inch Prototype $75.00 One Laptop Per Child Tablet with Dual-mode display by 2012? Via the Nick Barber

Below are pictures of the prototype of the XO-3, the third-generation of the One Laptop Per Child project, via gayakuman, and a video interview of Nicholas Negroponte discussing the third generation prototype of the One Laptop Per Child (OLPC) project, OX-3.





Nick Barber, of the IDG News Services, reporting from the MIT MediaLab:


RELATED
One Laptop per Child and Marvell Join Forces to Redefine Tablet Computing for Students Around the World PR Newswire 5/27/10
MOBYLIZE by Marvell - Marvell's Moby Tablet (below)

OLPC's Negroponte says XO-3 prototype tablet coming in 2010
Nick Barber, IDG News Service 5/27/10
OLPC XO-3 Dream Tablet May Come True After All
Brian Barrett, Gizmodo 5/27/10
OLPC Wiki