"Kinect-based Telepresence with Room Sized 3D Capture and Life Sized Display", Includes Behind-the Scene "how-to" (UNC Chapel Hill)

Jim Spadaccini, of Open Exhibits, recently told me about a project that involves the real-time, interactive 3-D capture of people in a room.  As the viewer moves around the screens, the depth-detecting feature of the Kinect is harnessed to set the stage for a realistic telepresence experience.  


Take the time to view the video, which contains some interesting views of how the system works:





Thanks, Jim, for the link!

RELATED/SOMEWHAT RELATED
Kinect Real-Time Room Telepresence
Kinecthacks, 1/3/12
From the project's website:

Maimone, A. and H. Fuchs. "A First Look at a Telepresence System with Room-Sized Real-Time 3D Capture and Large Tracked Display." The 21st International Conference on Artificial Reality and Telexistence (ICAT) (Osaka, Japan, November 28-30, 2011) [paper] [video]

Maimone, A. and H. Fuchs. "Encumbrance-free Telepresence System with Real-time 3D Capture and Display using Commodity Depth Cameras." The IEEE International Symposium on Mixed and Augmented Reality (ISMAR) 2011 (Basel, Switzerland, October 26-29, 2011) [paper] [video]

Open Exhibits

"Out My Window", a web-based, interactive 360-degree panorama, multimedia storytelling/documentary creation, directed by Katerina Cizek, of the National Film Board of Canada

National Film Board of Canada's "Out My Window",  part of the Highrise Project. 


Tracey Boyer, founder and managing editor of the Innovative Interactivity blog, recently posted about "Out My Window", a web-based interactive documentary filmed in 360-degree panoramic video, directed by Katerina Cizek,  a participant of the National Film Board of Canada’s Filmmaker-in-Residence (FIR) project.


After I read Tracy's blog post and explored her link to Out My Window,  I felt that it was worth sharing it on this blog.  Tracy's post provides a good description of the project, which represents a collaboration of over 100 people from around the world:  Multimedia must-see:  NFB's interactive 360 panorama documentary, "Out my window"


The opening page/view/scene of Out My Window provides the following description to the viewers/participants:


"You see them all over the world.
Concrete residential highrise buildings are the most commonly built form of the last century.
On the outside, they all look the same.  But inside these towers, people create community, art, and meaning.  Explore."


For the best experience, I recommend that you take the time to find a nice large touch-screen display or IWB to participate in the documentary - and maybe with a friend or colleague. (Some of the content is in French.)  I also recommend that you take Tracy Boyer's advice and make sure you experience the interactive documentary with a good speaker system.  The soundtrack is awesome! 


HIGHRISE Trailer



The NFB's Highrise project unveils its new interactive web documentary, Out My Window
Julie Matlin, NFB.ca Blog, 10/15/10


About HIGHRISE and Director Katerina Cizek (From the Highrise website)

"HIGHRISE is a multi-year, multi-media, collaborative documentary project about the human experience in global vertical suburbs. Under the direction of documentary-maker Katerina Cizek, the HIGHRISE team will be making lots of things. Web-documentaries, live presentations, installations, mobile projects and yes, documentary films. We will use the acclaimed interventionist and participatory approaches of the award-winning National Film Board of Canada’s Filmmaker-in-Residence (FIR) project. Our scale will be global, but rooted firmly in the FIR philosophy — putting people, process, creativity, collaboration, and innovation first.

In the first Filmmaker-in-Residence project, we worked at a major inner-city teaching hospital with doctors, nurses, researchers, and patients, challenging the conventional notions of “documentary” and academic research — and did more than just observe and record. We produced dozens of documentaries — and photo exhibits, participatory media workshops, and a feature-length web documentary.

How can the same approach be applied to urbanism? How can documentary help re-invent our cities at their edges? By going global and local at the same time, HIGHRISE is based in intensive community collaboration, married to an international vision for what documentary can be."


RELATED
Yellowbird 360 Video:
"See the world like never before with 360 video.  Can you imagine?  People like to look at 360 photos of the streets they live on, or discover their next holiday destination.  How exciting, if it was a full-motion 360 video instead of a still image?  With 360 video you can create ultimate online expereince for your clients. Share  real-life settings of the environments or events...Press play and look anywhere you want to by clicking and dragging your mouse."


Interactive 360-degree video demo-reel from Yellowbird
Yellowbird 6-lens 360 degree video camera creates web-based, interactive 3D videos

Future of Screens, Interactive Video Research, XNA 4.0 Multitouch, Pogue's Recent Post

Here are a few topics and links of interest:
THE FUTURE OF SCREENS, CIRCA 2014


If you haven't seen this short video clip, take a look. It was created by TAT MobileUI, and includes examples of how screen interaction can take place across a variety of surfaces and settings.

"This is the result of TAT's Open Innovation experiment. It is an experience video showing the future of screen technology with stretchable screens, transparent screens and e-ink displays, to name a few." TAT MobileUI. TAT stands for  The Astonishing Tribe. According to information from the TAT website, "TAT is a Swedish software technology and design company offering products and services that differentiate and enhance the user experience of portable devices. TAT is headquartered in Malmö, Sweden, and has local offices in Korea and USA."


Ola Larsén, the Vice President of Marketing at TAT, recently spoke at the ARM Multimedia conference in Taiwan, focusing on UX trends and tablets:    "Enabling a cross device connected user experience". To dig deeper, see the original post in Gizmodo and take a look at the comments:
The Future of Screens Circa 2014  (Kyle VanHemert, Gizmodo, 9/1/2010)


INTERACTIVE VIDEO RESOURCES/EXAMPLES
 I have a need to create "touchable" interactive video, and I plan to share what crosses my path on this blog from time to time.   Over the past several months, there has been some growth in the world of interactive video. The information below is just a beginning:


Interactive Video Research
Thanks to a link from Johannes Schöning,  I learned about the Dragon project, headed by Dr. Jan Borchers of the Computer Science Department at RWTH Aachen University in Germany. I haven't yet explored the research or used the application, which can be downloaded from the Dragon project website.  


Description (from the Dragon website): " DRAGON is a direct-manipulation interaction technique for frame-accurate navigation in video scenes. This technique benefits tasks such as professional and amateur video editing, review of sports footage, and forensic analysis of video scenes."
     "By directly dragging objects in the scene along their movement trajectory, DRAGON enables users to quickly and precisely navigate to a specific point in the video timeline where an object of interest is in a desired location. Examples include the specific frame where a sprinter crosses the finish line, or where a car passes a traffic light."
     "DRAGON is a research project by Thorsten Karrer, Malte Weiss, Moritz Wittenhagen, Jan Borchers and others at the Media Computing Group. It is funded in part through the German B-IT Foundation and the UMIC DFG Excellence Initiative."
Related Publication (more can be found on the Dragon website)
Christian Brockly. Evaluation of direct manipulation techniques for in-scene video navigation. Master's thesis, RWTH Aachen University, Aachen, Germany, February 2009.  

The following companies are involved in some form of interactive video work, mostly focused on marketing and advertising: INNOVID in-video innovation: "Existing Video ads can be intrusive and out of context. At Innovid, we believe that online video should be as interactive as the Internet is today, with many interchangeable elements that give the viewer dynamic content rather than a static succession of frames. We`re bringing this vision to life by innovating the way viewers interact with and engage with online video ads . iRoll®, our interactive pre-roll ad product, has powered the most interactive pre-roll campaigns with well-known brands and agencies." Innovid Platform iRoll® - Interactive Pre-roll Innovid Studio Innovid is hiring! Sales Account Executive - NYC Office Flash Programmer/designer Flash/Flex Developer Computer Vision Specialist Linux Server-Side Developer with strong IT orientation Article: Interactive video: A powerful beast (David Rossiter, iMedia Connection) QUICK.TV Quick.tv is a company co-founded by Nick Bell and Tod Yeadon.  The information about the company was in the process of being updated when I visited the website. I knew if I went to the iTVT website, I'd find more information about Quick.TV and interactive video, thanks to iTVT co-founder Tracy Swedlow: QuickTV in Interactive Video Partnership with Ooyala  (Tracy Swedlow, iTVT, 7/11/10) "According to Quick.tv, Ooyala customers will see a "seamless integration" of Quick.tv overlays into the Ooyala video player. The company says that a library of off-the-shelf widgets can be used to deploy interactive features quickly, while an advanced editor allows users to create their own custom widgets. "The widget approach is efficient and flexible," Bell continued. "It offers a wide range of monetization, engagement and editorial overlays for immediate use, whilst offering an editing facility with limitless creative potential for those who want to deliver a more specific user experience. The analytics generated by viewer interactions take video accountability and market intelligence to a new level." Howto.TV in interactive video partnership with Ooyala (Tracy Swedlow, iTVT, 7/13/10) Hotspotting: Overlay.TV integrates with MarketLive to power interactive video commerce for Armani/Exchange(Tracy Swedlow, iTVT, 9/1/2010) OOYALA Ooyala is a provider of video platform applications and services. Ooyala is involved in interactive video and has an iVideo API.  The company is expanding into the field of on-line educational video.  I'd would be great if they worked towards providing interactive video content for use on classroom interactive whiteboards! I Googled Ooyala and found out that the company recently raised more than $22 million: Ooyala Raises $22 Million to Accommodate Global Expansion.   OVERLAY.TV Overlay.TV: "Helping online retailers create, manage, and deliver interactive video experiences to drive traffic, increase conversion, and engage shoppers".  While browsing through the Overlay.TV website, I came across a series of posts worth reading: "Six Interactive Video Ideas to Get You Started" Related article: Ottowa company makes clickable fashionable (CBC News, 9/8/10) Comment:  I wish the emphasis was on touch, NOT click! KOKOKAKA Kokokaka Interactive (Ad company) Wrangler's Blue Bell Website - You can drag the character around, right inside of the video!

Demonstration of Drag Video Function (Bluebell Inventions)

I'm continuing my search for examples of the use of interactive video in education- particularly video produced for larger touch-screen displays or IWBs.


INTERACTIVE MULT-TOUCH GAMES SUPPORTED BY XNA 4.0?
Shaun Hargreaves Explains It All:
Touch input on Windows in XNA Game Studio 4.0

DAVID POGUE'S RECENT POST & COMMENTS FROM HIS READERS
RELEASED TOO SOON (David Pogue, NYT, 9/16/2010)
Since most of my readers are interested in emerging technologies, I thought I'd share a link to one of David Pogue's recent posts. David Pogue knows what he's talking about.  He's a guy who keeps up with new technologies better than most.   In the article, Pogue discusses his review of a new product, Vulkano, which was designed to record TV shows and also provide a means to watch shows via the internet.  Although his post focuses on the problems that can ensue when a product is marched to market before it is ready, the best part is the conversation generated by his post about this problem.  Pogue's comments struck a chord among people across a range of industries.  

The comments should be must-reads for university students preparing for careers in the world of business and technology!

Link to TechPsych Post: Laying the Groundwork for Interactive Video Activities for Students with Special Needs: Community Places Road Trip

If you follow the link, you can see the first "draft" of the video.  My goal is to transform the linear video into an interactive video, with hot spots and links:


Laying the Groundwork for Interactive Video: Activities for Students with Special Needs: Community Places Road Trip

Note:  This was my first time trying to shoot video of buildings and signs from a convertible car with the top down ; )

Video Experiments: Sea Life Medley: Extended version with music

I'm playing around with my sea life video clips to get ideas for creating interactive educational videos in the future for use on interactive whiteboards and other large touch-screen surfaces.

Here's my YouTube description:
This is a medley of several video clips taken of sea life, mostly living in aquariums. Awesome jellyfish can be viewed from 1:34- 3:45. The music consists of clips from iMovie, and then William Orbit's "Sea Green" and "Surfin". I created this video for students I work with who have more complex disabilities, such as severe autism.

Interactive 360-degree video demo-reel from Yellowbird

A while ago I posted about the Yellowbird 6-lens 360 degree video camera, and promised I'd share some a Yellowbird video when I had a chance. As you watch the video, you can click and drag anywhere in the scene to get another view.

Enjoy!

Yellowbird Interactive 3D Showcase


RELATED
About Yellowbird
Yellowbird Newsletter
Yellowbird created a 360-degree interactive video tour for the Dutch Batavia Stad website, a fashion and style company. Some of the navigation is in "3-D", and at various points, the viewer can click on an icon to view a regular video-clip to obtain additional information.
Batavia Stad 360 Tour
Choose and Move Your Own POV (Point of View): Interactive 360-degree video of on-line performances from the MTV-U Woodie Awards

Jonathan Kessler's Hand Eye Technologies: Coordinating your cell phone with Interactive TV

Hand Eye Technologies is developing ways to use your smart-phone over remote control driven interaction.  Jonathan Kessler, the CEO of the company, was interviewed by Tracy Swedlow, of ITTV, about his background and his ideas for the future of interactive television.

Podcast Link:  Hand Eye Technologies Interview
Here is a video from the Hand Eye Technologies website:



If you happen to have an HIT-enabled mobile device, near an HIT enabled display, two-way communication is established, via a LAN, WiFi, or wireless 3G carrier. The mobile device's camera is used to manipulate things on the interface, and the set-box takes care of some of the rest.

Interactions include selecting text and objects, "drag and drop", insert/delete, inputting text or annotations, and drwing on the screen. Hand Eye offers a drawing application called Video Graffiti, and traces the movements you make when you move your mobile device.


"Hand Eye Technologies' mission is to create and communicate the premier software platform that enables mobile devices to interact with the digital world around them... any time, anywhere." - Hand Eye Technologies

"It is more about human-computer interface than remote control". -Jonathan Kessler


This looks like it is moving towards the next level of 2-way TV interactivity, much better than what the traditional remote control can do.

RELATED


Hand Eye Technologies Management Team
CNET – Hand Eye wants your smartphone to watch TV with you
Venture Beat – DEMO: Hand Eye Technologies lets your mobile phone watch TV with you 
TheWrap.com – Coming Soon: Real-Time Interactivity Between TVs and Smartphones
Ubergizmo -With Hand Eye Technologies, the TV show continues in your handset

Interactive TV Today
About InteractiveTV Today:
"Founded in 1998 by Tracy Swedlow and co-owned by Richard Washbourne, InteractiveTV Today [itvt] is the most widely read and trusted news source on the rapidly emerging medium of multiplatform, broadband interactive television (ITV). We provide concise, original coverage of industry developments, technologies, content projects, and the people building the business. Our readership is mostly made up of hundreds of thousands of executives from around the world."

Yellowbird 6 lens 360 degree video camera creates web-based interactive 3-D videos

Update 6/4/10:   Take a look at the new Yellowbird 360 degree demo reel video on the following post: http://interactivemultimediatechnology.blogspot.com/2010/06/interactive-360-degree-video-demo-reel.html


Harry Brignull, who blogs at 90% of Everything, has an interesting video taken at what looks like an electronic music festival - "Ever wondered what it's like to be trapped in a crowd of 600,00 Dutch clubbers?"

THIS IS JUST THE PICTURE

You will need to visit Harry's blog to explore and pan around the video.

“By using a Google Streetview-like camera, a system with six lenses, not as a photo but as a video camera, an all-encompassing picture is captured. [...] From the point where the images were recorded, the viewer can look in any direction, let his eyes wander through the crowd, or stare at the ground or the air, which makes viewing a video an experience without boundaries.” - Yellow Bird press release

The Gear in Action

If I can find additional examples of Yellowbird videos, I will post them here. They are easy to work with a touch-screen. I bet these videos would be great on SmartBoards or other interactive displays.

By the way, Harry Brignull blogs about "Experience Design, User Research & Good Old Fashioned Usability"