Big Data: What are the possibilities for collaborative interactive information visualization? (Video interviews of Roger Magoulas, director of research at O'Reilly)

When I return to graduate school (hopefully I'll have the means to attend full-time), I want to flesh out my ideas for a "interactive multi-dimensional multi-media multi-user timeline" for use on interactive multi-touch/gesture tables and displays.   Although I've limited my work to a prototype of a template, I know that this concept won't work unless the application can incorporate an efficient means of handling large volumes of data, as well as data in various formats.

I want this template to be useful to people in a variety of contexts, such as students studying world history and humanities, education administrators looking at educational data over time, producers and viewers of interactive documentary programs (think interactive TV), the health industry, urban planners, the military, serious games, etc.

One of my stumbling blocks is how all of the data would be stored and analysed.  What I learned a few years ago in my computer classes simply won't work.

So now what?!  I think that Roger Magoulas, the director of research at O'Reilly, has some good things to say about the critical problem of handling what he calls "Big Data".   Here are a few videos that I think are worth watching.

The Future of Work
Part One



Next Device (SmartPhones, netbooks, creation & consumption factors - supporting usability in multiple contexts)



You Tube Series: O'Reilly Media
Big Data: Technologies & Techniques for Large-Scale Data (Emphasis on experimental approach) Part I


Part II (Discusses new forms of databases and the user of parallel processors to handle Big Data)


Part III Key Technology Dimensions


Part IV, Focus on hardware- Solid state disks, new data structure called "triadic continuum" which handles real-time data and ongoing probability estimates of data.


I would be happy to hear from anyone who is working on a project similar to the one I'm working on as a "hobby".

RELATED

Triadic Continuum
"Phaneron, KStore, Knowledge store, or simply K, is a dynamic data model that is based on the cognitive theory of C. S. Peirce. Phaneron efficiently organizes data into a unique, compact, interconnected, and fully-related data model. Phaneron is constructed using the Triadic Continuum."  


For those of you who like visual representations of geeky-techy concepts, here a few visuals and related descriptions of  KStore fundamentals from the Triadic Continuum website:


"The KStore data model is constructed using the basic triad. For example, the event sequence 'cat' would be recorded as shown in 'a sequence' below. A new level of nodes is created above a lower level of nodes as a result of the triadic process. In this case the lower level of nodes contains a node for each character of the alpha-numeric character set and the new nodes reference the lower level nodes to record the sequence 'cat'. Each sequence is initialize with a reference to a BOT (beginning of thought) and terminated with an EOT (end of thought) reference."





"The data set above was used to create the K structure below with the lowest level that contains the alpha-numeric character set, the second level is created to record sequences that represent the field variables. Then a third level is created using the field variables of the second level to record the record sequences. Records recorded in this K structure reuse the field variable nodes so that these field variable sequences never have to be recorded more than once. This is just one of the attributes of a K structure that makes it very efficient." -Triadic-conintuum.com



Mazzagatti, J.C. (2006) The Potential for Recognizing Errors in a Dataset Using Computer Memory Resident data Structure Based on the Phaneron of C.S. Peirce (doc)


Personal Note:
Due to the economic downturn and its impact on my family (two kids in college), I returned to work full time in mid 2008. I have a very busy day job as a school psychologist, working at two high schools as well as a program for students with multiple, severe disabilities, including autism. This has limited my ability to work on my project.