Interactive Big Data for Gov, Biz, - and the rest of us? (long post, resources and links)

I've moved this post to the "pages" section of my blog:

Big Data for All: Resources, References, Links

Photo Credit: DARPA

Quick link: Nextgov, "the all-day information resource for federal technology decision makers"

I've been thinking more about interactive data exploration interfaces lately, and what might be needed to support people who do not have technical or mathematical backgrounds.  From time-to-time, I will share information related or somewhat related to this topic.


I came across a link to the Nextgov website in my FB newsfeed and thought I'd bookmark it for future reference. Although the Nextgov is geared for people who work as CIOs in government settings, it contains a lot of interesting food for thought if you happen to be a knowledge junkie or just want to be a a well-informed citizen.  


 Below is a quote from "About Nextgov" page of the website:
"Nextgov is the all-day information resource for federal technology decision makers. Through news, analysis and insights from our award-winning journalists and a nationwide community of expert voices, Nextgov provides the first word on technology and government." 


"Nextgov’s editorial mission is to lead the national discussion about how technology and innovation are transforming the way government agencies serve citizens and perform vital functions. Central to this mission is the exploration of emerging technologies and their potential impact on government. Nextgov contributors include influential thinkers across government, academia and the private sector providing fresh and provocative insights on key federal IT topics."


Nextgov is produced by Government Executive Media Group...."


For your convenience, below are links to various sections of the Nextgov website:
News
CIO Briefing
Emerging Tech
Cloud
Cybersecurity
Mobile
Health
Defense
Big Data


I was disappointed that there wasn't a section about education, so I've some related links below:

RELATED
Big Data - Avalanch? Flood? Tsunami?  What does big data mean for educators?
Evidence Framework for Innovation and Excellence in Education
Karen Cato, Director, Office of Educational Technology, U.S. Dept. of Education


The article contains a link to a draft of a related publication released for public comment:


Enhancing Teaching and Learning Through Educational Data Mining and Learning Analytics: an Issue Brief 
U.S. Department of Education, Office of Educational Technology
April 10, 2012


FYI: The above document was prepared by Marie Bienkowski, Mingyou Feng, and Barbara Means, under the guidance of Karen Cator and Bernadette Adams. A number of experts were interviewed for the report, including Shelby Andrews, Linda Chaput, Michael Freed, Dror Oren, Dave Gutelius, Michael Jahrer, Andreas Toescher, Jeff Murphy, Peter Norvig, Sunil Noronha, Ken Rudin, Steve Ritter, Bror Saxberg, David Niemi, and Chuck Severence.  Others cited in the document include Ryan S.J. d. Baker, Gautam Biswas, John Campbell, Gredg Chung, Alfred Kobsa, Kenneth Koedinger, George Siemens, and Stephanie Teasly


SOMEWHAT RELATED
Nonprofit Data Visualization: a Gallery
Philanthropy:  Connecting the nonprofit world with news, jobs, and ideas
Nicole Wallace, 3/4/12

The Digging Into Data Challenge and List of Online Data Repositories for Humanities and Social Sciences

I came across the Digging Into Data Challenge website today and thought it might be of interest to IMT readers.  Below is a description of the project, a few related links, and an abbreviated list of links to a wide range of data repositories on the web.  

 ABOUT 

"Welcome to the second round of the Digging into Data Challenge. During the first round, in 2009, nearly 90 international research teams competed in the challenge. Ultimately, eight remarkable projects were awarded grants."

"In 2011, the Digging into Data Challenge has returned for a second round, this time much larger, with sponsorship from eight international research funders, representing Canada, the Netherlands, the United Kingdom, and the United States."

"What is the "challenge" we speak of?  The idea behind the Digging into Data Challenge is to address how "big data" changes the research landscape for the humanities and social sciences. Now that we have massive databases of materials used by scholars in the humanities and social sciences -- ranging from digitized books, newspapers, and music to transactional data like web searches, sensor data or cell phone records -- what new, computationally-based research methods might we apply? As the world becomes increasingly digital, new techniques will be needed to search, analyze, and understand these everyday materials. Digging into Data challenges the research community to help create the new research infrastructure for 21st century scholarship." 

"Applicants will form international teams from at least two of the participating countries.  Winning teams will receive grants from two or more of the funding agencies and, two years later, will be invited to show off their work at a special conference sponsored by the eight funders."

"Let's get digging."

RELATED
Digging into Data Challenge Second Year Request for Proposals
Press Release    Round Two: Digging Into Data Challenge: Social and computational scientist asked to design methods and tools for analyzing large data sets (National Science Foundation)

Times Higher Education, April 28, 2011, "Research intelligence - Let's dig a little deeper"
The New York Times, November 16, 2010, "Digital Keys for Unlocking the Humanities’ Riches"
The Globe and Mail, June 18, 2010, "Supercomputers seek to ‘model humanity’"

Be sure to visit the Digging Into Data Challenge data repository page.  Each repository is annotated in detail and includes links and in some cases, APIs. The page is updated regularly, and at the time of this post, was updated on May 26, 2011. For your convenience, here's the abbreviated version of the Digging Into Data Challenge list of repositories: 

The Archaeology Data Services (ADS)
http://archaeologydataservice.ac.uk/


ARTStor.org
http://www.artstor.org


Biodiversity Heritage Library
http://biodiversitylibrary.org


The Centre for Contemporary Canadian Art Canadian Art Database Project
http://ccca.ca


Chronicling America, Library of Congress, National Digital Newspaper Program
http://loc.gov/chroniclingamerica


Data-PASS
http://www.icpsr.umish.edu/DATAPASS/


Digital Library for Earth System Education (DLESE)
http://www.dlese.org


Early Canadiana Online
http://canadiana.ca


English Broadside Ballad Archive
http://ebba.english.ucsb.edu


Great War Primary Documents Archive
http://www.gwpda.org


Harvard Time Series Center (TSC)
http://timemachine.iic.harvard.edu/search/


HathiTrusthttp://www.hathitrust.org


The History Data Service (HDS)
http://hds.essex.ac.uk


Infochimps.org
http://www.infochimps.com/


Internet Archive
http://www.archive.org


Inter-university Consortium for Political and Social Research
http://icpsr.umich.edu


JISC MediaHub
http://jiscmediahub.ac.uk


JSTOR
http://jstor.org

Marriott LibraryUniversity of Utah
http://ww.lib.utah.edu/portal/site/marriottlibrary/

NASA ADS: Smithsonian/NASA Astrophysics Data System (ADS)
http://adsabs.harvard.edu/index.html

National Archives, London
http://nationalarchives.gov.uk

The National Library of Wales
http://www.llgc.org.uk


National Science Digital Library (NSDL)
http://www.nsdl.org

National Technical Information Service (NTIS)
http://www.ntis.gov/

Nebraska Digital Newspaper Project
http://cdrh.unl.edu/nebnewspapers/

New York Public Library
http://NYPL.org

The New York Times Article Search API
http://developer.nytimes.com/


Opening History
http://imlsdcc.grainger.uiuc.edu/history/

PhilPapers (philosophy)
http://philpapers.org

Project MUSE
http://muse.jhu.edu/

PSLC DataShop (Pittsburg Science of Learning Center)
http://pslcdatashop.web.cmu.edu/

Scholarly Database at the Cyberinfrastructure for Network Science Center, Indiana University
http://sdb.slis.indiana.edu

ScholarSpace at the University of Hawai'i at Manoa
http://scholarspace.manoa.hawaii.edu/community-list

Statistical Accounts of Scotland
http://edina.ac.uk/stat-acc-scot/

University of Florida Digital Library Center
http://www.uflib.ufl.edu/ufdc

University of North Texas
http://digital.library.unt.edu/browse/?browseby=collection

Just a thought....
I'd be interested in learning if any of the researchers involved in the Digging Into Data Challenge plan to work on an interactive, multidimensional, multimedia timeline to support collaborative data sharing and analysis.  

A few links to start the week: AR for libraries, Accessible Musical Slideshows (Spectronics), Health Care Design, Big Data (GigaOM)

AR FOR THE LIBRARY (Supports productivity for librarians)
The article below discusses an Android-based app developed by Miami Unversity's Augmented Reality Research Group:
Awesome Augmented Reality App Could Save Librarians Hours Audrey Watters, ReadWriteWeb, 3/27/11
In the related video, Dr. Bo Brinkman (Miami University-Ohio) demonstrates the prototype AR app using a Samsung Galaxy Tab:

Although there are probably more creative ways to use AR in libraries, this application is a start!


INTERACTIVE PRESENTATION APP
Make musical slideshows in minutes - switch accessible too! Barbara Landsberb, Spectronics Blog, 12/17/10

Slideshow Maker was designed to work on a range of displays, including touch screens. It it easy to use.  
Slideshow Maker Manual  (1.8 MB)  Note:  SlideshowMaker was not yet available in the US at the time of the above post. If I find a video about Slideshow Maker, I will post it here.


HEALTH CARE
The article below is a must-read for anyone thinking about developing applications for health care professionals, clinics, hospitals, patients, etc.  In my opinion, it is very important to look at the "Big Picture", especially for techies who are responsible for  information architecture in this field.
For the Future of Health Care Design, Look Beyond the  Hospital Fast Company Design, 3/25/11 "Here, Jean Mah and Robin Guenther, two health-care design experts at architecture firm Perkins + Will, look at what the future holds for our health care systems." -Ed. (Fast Company)


BIG DATA
In the article below, the topic of "Big Data" is addressed.  Low power computing solutions are needed in order to handle the number-crunching and information visualization tasks necessary for humans to understand and effectively make use of "Big Data".

The World of Big Data Needs Low Power Computing

Katie Fehrenbacher, GigaOM, 3/24/11

gigaombigdata on livestream.com. Broadcast Live Free

Blog posts from the GigaOM Structure Big Data Conference

Johnny Holland on-line Magazine- some of my favorite posts.

If I had the time, I could spend several hours catching up with the on-line Johnny Holland Magazine.  Why?  It is full of stuff I like to think about.

"Johnny is an open collective talking, sharing, and finding answers about the interaction between people, products, systems or processes."

Here are a few of my favorite posts:

Learning from Games: A Language for Designing Emotion
Joe Lamantia, 8/3/2009

Lamantia's article discusses the thoughts of Nicole Lazzaro, a games researcher and design consultant, who suggests that there are four kinds of fun, as outlined in the chart below:
  --XEO Design Inc.

(The references for Lamantia's article are worth taking a look.)

Engaging the User: What We Can Learn from Games
Marc Sasinski, 8/31/09

When Data Gets Up Close and Personal
Stephen Anderson, 1/27/10

The Social Life of Visualization (4-part series by Jeremy Yuille and Hugh Macdonald)
The Social Life of Visualization, Part 2:  Creation Phase
The Social Life of Visualization, Part 3: Interpretation
The Social Life of Visualization, Part 4: The Capture Process

"Jeremy Yuille and Hugh Macdonald are interaction design researchers with ACID at RMIT University. They research the effects of social media on different industries, ranging from sport to finance."

The Future is Touchable
"Jeroen van Geel is founder of Johnny Holland and a senior interaction designer. He works part-time for Fabrique Communication & Design and the other part for his own clients."


Visit Johnny Holland Magazine to find out more about the "Johnnies" below:


:





(The "Johnnies" without pictures can be found on the Johnny Holland website.)

Big Data: What are the possibilities for collaborative interactive information visualization? (Video interviews of Roger Magoulas, director of research at O'Reilly)

When I return to graduate school (hopefully I'll have the means to attend full-time), I want to flesh out my ideas for a "interactive multi-dimensional multi-media multi-user timeline" for use on interactive multi-touch/gesture tables and displays.   Although I've limited my work to a prototype of a template, I know that this concept won't work unless the application can incorporate an efficient means of handling large volumes of data, as well as data in various formats.

I want this template to be useful to people in a variety of contexts, such as students studying world history and humanities, education administrators looking at educational data over time, producers and viewers of interactive documentary programs (think interactive TV), the health industry, urban planners, the military, serious games, etc.

One of my stumbling blocks is how all of the data would be stored and analysed.  What I learned a few years ago in my computer classes simply won't work.

So now what?!  I think that Roger Magoulas, the director of research at O'Reilly, has some good things to say about the critical problem of handling what he calls "Big Data".   Here are a few videos that I think are worth watching.

The Future of Work
Part One



Next Device (SmartPhones, netbooks, creation & consumption factors - supporting usability in multiple contexts)



You Tube Series: O'Reilly Media
Big Data: Technologies & Techniques for Large-Scale Data (Emphasis on experimental approach) Part I


Part II (Discusses new forms of databases and the user of parallel processors to handle Big Data)


Part III Key Technology Dimensions


Part IV, Focus on hardware- Solid state disks, new data structure called "triadic continuum" which handles real-time data and ongoing probability estimates of data.


I would be happy to hear from anyone who is working on a project similar to the one I'm working on as a "hobby".

RELATED

Triadic Continuum
"Phaneron, KStore, Knowledge store, or simply K, is a dynamic data model that is based on the cognitive theory of C. S. Peirce. Phaneron efficiently organizes data into a unique, compact, interconnected, and fully-related data model. Phaneron is constructed using the Triadic Continuum."  


For those of you who like visual representations of geeky-techy concepts, here a few visuals and related descriptions of  KStore fundamentals from the Triadic Continuum website:


"The KStore data model is constructed using the basic triad. For example, the event sequence 'cat' would be recorded as shown in 'a sequence' below. A new level of nodes is created above a lower level of nodes as a result of the triadic process. In this case the lower level of nodes contains a node for each character of the alpha-numeric character set and the new nodes reference the lower level nodes to record the sequence 'cat'. Each sequence is initialize with a reference to a BOT (beginning of thought) and terminated with an EOT (end of thought) reference."





"The data set above was used to create the K structure below with the lowest level that contains the alpha-numeric character set, the second level is created to record sequences that represent the field variables. Then a third level is created using the field variables of the second level to record the record sequences. Records recorded in this K structure reuse the field variable nodes so that these field variable sequences never have to be recorded more than once. This is just one of the attributes of a K structure that makes it very efficient." -Triadic-conintuum.com



Mazzagatti, J.C. (2006) The Potential for Recognizing Errors in a Dataset Using Computer Memory Resident data Structure Based on the Phaneron of C.S. Peirce (doc)


Personal Note:
Due to the economic downturn and its impact on my family (two kids in college), I returned to work full time in mid 2008. I have a very busy day job as a school psychologist, working at two high schools as well as a program for students with multiple, severe disabilities, including autism. This has limited my ability to work on my project.