Saturday, 21 September 2013
Stone software and Open Sourced global libraries
I'm online, therefore I am.
Piece in the Guardian about the open-source data used by Stone software developed at the University of Maryland to predict the organizational restructuring of terrorist groups. The arc of history may be long but ultimately it tends towards open-sourced information and that means that that data is pushed from the government to the public domain. There may be a current furore over the NSA procedures (and the slightly more unofficial applications) but the social media investigation of the Boston marathon bombing, complex and rumour-driven, points to new directions in which data is aggregated. Rolling newsfeeds are now largely driven by combing social media for information, and alive to the exabytes of information stored across all social media datasets, companies are designing software to create rapid composites of such data across different platforms to generate usable information on individuals and groups. One of the foremost projects is Raytheon's Rapid Information Overlay Technology. It's an acronym bound to travel quickly in social media when its use is first documented. Perhaps that is why the Snowden revelations were apparently so damaging - it's a fast-paced battle between public expansion and government retention of information in which the NSA had obviously circumvented certain rules in order to stay a step ahead and moreover, the extent to which it has embraced Big Data has been met with surprise. Not that the NSA has ever shied away from Big Data, but the Internet now makes that dataset orders of magnitude larger. We are all thinking open source now - from Westpoint to Martha Crenshaw at Stanford the datasets are available, and the ways in which we leverage that data will continue to be one of the most important issues of our time and one in which precision is often in tension with accuracy. Still, the gselevator Twitter feed tweets the whole thing best: "Only idiots get bored when we've all got handheld devices containing infinite knowledge at our fingertips."