We've gone to plaid.
It is now easier to store any and all information that we can because it _might_ be useful later. Like a data hoarder, we would rather keep everything than throw any of it away. As a result, we now are knee-deep in bits that we are not quite sure are useful or meaningful.
Fortunately, there is now a mature, and growing, family of open-source tools that make it straight-forward to organize, process and query all this data to find useful information. Hadoop has been synonymous with, and arguably responsible for, the rise of 'The Big Data'. But it's not your grandfather's mapreduce framework anymore (ok, in internet time). There are a number of open-source frameworks, tools, and techniques that are emerging that each provide a different speciality when managing and process fast, big, voracious data streams.
As a Geo-community we understand the potential for location to be the common context through which we can combine disparate information. In large amounts of data with wide variety, location enables us to discover correlations that can be amazing insights that otherwise were lost when looking through our pre-defined and overly structured databases. And by using modern big data tools, we can now rapidly process queries which means we can experiment with more ideas in less time.
This talk will share open-source projects that geo-enable these big data frameworks as well as use case examples of how they have been used to solve unique and interesting problems that would have taken forever to run or may not have even been possible.