Minds Mastering Machines 2019 – Keynote: Dark Data is your friend (Mark Whitehorn)

Keynotes traditionally look to the future but let's briefly consider the recent past because we have seen so much change in recent times.

In the early 2000s most people were still struggling with BI when Big Data suddenly appeared and we embraced the new, NoSQL database engines in order to analyse it. Just as we were getting used to that, neural nets finally emerged from the long shadow of their 50 year gestation period and gave us an entirely new approach to analytics. Their importance is difficult to over emphasise because suddenly we (the humans) are not the only ones who can induce the rules. That brings benefits but also huge ethical and moral issues.

All of this means that the world of analytics has been experiencing hugely turbulent times. In truth my prediction for the immediate future is a period of consolidation, albeit with further developments. So, not much food for a keynote then? On the contrary, I think there is a fascinating class of data that has been around for a while which most people are ignoring to their cost – Dark Data.

So this talk will define the different flavours of Dark Data and give examples where its analysis has proved particularly fruitful.