Rufus Pollock, Founder and Co-Director of the Open Knowledge Foundation, wrote a wonderful article in the Guardian yesterday that summarized what I’ve been thinking about for quite a while (but in a much more erudite, focused way). Basically, with all the hype about mining Big Data, there is a bigger, more immediate need – to deal with the daily stats that are in front of our noses.
As Pollock states, “Size in itself doesn’t matter – what matters is having the data, of whatever size, that helps us solve a problem or address the question we have. For many problems and questions, small data in itself is enough.” Amen to that. I shudder to think of all the market research that is done without doing a very basic review of what is freely available. Yipes. But I am thrilled to see the advances in the open government movement in regards to organizing all the little bits-and-pieces into some sort of accessible index / database / portal.
Yes, we need more data analysts and scientists to deal with big data. But we also need more basic data literacy through our organizations, so that we can see the stories behind the small data and act upon them in a nimble, innovative way. I haven’t worked on any open-source dashboard software out there yet (Stashboard? SWING?). Let me know if you have played with any.