Numbers matter. They matter a great deal in business and in life. But as Nate Silverman pointed out in his book, The Signal and the Noise: Why So Many Predictions Fail — But Some Don’t, “Numbers have no way of speaking for themselves.” So it’s up to us to make sense of them, to make sure they “add up,” and to provide them with a voice.
That begs the question: what happens when there are so many numbers – so much data – that the “making sense” part becomes incredibly overwhelming and it’s a challenge to speak above a whisper? A large data set is like the largest litter of puppies you’ve ever seen: it is a handful, to put it mildly, and your attention can get torn asunder.
To carry the analogy further, think of the puppies as “bytes.” Where do you store them? How do you search them? How do you analyze them? Welcome to the new world of big data. It’s a wondrous, wild, wild, west, and no one has the perfect map or set of coordinates to navigate through the vast, and rocky terrain. Oh yeah, and it’s not just about numbers anymore. Tweets, emails, a Reddit post, a Yelp review – even a simple “click” – all those count, too. It’s as if kittens are now mixed in with your puppies.
What is big data exactly, you ask? Gartner describes it as “high-volume, high-velocity, and/or high-variety information assets that require new forms of processing to enable enhanced decision making, insight discovery and process optimization.” Think of the data as being all shapes and sizes. That makes it difficult to store and index in traditional ways. And why is there suddenly so much of it? Another good question!
These days, there are so many more (so, so, many more) devices and machines that allow us to seek and obtain information in the manner to which we’ve grown accustomed. It has resulted in a Mt. Everest of data bytes. Corralling all those bytes under one roof is more of a challenge when you consider this fun factoid: “the volume of data traffic is growing by 59% every year,” according to Gartner. Obviously, information technology professionals have a big challenge on their hands; they need new systems to store all the data, and analysts and researchers need new approaches to comb through it.
So where is all of this data coming from? Think about all the data transactions you make during a typical day on your smartphone, work computer, personal computer, and tablet – and that’s just a sub-set of big data sources. When you buy gas at the gas station, food at the grocery store, or visit a nearby café chain for an afternoon pick-me-up, your purchases are recorded. All that recorded information is thrown onto an ever-increasing pile of data, along with everyone else’s. Companies want to find out what you like to buy and how to sell you more of it. How are they doing that? They are looking for patterns in the data. And they are looking for patterns that aren’t so readily detected.
But big data is not just the purview of commerce. Take this example: In his article “Why Big Data is a Big Deal,” L. Gordon Crovitz describes how “Google is better than the Centers for Disease Control at identifying flu outbreaks. Google monitors billions of search terms (“best cough medicine,” for example) and adds location details to track outbreaks.”
In another example, Crovitz described researchers in Canada who examined the records of premature births (“1,000 data points per second”), which uncovered the fact that when infants’ “vital signs are unusually stable, that correlates with a serious fever 24 hours later.” Though doctors do not yet know why this occurs, they know now to take precautions.
You may be thinking, “Hmm, I don’t work in information technology, or even have plans to – so why should I care about big data?” In Jer Thorp’s article, “Big Data is Not the New Oil,” he argues that it’s time to reframe the way all of us think and talk about big data. As he explains it, most of our personal data transactions (Facebook posts, web surfing proclivities, Tweeting our locations, etc.) are being “monetized,” and it’s easy to forget that. We don’t consider that all that data is made up of bits and pieces of our lives. Or, as Thorp says, “It is a dense condensate of our human experience.”
Thorp believes we need to begin humanizing big data because it will get us to think beyond how everyone’s data is currently being used, and how it could be used in the future. Pondering future uses (and how it will be managed) can shape how we deal with it now. So how do we start? Thorp has three recommendations that apply to everyone who engages in data transactions – and that’s almost everyone. They aren’t a part of the current discourse, but we think they should be. They are:
Deal with your data: When you experience firsthand how to store, search through, and protect your own personal data, you are dealing with a microcosm of big data. And just as with big data, there are inherent risks and rewards, albeit on a smaller scale. Examples of risks: identity theft or a stolen credit card number. Examples of rewards: helping a friend locate a missing pet through social media, or fundraising online for a local cause. By dealing with your own data, it’s much easier to understand the dangers of mishandled big data, as well as the innovative solutions it can produce when it’s managed successfully.
Don’t forget ethics: There are real human beings on the other side of those data bytes, says Thorp, and it’s important to remember that. “Of the dozens of start-ups who have approached me for advice on their personal data-centered ventures over the last year, not a single one has mentioned the rights of the people from whom the data is being extracted. This needs to change.” Thorp predicts that big data companies committed to being “data humane” will ultimately have a competitive edge.
Leave a chair at the table for artists: You read that right. Thorp believes the potential for big data is, well, big. So how do we get our minds around that in order to begin figuring out how use big data in innovative, helpful ways, as well as minimize the risks and ethical gray areas? He has this to say: “As we proceed towards profit and progress with data, let us encourage artists, novelists, performers and poets to take an active role in the conversation.” Creative minds breed creative solutions.
All of us, whether we are fully cognizant of it or not, are constantly generating information that adds to the larger body of big data. It’s possible you are in generation mode as you read this post right now. We at Hult Labs think it’s time for people to become mindful of the data they’re creating, where it’s going, and how it’s being used. In essence, we all have a big litter of invisible puppies and kittens (in the form of bytes that contain our personal information), running in all directions. You may ask yourself: Where are they going? And do I care? We think it’s worth looking into.
Picture courtesy of Infocux Technologies.