It fills hard drives, but to extract value from it, we need methods that learn patterns in the data and allow us to make predictions and intelligent decisions.
- Zoubin Ghahramani
“I keep saying that the sexy job in the next 10 years will be statisticians, and I’m not kidding,” Hal Varian, Chief Economist at Google famously observed in 2009. It seems a difficult assertion to take seriously, but six years on, there is little question that their skills are at a premium.
Indeed, we may need statisticians now more than at any time in our history. Even compared with a decade ago, we can now gather, produce and consume unimaginably large quantities of information. As Varian predicted, statisticians who can crunch these numbers are all the rage. A new discipline, ‘Data Science’, which fuses statistics and computational work, has emerged.
“People are awash in data,” reflects Zoubin Ghahramani, Professor of Information Engineering at Cambridge. “This is occurring across industry, it’s changing society as we become more digitally connected, and it’s true of the sciences as well, where fields like biology and astronomy generate vast amounts of data.”
Over the past few years, Richard Samworth, Professor of Statistics, has watched the datarati step out from the shadows. “It’s probably fair to say that statistics didn’t have the world’s best PR for quite a long time,” he says. “Since this explosion in the amount of data that we can collect and store, opportunities have arisen to answer questions we previously had no hope of being able to address. These demand an awful lot of new statistical techniques.”
‘Big data’ is most obviously relevant to the sciences, where large volumes of information are gathered to answer questions in fields such as genetics, astronomy and particle physics, but it also has more familiar applications. Transport authorities gather data from electronic ticketing systems like Oyster cards to understand more about passenger movements; supermarkets closely monitor customer transactions to react to shoppers’ predilections. As users of social media, many of us disclose data about ourselves that is as valuable to marketing as it is relevant to psychoanalytics. Increasingly, we are also ‘lifeloggers’, monitoring our own behaviour, health, diet and fitness, through smart technology.
Read the full story
Image Credit: Automatic Statistician
Reproduced courtesy of the University of Cambridge
______________________________________________