大数据的预测盲区

    Are there any questions out there that can't be answered using data and analytics?

    So I think it all exists along a spectrum. It's important to know, too, that there's a difference between how good we are relative to our potential and how intrinsically predictable something might be. So for example if you look at baseball where analytics have come an awful long way, well it's still the case that the best baseball teams only win two-thirds of their games. The best hitters only get on base about 40% of the time. So it's still intrinsically unpredictable in a sense, but we have a good way of measuring and knowing what we know and what we don't know.

    But there are a lot of fields where analytics have not come very far. I discuss earthquake forecasting in my book [The Signal and the Noise: Why So Many Predictions Fail -- but Some Don't] for instance, where people have been trying for centuries. We know something -- there are more earthquakes here in California than in New Jersey -- but the ability to anticipate a particular earthquake with any precision at a particular moment in time has not gone very well at all. Even economics -- when we try to do long-term economic forecasting, it has been pretty poor for the most part.

    Are there any industries out there that are overlooking the possible impact of big data analytics?

    It's sometimes industries that aren't very sexy necessarily, so big retail businesses for example have tons of records on every consumer transaction they have [completed]. They have a ton of data on supply chain management, so things like that in terms of optimal inventory strategies or optimal pricing strategies or robust strategies for disruption in the supply chain. Not super-sexy stuff, but those people have really good data sets, often high quality data, and can make better decisions as a result. I'm sure that some are doing it already, and there are some efficiencies there that you weren't seeing before.

    There are also cases like if you look at how people consume television, for example. I think the advertising industry is something that's gotten more sophisticated in terms of targeting customers. The irony is that efficiency has become harmful in some ways for media companies, because what was the old adage? "Half your advertising budget is well spent, but you don't know which half." Now people might know which half, so they're only spending half as much.

    Can people use data or analytics to accurately predict the stock market in any way?

    The problem is the stock market is this whole contest where you're competing against other creators. So the question is: Are there some traders that are better than others? I think the answer is probably, Yes. I'm not a pure markets guy, I played poker long enough which I think has parallel skills to trading in many respects where you know that some people are better over the long-term and better at accounting for uncertainty and so forth. But there's a lot of volatility and a lot of luck where a market cycle can last for months or years. There are a lot of perverse incentives that get in the way. So while I think there are some good traders, in the near term, even over a period of five or 10 years, it will mostly be dictated by luck, so it's tricky.