Predicting a Crash or Data Mining?

data mining

data miningMark Hulbert wrote a piece for MarketWatch last week where he posted a current chart of the Dow against a chart of the Dow right before the crash in 1929. The charts look very similar and the hook of the article is that we may be on the brink of another crash.

I used to listen to Gary Kaltbaum’s podcast almost everyday. He was big on suggesting that gold was retracing the epic move it made in the 70s. (I believe it is still on track.)

This type of analysis is based on the idea that history is doomed to repeat itself. However, each of these guys is a bit smarter than that. They will both be the first ones to tell you that they would be the last ones in line to actually bet money on these predictions.

Hulbert continues his piece by illustrating a different view:

David Leinweber, founder of the Center for Innovative Financial Technology at the Lawrence Berkeley National Laboratory, isn’t impressed. In an email, he said that “if you looked at enough periods of the same length, you’d find all sorts of very similar pictures, most without a crash at the end.”

Leinweber views charts such as this one as an example of a potentially dangerous practice known as “data mining”— endlessly analyzing a database until you “discover” a pattern. The result of this practice is “the analytical equivalent of finding bunnies in the clouds. If you did enough poring, you would be bound to find that bunny sooner or later, but it would be no more real than the one that blows over the horizon,” he said.

He suggests that traders should always be leary of a bear market:

To be sure, a bear market could happen at any time, and drawing the analogy between now and the late 1920s can serve a helpful purpose: If you don’t think you can stick with your stock holdings through a market decline, you should reduce them now to whatever level you would be comfortable holding through that decline.

I like to think that looking for major patterns to repeat themselves is similar to making predictions based on any other form of analysis. There just seems to be an awful lot of room for error. There also doesn’t seem to be a plan for what to do if that prediction is wrong.

The best course of action in any endeavor is to have a plan that can adapt to whatever happens. That is true in trading as well. You should know what you will do if the market goes up for the next 6 years, and you should know what you will do if the bottom falls out tomorrow. Having a plan that can handle whatever the market throws at it is the only way I believe that you can be consistently successful.