Book Review by Victor Niedehoffer: Data Driven Investing by Bill Matson and Mitchell Hardy
Data Driven Investing discusses techniques used by the authors to turn approximately $0.6 million into $4 million over the period July 2000 to December 2004 by investing with margin, in the main in nanocap value stocks. The authors' methodology in motivating their results reminds me of a piece of silver I own where a mad painter rides an octopus on top of a squid with sea monsters jumping up at him from a turbulent sea. In more familiar terms, its like what you would end up with if you had a magnet and Googled every academic journal for an anomaly, every issue of the Stock Trader's Almanac for a seasonal, and every technique in every book in the last five years with a system to beat the market, and all of these were made of iron.
Here are just some of the techniques that the authors
tested and/or used to select the buys in their portfolio:
The managers also buy put options when they think the market is likely to go down, especially during June to September, and they sell Friday afternoons.
There are no statistics on the variability of any of the techniques. And the authors rely on Compustat data that assume perfect knowledge of income statement and balance sheet figures as of the end of the year they invest in.
While the authors understand that data have survivorship problems because the small nanocaps that had been added as of 2000 were not selectable prospectively, they apparently donít realize that using perfect knowledge of earnings is guaranteed to lead to value's beating growth because of the regression bias, and unexpectedly good earnings leading to subsequent price increases when they are announced. The authors also donít seem to appreciate the principle of everchanging cycles, or numerous other biases that would take longer to enumerate than the previous list of techniques used.
The authors intend to create a video library of information on companies, including plant tours and management discussions of financials, and this is to be applauded. And they intend to schedule 52 fundraising events per year until they've raised $3.25 million, with 5% to be given to Babson College.
I'd recommend this book highly for teachers of investment classes who wish to use it as a take-home exam. "Comment and criticize."
The authors have kindly agreed to comment on this review. And many of the analytical deficiencies I find in the book do not deflect from what may be the practical value of the book. Indeed, I might be tempted to open a trial account with them myself. However, the problem with techniques that are not properly scientific is that it's impossible to separate the wheat from the chaff, the permanent from the transitory, the recurring from the everchanging. That is the great tragedy of this monumental effort so marred by multiple comparisons and look-back biases.
Coauthor Bill Matson replies:
To the extent you found fault in our work, it appears that the problems generally arose from our being insufficiently clear in making our points. We will certainly take your comments to heart in future editions. In particular, we should have been more emphatic in making the point that we only bet on anomalies that are likely to be sustainable. Sustainable anomalies are likely to arise from such things as:
When we backtested with Compustat data, we assumed perfect knowledge of income statement and balance sheet figures as of the beginning (not the end!) of each year whose returns we tested. As we admit, this introduces about one quarter's worth of look-ahead bias; however, we also note that side-by-side comparisons with O'Shaughnessy's lagged data (from What Works On Wall Street) are consistent with our major conclusions.
Coauthor Mitchell Hardy adds:
Data Driven Investing was intended to help individual investors make money in the market, not as a work of scholarly erudition. We're practical men, not theoreticians, hence the "Data Driven Test Portfolio." To the extent our book lacks scientific precision, that was intentional in that we purposely avoided presenting any advanced statistical analysis that might have limited its practical utility to a wider audience.
Also, it should be recognized that it's probably impossible to explain with scientific precision an activity as complex as the management of a real money portfolio encompassing more than 10,000 trades involving hundreds of individual stocks. A lot of different ideas went into the mix, and if we didn't nail down our every assertion with enough analytical rigor to please our most sophisticated critics, so be it. The proof of the pudding, it is said, is in the eating, and we have been dining rather well off our 890% returns. Anyone who reads, understands, and applies the ideas we presented in our book would likely do pretty well.
Regarding value versus growth stocks, there are, of course, numerous analytical and practical difficulties, not the least of which is defining just what is meant by a growth stock. A mechanical approach that might work to define a value stock (e.g. low P/E) tends to break down in picking growth stocks. A high P/E stock might be a growth stock or it might just be an overvalued stock. People who invest in growth stocks tend to think they are smart enough to pick companies whose earnings really will grow fast enough for long enough to justify the high prices those stocks typically command, but "smartness" in this case is hard to define quantitatively. We've made a lot of money by trading on "unexpectedly good earnings" announcements, but the bar of expectations is generally set very high for growth stocks; they really have to perform to create unexpectedly good earnings, but the penalty for unexpectedly poor earnings can be severe. Having said that, the effect of one quarter's worth of look-ahead bias on the significant differences we've observed between the returns for low price ratio (value) stocks and high price ratio (growth) stocks does not materially affect our conclusions.
Lastly, I rather like the image of the mad painter atop the octopus, etc. I've never heard my coauthor described so aptly.