The latest issue of SAP’s Business Flash newsletter just popped into my in-box. Under the email’s catchy headline “Companies That Run SAP Have 32% More High Fives At Their Staff Meetings” runs this sentence: “A recent study of companies listed on NASDAQ and NYSE found that companies that run SAP are 32% more profitable than those that don’t.” At the end of the sentence is an asterisk that leads to a footnote: “Based on a 2005 Stratascope Inc. analysis of publicly available fiscal results of all non-financial companies listed on NASDAQ and NYSE.” OK, I’m intrigued. A broad study that links a particular corporate software program to vastly outsized profits is interesting. I want to learn more.
So I click on a link in the email that says “We invite you to see for yourself,” figuring it will bring me to a copy of the study, or at least to some details on the research. Wrong. The link brings me to a marketing page on the SAP site filled with the usual slogans, like “enable business flexibility.” The only information on the study is this: “Companies that run SAP are 32% more profitable, according to results from a 2005 Stratascope Inc. study, which analyzed publicly available financial results of NASDAQ and NYSE companies. The study also found that these companies delivered 28% more return on capital. Clearly, SAP customers have a strong track record of outperforming their peers.”
Clearly? Seems pretty opaque to me. I mean, where’s the data?
Not to be put off, I do some searching, assuming that the details of the study have been published somewhere. Nope. I can’t find any trace of this research on the web. I do, though, find the home page of Stratascope, the company that did the research. Its business consists mainly, it seems, of providing IT sales forces with financial data on public companies. On every page of its site is a large promotional advertisement highlighting some of its key clients, one of which is SAP. Hmm. I also find that the chairman and president of Stratascope, Juergen Kuebler, “was employed at SAP AG for 9 years where he was responsible for the sales launch of the product ‘R/3 on AS/400’ as well as the sales training of the complete staff at SAP AG.” Now, the fact that SAP and Stratascope are cozy – and that it’s in Stratascope’s interest to make its client happy – doesn’t mean that Stratascope’s research is necessarily unsound. But it does raise questions – questions that can only be answered through a careful review of the research methodology and results. A review rendered impossible by the fact that neither SAP nor Stratascope is revealing details about the research.
This isn’t an isolated problem. IT companies are always throwing around seemingly precise statistics claiming to show that their hardware or software is associated with competitive advantage or superior financial results. As far as I’ve been able to discover, the research is almost always dubious. Either the methodology is flawed (tiny or biased samples), or the research is carried out by the company itself or some sycophantic supplier. And rarely are the full details of the study divulged. IT buyers shouldn’t pay any attention to such faux statistics. If a vendor dresses up its marketing slogans with research results, then it should show us the data – all of the data.