You’ve seen the ads: Crowds of smiling employees clapping their hands and exchanging high-fives. Why are they so happy? Because their companies had the good sense to install software from SAP. The tagline: “Companies that run SAP are 32% more profitable than companies that don’t.”
That’s an awfully big claim – associating significantly higher profits with the installation of a particular software package – but, as I’ve pointed out before (here and here), there’s no way to validate it. SAP has spent millions on the ad campaign, but it hasn’t made the research public. We get the tagline, not the facts.
Next week, though, a study examining the relative profitability of SAP customers will be made public. It’s not the SAP-sponsored study, however. It’s a study by Nucleus Research, an independent reseach house that specializes in helping companies assess the financial returns on IT investments. The headline of the Nucleus Research study is a bit different than SAP’s advertising tagline: “SAP customers are 20% less profitable than their peers.”
Nucleus Research looked at the 81 public companies that SAP lists as customers on its own website. It determined the return on equity (ROE) earned by each company, based on formal financial filings, and then compared that number with the average ROE for the company’s industry, as calculated by Hoovers. The upshot, as stated in Nucleus’s research note: “Nucleus found SAP customers had an average ROE of 12.6 percent, compared to an industry average ROE of 15.7 percent. It is interesting to note that three areas of significant focus for SAP, customer relationship management (CRM), enterprise resource planning (ERP), and supply chain management (SCM), had customers who fared quite poorly, with CRM customers achieving profitability 18 percent lower, ERP customers achieving profitability 32 percent lower, and SCM customers achieving profitability 40 percent lower than their peers.” In its note, Nucleus documents the data it collected for each of the 81 companies.
In an email exchange, I asked Nucleus’s vice president of research, Rebecca Wettemann, why her firm undertook the study. She replied, “Nothing inspired us other than a few of our clients who commented about [the SAP claims]. We do a lot of ROI investigation and since SAP opened the door to talking about profitability, it seemed reasonable to take a closer look.” She also noted that “no one paid us or commissioned this in any way.” I also asked her if she could explain the discrepancy between the Nucleus findings and the findings promoted by SAP in its ads. “It’s hard to comment on SAP’s study since they won’t share it with us,” she said.
The Nucleus Research study is a limited one, but at least the firm has clearly documented its data and analysis, so readers can draw their own conclusions. One wonders if SAP will now at long last release the documentation of its study. Until it does, Nucleus Research will have the last word in this game of strained statistics: “Despite SAP advertising claims to the contrary, factual analysis of ROE data shows the best-run companies don’t run SAP.”