Here comes HaaS

There’s been a ton of discussion about the delivery of software applications as subscription services over the internet. The software-as-a-service, or SaaS, model is pretty easy for people to understand – most everyone these days uses a browser to tap into web services of one sort or another – and the spread of the idea has benefited greatly from the charisma of promoters like Salesforce.com CEO Mark Benioff.

Much less discussed has been the idea of buying IT hardware – or even an entire data center – as a pay-as-you-go subscription service that scales up or down to meet your needs. But as a result of rapid advances in hardware virtualization, IT automation, and usage metering and pricing, I think the concept of hardware-as-a-service – let’s call it HaaS – may at last be ready for prime time.

Last year, I had the chance to visit Savvis Inc. in St. Louis for a demonstration of what it calls its “virtualized IT utility services platform.” In essence, Savvis has built a fully virtualized IT architecture based on sophisticated Egenera servers and 3PAR storage systems, and it’s renting it out “by the slice” to companies. Hardware is deployed automatically, and almost immediately, in response to shifts in a company’s needs, usage is metered continuously, and it’s all reflected in a monthly utility bill. Savvis was already referring to its offering as “hardware-as-a-service.” (That was the first time I heard the phrase, though I was recently reminded of it by a post at Vinnie Mirchandani’s blog.)

Today, I noticed an announcement that the Federal Election Commission (FEC) has signed up to use Savvis’s HaaS service for the next five years. You can see why such a service would be attractive to an organization like the FEC. Its computing requirements vary widely, shooting way up during election years, and for it to build its own infrastructure to meet peaks in demand would be extremely wasteful. The hardware capacity would lie dormant most of the time.

Oranizations with highly variable computing demands will likely be the first to embrace the HaaS model, but given the extremely low levels of capacity utilization in most corporate data centers, it may become an attractive option for a lot of mainstream organizations as well.

3 Comments

Filed under Uncategorized

3 Responses to Here comes HaaS

  1. crabshack

    Worth noting that HaaS has been happening in the telecom sector for a number of years, although it’s never gone by that name. More recently, start-up companies such as Tellme have blended Web and telephone services in a way that allows companies effectively to rent telecom equipment on a per-minute basis. HaaS particularly makes sense in the telecom sector. Demand for capacity fluctuates wildly, the equipment becomes outdated quickly, and expertise in maintaining high-availability systems is hard to come by.

  2. ordaj

    The problem I have with the whole idea comes down to trust and it doesn’t matter what agreements you sign because there are always ways around them. Basically, this guy summed it up:

    quote:

    I’m going to reveal a dirty little secret of the computing services industry. They deliberately fail. I used to work for a F500 company that delivered services on a global basis. Ready for the business model? It goes like so:

    * Sign an agreement with the customer at a rock bottom price, and promise the same SLA as your competitors.

    * For the first few months of the contract, devote an overwhelming amount of resources towards supporting and delighting that customer, at the expense of your other customers.

    * After the first few months, under support the customer, as you do all of your customers, because otherwise you won’t make a profit.

    * When the customer complains, or contract renewal is near, devote an overwhelming amount of resources towards supporting and delighting that customer, at the expense of your other customers.

    Yes, this was for on-site and call-center support. But how does this apply to software-as-a-service? In a VERY scary way! Let’s re-write that previous script a little bit…

    * Promise the customer that your servers will be fully patched, backed up regularly, help-desk support will be awesome, and the software will be up to date, and give them a great price too.

    * For the first few months of the contract, users in that contract get sent to your biggest, best servers that get the most attention. Older customers get diverted to older servers, or servers that aren’t so hot. Those older servers get serviced last when new patches or software comes out. For those older customers, let’s try following the “gym membership” model when it comes to RAM and disk space… sure, we’ve sold 10GB storage per user, times 2,000 users, but why actually have that much when each user only is using 2.5GB on average? Etc. While we’re at it, let’s set up our routers to give the new customer 50% of our available bandwidth, and the other 50% get divided between our other 10 major clients. Support calls from that client get routed as “high priority” and some help-desk agents are only taking that one customer’s calls, to reduce the average speed of answer.

    * After a few months of making the new customer happy, divert them to the older/slower servers, strip away the bandwidth, etc. Help desk calls are dumped back into the general queue. Bandwidth is decreased to normal levels. The new customer is now receiving poorer service until they complain or until they are up for contract renewal.

    Don’t even pretend this isn’t what happens, because I used to see it happen every single day. Remote NOC work, help-desk calls, on-site maintenance. It’s not just one company that does this either. IBM, HP/Compaq, Dell, Cisco, NCR, etc. Heck, half of them subcontract it to someone else, so you’re paying big bucks for a particular company’s reputation and really another company is doing the work.

    Here’s my final question:

    How is it possible for a third party to provide a software service so incredibly cheaply that they can charge you less money than you would have spent on doing it yourself, and still make enough profit to make it worth doing? Either there are some phenomenal economics of scale happening, or they are cutting corners.

    Economics of scale? Hmmm. My experience has been that economics of scale drop off past a certain point with this stuff. Margins on hardware are already razor-thin, Dell or HP or whoever isn’t going to offer such a huge discount to a SaaS vendor on servers, storage, switches, etc. that it’s going to make it much cheaper to have 500 servers versus 10. Microsoft does not offer huge discounts on licenses. Backup tapes are pricey no matter who you are. Bandwidth, sure, it’s cheaper per bps to get an OC3 than a ton of T1s, there’s some E.o.S. there, I’ll grant that. The only time you’re going to see real economics of scale is in labor. In other words, is it as cheap for you to maintain 500 servers as it is to maintain 10?

    Cuting corners? My experience has been that this is how it’s done.

    One last point here (I know, I said the last one would be the last).

    If your SaaS vendor is not meeting SLA, what do you do about it? If they deliberately understaff the help desk, so that 80% of calls get answered within the contractually obligated 90 seconds, but during the heaviest periods the average speed of answer is 10 minutes (I have been somewhere where they laid people off the help desk because they were exceeding contractual obligations, the help desks learned to never beat contract by more than 5%!) you’re in trouble. What do you do? Where do you go? If the vendor’s bandwidth to you was NOT contractually obligated, or you need more bandwidth, guess what? You’re over a barrel!

    I’ve been at places where we let someone else manage our network. The result was that when we went down, if they didn’t pick up the phone, we were SOL. I’ve been at places where we let a 3rd party vendor handle our email and web hosting. When they went down, and they didn’t pick up the phone, we were SOL. Heck, I remember when that web host went down, and we were then informed that the “nightly backups” we were paying for were not performed as contractually promised… what recourse did we have? None, outside a court of law. That would take so many YEARS to resolve, meanwhile our data is lost TODAY.

    I don’t trust a third party to manage my network. I don’t trust a third party to store my data. I don’t trust a third party to maintain anything. Third parties will lie cheat and steal the moment you turn your back. They talk about how they have all of this experience and this and that, but at the end of the day I guarantee that you are being lied to. Their “Cisco certified network engineers” are high school grads halfway through junior college making $15 an hour. Their “MCSE”‘s list “Geek Squad” as their most recent resume experience (no dis to Geek Sqaud, but re-installing XP 10 times a day is a heck of a lot less intense than what MCSE’s supposedly do). Their “Unix gurus” are a couple of crusty old-timers who used SunOS 2.6 back in 1995. The third-party vendor is ALWAYS a black box, where you don’t know what’s happening inside of it. All you know is that you put your dollars in and (hopefully) get service back. If there’s a foulup there is no accountability. You rely upon them to proactively maintain and prevent problems. When you ask them if they are being “proactive”, they always say they are. At worst, they are lying to you, they screw up and go under, and you lose everything. At best, you end up saving some bucks and some headaches, but get to stay awake each night wondering what is really happening with your data.

    Here’s another privacy issue related to third party vendors…

    Since we know just how honest and open third party vendors are with their customers as I discussed previously, I’m sure that if someone there did something naughty that they’d immediately alert their customers. Here are some great examples of things that could happen with a third party vendor that I’m sure they would tell their customers if it ever happened:

    * A remote NOC for a bank has a disgruntled employee. On his last day at work, he runs a quickie batch script that tears through the database of their customers’ network devices, adding in a new IP address for the “allow to manage” system, then sits at home running debug tools on the routers, capturing the packets, and analysing ATM transactions, end-of-day processing, and other data transmissions at his leisure.

    * A disgruntled employee says to himself, “gee, I bet these people use the same passwords for this system that they do on Amazon and eBay!” In his spare time, he runs some code to hack people’s passwords on their system, then proceeds to try out the passwords to make some purchases or otherwise detroy someone’s life.

    * A third party vendor decides to hire someone, and in the interests of saving money, they do not perform a background check, despite the contract requiring one. The TPV accidentally hires a Kevin Mitnick. Film at 11.

    * A third party vendor’s storage system for one customer gets wiped by accident. While loading the data from tape, someone clicks on the wrong option, and also puts the data from another customer into the database. The customer now has full access to another company’s data, maybe even a competitor. Hope that wasn’t the prospective customer list, legal documents, or medical records!

    At the end of the day, when you hire a TPV, or even have a lot of data shuffling back and forth over the wire, you are EXPOSED. With the TPV route, you lose control over the people that are being hired and the processes used, and to make it worse, you don’t even know that the contract is being broken until the thing that you are trying to prevent happens. When I worked for a major services outfit, we were regularly instructed to LIE TO THE CUSTOMER AND END USERS about the following items (not the whole list, either):

    * The number of people working on the shift

    * Our current workload

    * The fact that we were servicing other customers (on contracts where it was specified that a certain number of people were dedicated to one customer)

    * Who we were employed by

    * Our qualifications to hold the position (usually things like certifications)

    * What country we had offices in

    * What country certain tasks were performed in

    * Our positions within the company (I often was a “team lead”, “manager”, “lead agent”, “supervisor”, and so forth, despite the fact that I was at the same level as everyone else)

    * ETAs, current inventory, and locations of parts and technicians

    * Whether or not we were subcontracting work to a TPV of our own

    * Reasons for delay or otherwise not meeting contractual obligations

    * The weather conditions at a remote site

    * The amount of training we had been given, as well as the qualifications of the instructors

    Do I REALLY need to go on? These are just a fraction of the things that a TPV will lie to you. Some of these can get you into big confidentiality/privacy problems. Let’s say that the contract mandates that all systems engineers have completed a training program in network security, but the vendor cheats and doesn’t put the SEs through the course. The server gets hacked in a way that the course taught how to prevent. Now the vendor lies about the attack, if they tell you at all. Your data is floating around somewhere because of something that the contract very specifically was written to prevent, and now you may be in violation of any number of laws. Hope you got assumption of liability in the contract my friend.

    Here’s another good one. Someone doesn’t set the permissions properly on a particular directory. Customer A now is able to access Customer B’s data. The TPV sees this and quietly corrects it. Customer B doesn’t say a word, of course, and neither does the TPV, but Customer A can’t figure out why sales reps from Customer B have already closed deals with all of their prospective clients.

    I mean, c’mon folks! Do you really want to have your data on the same drive or SAN or even backup tapes as other customers? Unless they dedicate (not just in the contract, but in reality) an entire staff and hardware and network to you, let you do the training and screen the employees and everything else, you have no way of knowing that your data security standards are being met! And at that point, not only is there no economics of scale left for the vendor, but you’re now spending money to oversee the contract, on top of paying for a contract where the vendor’s costs are equal to what your costs would have been.

    One final shot, before I head to sleep: when I worked for a TPV, we found out from the customer’s people that we were replacing that they were getting paid THREE TIMES what we were. Why do you think that was? I’ll give you a hint: they only people that the TPV hired who had close to the qualifications as the customer’s people, were folks who had been unemployed for a while. They all left as soon as they could. The other employees, after gaining the experience at the TPV, then left for better jobs as well.

    J.Ja

  3. JCowan

    Nick,

    Very insightful commentary. I am wondering if you, or any of your readers, know what the basic charge rates are for such HaaS? Ballpark, what would I expect to pay per GB of storage, per Mhz cpu, per MB memory, etc etc.?

    John