Open source and the programmer's dilemma
April 25, 2007
A new article in IEEE Computer, "The Economic Motivation of Open Source Software: Stakeholder Perspectives," sheds some interesting new light on an old question: Is open source software development good or bad for programmers?
Much has been written about what motivates software writers to contribute to open source. The motivations are, as you might expect, diverse, ranging from an ideological belief in free software to a desire to impress one's peers to a desire to bolster one's skills, reputation, and career prospects to receiving pay from a company in the open source business to having fun. As open source has become more popular, the motivation to participate has, naturally, become stronger. But knowing that an individual coder has rational incentives to contribute to open source development doesn't tell you whether or not open source improves the economic standing of programmers in general. It doesn't tell you whether the programming profession ends up with more or less money.
The author of the IEEE Computer article, Dirk Riehle, a researcher with SAP, doesn't look at that question directly. Rather, he examines, in a theoretical way, how open source changes the economics of the IT markets in which programmers participate. He first looks at why big systems integrators and other "solutions" providers, like IBM, have been promoting open source. He argues that these companies, which sell bundles of products and services to their clients, like open source because it allows them to reduce the amount of money they have to pay to software vendors without requiring that they pass along the savings to customers in the form of lower prices. In other words, the software savings turn into additional services profits, which fall to the solutions providers' bottom lines. Ultimately, that means that open-source software developers are subsidizing the big solution providers at their own expense. Writes Riehle: "If it were up to the system integrators, all software would be free (unless they had a major stake in a particular component). Then, all software license revenue would become services revenue." (I would think it's an overstatement to say that all software license revenue turns into services revenue; assuming there's competition between solutions providers, some of the savings would go to the customers.)
Riehle also looks at the economic effect of open source on software markets themselves. He argues that, by tearing down the barriers to entry in software markets (by obviating the huge up-front investments required to create a proprietary program), open source spurs competition, which in turn reduces prices and erodes the profits of software vendors. Riehle writes: "Customers love this situation because prices are substantially lower than in the closed source situation. System integrators love the situation even more because they can squeeze out proprietary closed source software." For the programmers themselves, however, much of the savings reaped by customers and added profits taken in by integrators comes out of their own pockets.
Riehle also notes that open source (because of its openness) tends to diffuse knowledge of particular programs among a much broader set of programmers. That will tend to increase competition among the programmers and hence depress their pay: "Technical skills around the open source product are a key part of determining an employee's value to a [vendor]. Anyone who's smart enough can develop these skills because the open source software is available to people outside the firm. Hiring and firing becomes easier because there's a larger labor pool to draw from, and switching costs between employees are lower compared with the closed source situation. Given the natural imbalance between employers and employees, this aspect of open source is likely to increase competition for jobs and drive down salaries."
If Riehle's analysis is correct - and while his thinking is logical, he offers no hard proof of the economic effects he describes - then what we're seeing playing out among coders is what I'll term the Programmer's Dilemma. Because skills in open source programming are increasingly necessary to enhance the potential career prospects of individual programmers, individual programmers have strong motivations to join in - and as more programmers join in, the incentive for each individual programmer to participate becomes ever stronger. At the same time, the total amount of money that goes to programmers falls as open source is adopted by more companies. Individual programmers, in other words, have selfish motives to engage in collectively destructive behavior.
Nick, I'm reminded of a certain Dilbert cartoon - the gist of it is showing programmer X writing incomprehensible code that nobody else understands, programmer Y knows that X is much smarter, and Y asks X "Why are you making such a mess?". Then X replies "Job security!".
Boiled down, the above analysis is that protectionisn is good for the protected. And it is! That's not a surprise.
The basic flaw in the reasoning, however, is that since the software industry changes very rapidly, it's much more rational to try for a comparatively smaller part of a bigger *service* market, than a bigger part of a smaller *production* market.
That is, in a closed-source system, the protected programmers have a good life, but the moment that system becomes obsolete, those individuals are on the outside again. And that happens very fast compared to a career-span. So given the number of different systems a programmer can expect to work on, that iterated situation overall can make it beneficial to cooperate by being open, than to try to defect by being closed.
places like Microsoft are an exception to the above, because Microsoft OS is (unfortunately) a very large and long-lived platform - but it would be a mistake to think that represents the whole programming industry.
Posted by: Seth Finkelstein at April 25, 2007 09:22 AM
Good points. I'm not arguing for "protectionism" or against open source; I'm just looking at the question - and it's an interesting question, and a question that, I think, remains open - of whether or not programmers are, as a whole, better or worse off, economically, as a result of open source. Whether the users of software or the IT industry or the world in general are better off is a completely different question.
Empirically, I observe a severe shortage of developers, whether with open-source or closed-source software skills; also generous pay, drifting upward quite steadily.
So I suspect that Mr. Riehle's model fails to account for some important facets of reality.
I don't doubt you're right.
That doesn't necessarily mean that Riehle's analysis is wrong (though it could mean that). If overall demand for software development goes up (and/or if supply of developers goes down), the upward wage pressure that would create could more than offset the downward wage pressure that Riehle's scenario implies. In other words, if Riehle's right, even if coders' overall wages are going up, the gains would not be as strong as they would have been in the absence of open source.
Of course, if it's the rise of open source that's pushing up, in whole or in part, general demand for software, that would also need to be taken into account as a countervailing force.
Posted by: Nick Carr at April 25, 2007 10:35 AM
As a developer, and one that uses and contributes to open source, I think some key points have been missed.
1. Seth nailed one of them -- it's not a closed system. If you think in terms of proprietary stacks and jobs for a given company, then sure. But the world is a big place. Big companies aren't innovating any more. Name something truly mind-blowing that has come out of Microsoft, IBM, SAP, or Oracle. It's not happening. If we developers let the market be stifled by corporate short-term thinking and mismanagement, the market as a whole will die off.
2. Joy. Good developers don't write good code because they have to, they write good code because they want to. It's like an artistic thing: corporate life hampers the interesting innovative stuff you want to work on, so when you go home you write interesting code and release it to the open source community. Simply for the joy of it.
3. Community. Many (not all) good developers are slightly introverted. Contributing to a community of like-minded people across the world gives a great sense of belonging that you simply don't get in a corporate environment.
4. Continuity. I've been working on the same OSS projects for years -- one through three jobs and two different career directions. Jobs are short term. Working for free sucks. But many (good) developers are either contractors or skip companies every 1-3 years. They go to where the interesting work and good pay is. This is a direct result of offshoring. Companies are getting back what they've been dishing out. So what if developers get paid less at IBM because they aren't building as many proprietary solutions any more -- go work for someone that will value you.
5. Gives you a sense of purpose. Not all of us are driven completely by money -- something that economists don't get. There are other levers, including contributing to the development of third world nations. Look at Ubuntu. It was created specifically to address the needs of Africans that can't afford proprietary software.
6. Street cred. Nothing gives you more cred and notoriety (and into interviews) than contributing something genuinely elegant or innovative back to a bunch of people that can appreciate it.
It's pretty obvious that the open source momentum is driven by service companies. They are the only ones who have a big enough incentive AND a big enough capital. So far, so good. But the rest of Riehle's argument (disclaimer: I only read it here, didn't bother to read the original) is, how shall I put it subtly, bollocks.
The margins of profits service companies cut are defined by supply, demand and competition. The funny thing about OS is that it increases all three. By reducing production friction, there's more quality software out there for a smaller price. The more affordable software around, the more people use it for more things. You don't buy photoshop to remove red eye from your grandkid's photos, but with gimp - hey why not?
And last, since the market is shifting from production to service, big companies are loosing their edge on the small & medium consumer front. Compare the car market to the food market. Yes, chains and corporations have an advantage over small local food suppliers, but when it comes to cars - there are no mum & pop shops. Why? entry costs are too high. Now, creating an office suit is like building cars (or spaceships). Servicing it is like selling pizza. Except that in an open source environment, if I don't like the curve of the streamers, I can change them.
So all this means that software is going to be free, the market is going to be service based, and software engineers are going to be very happy - because there's all the more demand for them on one side, and the option of flying solo on the other.
By the way, think google - there's another example of how to strive in a service market.
Posted by: yish at April 25, 2007 04:27 PM
To your question "it's an interesting question, and a question that, I think, remains open - of whether or not programmers are, as a whole, better or worse off, economically, as a result of open source.", offshoring trend has similar appeal as open source trend in being able to tap into a larger programming talent pool. Would studying its impact on programmers be a good substitute measure? If so, are there stats/studies on the impact of offshoring trend on programmer salaries?
The missing part of the analysis is lock-in for the programmers - if you work on a big proprietary system, and want to move jobs, you will lose all the code you have already written, and will need to start over to some extent.
If you work on an open source basis, you can bring a lot of it with you, thus being more productive immediately.
Posted by: Kevin Marks at April 25, 2007 06:17 PM
The "closed system" approach cuts both ways as well. As I programmer I might choose to work on my company's proprietary system in the belief that it offers me more job security and better pay, but I'm pretty much screwed when that company decides to discontinue development of that system (mod ongoing support etc.) My employer knows this and factors the small market for my skillset (one possible employer) into my salary. The net is lower pay and less job security.
If, on the other hand, I decide to work on some open-source project, I have a larger pool of potential employers to choose from and my current employer has to take this into account (or else lose me). If my current employer goes belly-up or decides to discontinue involvement in the project, I can go to another employer involved in the same project (assuming I have picked a project with a lot of interest and involvement).
As you point out, working on an open-source project also means that I have to compete against a larger pool of other programmers but, given the relative shortage of programmers in general, that's probably a risk I'm willing to take (did I mention that programmers are generally very egotistical?). The upshot is that, all things being equal, working on a closed system only makes sense if you think/know that you are not very good.
y th eway, William Hurley thinks microsoft loves open source (from today's /.)
Posted by: yish at April 25, 2007 06:50 PM
Was I dreaming, or was there a post following this one earlier today that bashed Wikipedia and Citizendium? I didn't have the time to read the whole thing carefully then (work and all) but was looking forward to checking it out (and probably disagreeing).
Nick, Riehle's comments are part of a common confusion in the open source debate, where there's no distinction between the benefits for corporations and users on the one hand, and the benefits for creators on the other.
Clearly, free software is a big boon to Google and every other large corporation, just as low wages are. But neither provide benefits to programmers. Many advocates of open source actually have explicit open source lobbying roles with corporates, but misrepresent their claims as being in programmers' interests. That is, their job is to lobby for free inputs.
Unfortunately the progamming profession is young and has no awareness of its own interests, nor consciousness of labour market disciplines that are part of the structure of older professions. Also, the lack of access barriers removes the consciousness of their own value that is implicitly taught to people entering other professions.
Worse, open source is riven by huge hypocrisy. For example, Riehle is part of a small "open source" group at SAP, yet SAP itself ruthlessly protects the source code of its main products. Other prominent open source advocates like IBM and Google follow the same pattern.
I wrote a paper of my own examining these issues in 2004, and you can see other comment at IP Central.
Posted by: Tony Healy at April 25, 2007 11:23 PM
Here's the URL for IP Central.
Some of the above comments are examples of the captured mentality prevalent among some of the programmer population.
For example, Seth writes as if the only options are to work for Microsoft or other companies, but fails to recognise the possibility of building your own business, based on the value of the IP you create.
Similarly, mardukvmbc sees open source as a way to do interesting projects, failing to recognise that, if he retained the IP and was good, that could be his full time job. That is, instead of giving his work away, he provides it for payment, just as other professions do.
Tony, these are comments, not a book. Sometimes I think I have to stick a disclaimer on my comments like the following:
"Disclaimer: This is a short comment, not an extended treatise. As such, it may use approximations, simplifications, compressions, generalizations, and omit extensively qualified phrasing, all in the service of conveying an idea in a small number of words. Please keep that in mind when making objections."
That is, if something is true for 90% of the cases, please don't think it's some great objection to note I didn't go into some 1/2 of 1% third-order approximation where it doesn't hold. There's very few systems where nobody benefits. Most startups fail, and this is well-known. Holding out the idea that "building your own business, based on the value of the IP you create.", is not going to work for something like 99.5% of all programmers. Thus, it's a tedious objection at best, and propagandistic think-tank deceptive flackery at worst.
Posted by: Seth Finkelstein at April 26, 2007 02:47 AM
Seth, it would be great if you didn't try to dismiss my comments as "propagandistic think-tank deceptive flackery" just because I disagree with you. No one tells me what to write, ever. If you want to discuss my papers or comments, please do so.
Second, you did write as if the only option for programmers is working for a corporate entity when it's not. I take your point that not everyone will or can create valuable IP, but if source code was valued more highly, programmers generally would be better rewarded.
I think, in this , Nick's final paragraph is correct. Programmers have been induced to contribute to undermining their own value, and the question as to why is fundamentally a sociological question.
For example, contrast the different way lawyers and programmers respond to web requests for information. The lawyer advises you to see a lawyer (and thus pay for the privilege.) Many programmers, on the other hand, provide copious information for free, in what can be seen as a quest for social validation. I know there are other reasons, some of which are useful, but the general principle remains.
There are some fascinating behavioural issues here.
Yeah, programmers need to learn to think more like lawyers. After all, so many successful technology companies are run by lawyers. Great, innovative tech companies like ... Viacom? The United States Congress??
Healy's M.O. is predictable - thow whatever crap he can at open source in the hope that something will stick. Seth's dismissal is right-on. If anything, Seth is too gentle with the guy.
The most salient part of Nick's original post is "he offers no hard proof of the economic effects he describes." Let's see some real research.
Doug, 99 percent of lawyers give the rest a bad name, so I wouldn't advocate that at all. I do advocate that good programmers retain their IP and in fact that is what Google, Microsoft, Pixar and other successful, innovative technology companies do.
Programmers at Google, Microsoft and Pixar retain control over their copyrights? I doubt it.
On the other hand, many open-source programmers DO retain control over their copyrights (I prefer that term to the ambiguous and misleading "IP"). take a look at MySQL, which is profitable and on the verge of going public. Oh yeah, they use the GPL.
"At the same time, the total amount of money that goes to programmers falls as open source is adopted by more companies." (1)
I think more accurate would be to say:
At the same time, the total amount of money that goes to COMPANIES selling close source software falls as open source is adopted by more companies. (2)
(1) follows from (2) only if:
most of programmers work at such companies,
programmers' wages make significant part of theirs expenses,
and companies-customers will not spend significant part of money they can save
on another software-related projects
We are poor but happy.
See my small cartoon.
Posted by: Oliver Widder at April 26, 2007 05:35 PM
Seth: "Most startups fail, and this is well-known. Holding out the idea that 'building your own business, based on the value of the IP you create.', is not going to work for something like 99.5% of all programmers."
The reverse is true. If you're good, you can create a company and sell your IP for good money. In return for the good money, you can hire more people and improve your IP and make it even more worthwhile for a customer to buy. I've done this. I am speaking from experience, and I am not just lucky: I persevered. The first few years can be hard, but when you find your niche, you can make a good living while being your own boss - which is what matters most.
There are others like me, occupying his or her own special niche. My company does SSH software for Windows. There's another colleague who has a small company developing and licensing stock market software. Then there's another colleague whose small company does network monitoring software.
We each make a better living this way than if we were employed anywhere else, and it's a proud and liberated way to live.
Compared to what I described, open source can be a hypocrisy. I've met some open source gurus who proselytize free software, complain about the scourge of commercialism and generally espouse Marxist tendencies during the night, and then they go to their jobs working at Big Corp in the morning. Some revolutionistas, these are.
Posted by: denis bider at April 26, 2007 07:06 PM
Open source tends to drive the adoption of new ideas because of its free and open nature. This accelerates the transition of new genres of software (and ideas) from being a potential source of competitive advantage, being novel and new, to being (if useful) a cost of doing business due to its ubiquity.
Open source also accelerates the adoption of new standards - for example REST over SOAP. Do you believe that the world wide web would have been anywhere near as vast if HTTP had been a proprietary licensed product?
The interesting thing about common and ubiquitous services, is that they allow for new ideas to be built upon them. You need commoditised power, communication and data processing before you can have something like Google.
Imagine building a virtual web business, if the first thing you need to focus on is how to manufacture a microchip, installing communication lines to customers etc. At some point, each of these concepts were novel and new - the real power of these "infra-structural" goods only becomes apparent when they are ubiquitous.
Open Source drives this transition from novel and new to common and ubiquitous ideas and services, which in turn encourages new ideas and services based upon them. I'd argue that open source drives innovation in a society and I'd agree with Riehle that open source tends to diffuse knowledge and increases competition.
But what if I'm a programmer spending all my time developing CODB (cost of doing business) like applications - does this threaten me? Of course it does, but then again the spread of any idea from novel and new to common and ubiquitous always threatens those who make their living from keeping that idea, that concept, that software service to a niche few.
An example of this is ERP. A very lucrative industry, which is often sold as providing a source of competitive advantage or strategic value, despite almost every company having ERP systems. The reality is more likely that ERP is just a CODB. Any advantage of customisation should be weighed against the cost of customisation. Being ubiquitous and common, I would suggest that any advantage can only be gained through being "cheap as chips" or "cheaper than your competitors".
I would argue that much of IT is now CODB. Hence the tendency to offshore, buy off the shelf or ideally rent on a utility basis (SaaS).
Of course, as a company, one of my concerns is always vendor lock-in and exit costs. Hence for example, though I think Amazon's EC2 is a great idea, my problem with it is that there is no Google EC2 or Microsoft EC2. There is no open standard, no portability for my applications other than spending CAPEX on building my own infrastruture or entering a contract negotiation with a hosting vendor. So for CODB like apps, I'll always seek the lowest cost with the most portability between vendors. Hence, open source, which tends to create emergent standards, supported by vendors, is my natural preference.
However, I am also always seeking for the novel and new, something to give an genuine advantage over others. Who better to help create it, but the same people who build those emergent standards, whom are well versed in the leading edge of technology and of whom I can clearly see their capabilities - the alpha geeks.
Hence, I use open source communities as a hunting ground for real talent.
This is a growing trend, at almost every open source conference that I've attended in the last year - the banners, even the presentations by companies saying "we're hiring" were everywhere. Head-hunting of top open source developers also seems rife within Europe.
To give a pertinent example, a well respected open source developer announced one morning on his blog his intention to leave his job. By the time I got through to him in the afternoon, the offers had been flooding in. The competition for the most talented and skilled developers - individuals who can make a real difference - is intense, something which I have never experienced before in the world of IT.
As for those who don't fit into those categories, well I don't want to pay large amounts of money for something which is really a CODB and whose price is over-inflated because it is kept arcane and proprietary with no open standard. I'm afraid their future does look fairly gloomy because with open standards you can only compete on talent.
That's the real beauty of open source for any programmer - because everyone can see your work, you get to find out if you really are that good or not. Unlike the old days, where everyone said they were an expert.
I know of a few concrete examples of top open source developers who loved what they were doing and have been head-hunted by financial institutions. In every case, there weren't interested but the banks wanted them and asked them what it would take to make them move. In every case, they give what they thought was a ridiculous figure, the sort of sum earned by a broker, to get rid of the bank.
In every case, they got that amount.
So Riehle is right at one end of the scale, but for the really talented - it means you can write your own meal ticket.
Posted by: Simon at May 3, 2007 07:14 AM
Write your own meal ticket - this is also what Riehle seems to say in "An employee who is a committer is likely to earn higher compensation."
He's bang on, from my experience.
Posted by: Simon at May 3, 2007 07:27 AM
Can someone please explain, how does closed source protect me as a developer? I worked for a startup, made beautiful code. The startup went no-where, and my code went with it. I worked for Cisco, rewrote some of my code, and a lot of new code. I even submitted 6 patents. What have I got? A few stories to tell the boys at the local.
Even as a consultant, my clients ended up with the rights to the fruits of my labour.
The thing is, even if I had the power to retain rights over my work, I wouldn't have the power to enforce them. If, on the other hand, I would have produced open source code, I would be the best placed man to service it, whoever the client or employer.
Posted by: yish at May 3, 2007 12:38 PM
"Can someone please explain, how does closed source protect me as a developer?"
If it is a niche and specialised area of high worth which requires a high amount of intrinsic or "arcane" knowledge - ideally an obfuscated system you've built yourself running at the heart of large corporation - you can call this "protection" of sorts.
The answer in general is it doesn't.
Talent, knowledge and skill protects you, and open source is the best way of getting noticed.
Posted by: Simon at May 3, 2007 03:21 PM
He argues that these companies, which sell bundles of products and services to their clients, like open source because it allows them to reduce the amount of money they have to pay to software vendors without requiring that they pass along the savings to customers in the form of lower prices. In other words, the software savings turn into additional services profits, which fall to the solutions providers' bottom lines.
No. If the cost of production goes down, that absolutely does not, in a functioning, competitive marketplace, go to those companies bottom line, because other companies will move in to compete.
Of course, there is a brief period of time when those 'first-movers' enjoy an advantage, but in a competive marketplace, that time is short.
It is interesting that software as a service is copybot resistant, while shrink-wrapped software is not.
Posted by: enigma_foundry at May 14, 2007 10:48 PM
Post a comment
Thanks for signing in, . Now you can comment. (sign out)(If you haven't left a comment here before, you may need to be approved by the site owner before your comment will appear. Until then, it won't appear on the entry. Thanks for waiting.)
"Riveting" -San Francisco Chronicle
"Rewarding" -Financial Times
"Ominously prescient" -Kirkus Reviews
"Riveting stuff" -New York Post