For The New York Times, a swing and a miss at Amazon Web Services


After years of indifference, it’s good to see prominent media companies taking a closer look at how enterprise tech operates. However, it’s disheartening to see them do it in a way that betrays their surface-level understanding of the companies, technologies, and concepts at hand.

With the kind of splash reserved for stories that grace the front pages of the big Sunday paper, yesterday The New York Times published a dramatic tale of how a rapacious Amazon Web Services is stomping all over the enterprise software industry with its relentlessness, drawing an explicit comparison between the cloud market leader and Amazon’s retail operation. But in doing so, it distorted the nature of how open-source software and cloud computing work, either out of ignorance or in service of a simple narrative.

Open-source concepts transformed enterprise software. Open-source projects allowed companies to build their own technology infrastructure using free packages of code that solved specific needs, rather than having to buy expensive one-size-fits-all enterprise software packages from big, slow-moving vendors.

It’s not easy, however, to put an open-source project at the heart of your tech infrastructure, which gave rise to a number of startup companies that offered commercial services designed to help customers implement open-source projects. Many of those startups wrote those projects themselves with funding from venture capitalists, releasing them to the community under widely used permissive licenses and charging for extra features and support.

As we’re all aware, a growing number of information technology customers want to consume software -- both commercial products and open-source projects -- on cloud servers managed by AWS, Microsoft, and Google. And because the underlying code behind those commercial software products is open source, there’s nothing stopping anyone from building a commercial service that manages the implementation and use of an open-source project, which is just as hard to do on the cloud as it is in a data center.

And so over the last few years, several companies built around open-source projects have started to grumble more loudly that AWS, given its enormous lead in the market for public cloud infrastructure services, is capping their potential revenue growth by offering its own managed services around those open-source projects. 

The Times article referred to this as “strip-mining,” comparing the (totally legal) decision to build a managed service for an open-source project to a mineral-extraction technique that leaves behind nothing of value and devastates the surrounding landscape. It also left the impression that open-source software is the foundation for most of the services that AWS offers, offering no basis for sweeping context-free claims like these:

Amazon has used its cloud computing arm — called Amazon Web Services, or A.W.S. for short — to copy and integrate software that other tech companies pioneered. It has given an edge to its own services by making them more convenient to use, burying rival offerings and bundling discounts to make its products less expensive. The moves drive customers toward Amazon while those responsible for the software may not see a cent.

(Side note: While I understand that the New York Times copy desk has exacting standards, “A.W.S.” looks absolutely terrible on both the page and the screen.)

This framing is ludicrously over the top, and it shows how little the Times understands about the history of the cloud and open-source software. In fact, the story didn’t even mention the words “open source” until about two-thirds of the way in, long after the “strip mining” narrative was well established.

In the introduction to the article, you’ll also find this line: “By lifting other people’s innovations, trying to poach their engineers and profiting off what they made, Amazon is choking off the growth of would-be competitors and forcing them to reorient how they do business, the companies said.” 

Those companies quoted in the article include Cloudflare, MongoDB, and Elastic -- big companies that have enjoyed surging revenue and public-market validation over the past few years -- as evidence of companies harmed by AWS’s practices, which really undercuts the point.

Last week Cloudflare CEO Mathew Prince tweeted figures that show Cloudflare has gained market share against AWS in a category in which they both offer similar services. Also last week MongoDB reported better-than-expected quarterly revenue thanks to strong growth for Atlas, a managed version of the MongoDB open-source project that it offers on all three major cloud providers; its stock is up more than 60 percent over the last year.

The article ignores a recent Times story about a rise in entrepreneurial engineers who honed their skills inside consumer tech companies but decided to launch their own enterprise tech companies, partly out of the belief that “making money from consumers was too daunting, they concluded, partly because of the tight grip that digital giants like Google and Apple had over distribution.” It lumps Snowflake Computing, a data-warehouse vendor that has seen enormous growth on AWS, in with some of these other companies without noting that Snowflake’s products are not available as open-source projects.

But it’s the “lifting other people’s innovations” line that really got me. The entire point of software (voluntarily) released under a permissive usage license is to pass innovation along to others who might use that code in interesting ways, and who might then contribute what they’ve learned back to the project.

Done correctly, this produces more vibrant software, developed in the open by a community of developers with diverse business interests. This way of building software has been embraced by nearly everyone in enterprise software to some degree or another, even at Microsoft, which for years spent a lot of money insisting the very concept of open source was illegal.

Yet it takes a lot of work to build and maintain a popular open-source project, and there aren’t many people who can afford to work for free. Bigger companies with established business models often participate in open-source communities as a way of recruiting engineers who want to get recognition for building cool stuff, while individuals and startup companies want to monetize their investment (not to mention their VC’s investments) in that work while participating in open-source communities that can improve the project and help spread the word, before they start asking customers for money.

What’s really happening at the moment is an open-source business model is being upended because cloud providers are able to offer open-source projects at scale far better than anyone else, and scale matters for a lot of business applications. The value of an open-source project is shifting to the distribution of the software, as opposed to the software itself.

AWS CEO Andy Jassy speaks at re:Invent 2019 earlier this month (AWS Photo)

This is where the “strip-mining” narrative really starts to fall apart. Companies that made business decisions to monetize open-source projects in certain ways are not, and have never been, entitled to all the potential revenue opportunities from those projects until the end of time. There are lots of companies that have built big businesses around open-source projects developed inside other companies, with Hadoop (conceived at Google, refined at Yahoo, and commercialized by Hortonworks and Cloudera) serving as a good example.

The cloud definitely makes a few things harder for independent software vendors. In the data-center era, they had a direct relationship with their customers. As those customers decide to move to the cloud, the software vendors have no choice but to work with cloud providers to meet their customers where they want to be.

Independent software vendors can offer their commercial services on cloud provider marketplaces (an enterprise App Store, basically) but that process generates a separate bill that can create hassles with the finance department, and such products are a little harder to find within the AWS system. So in lots of cases, enterprise software companies have struck revenue-sharing commercial deals with cloud providers to offer their services directly alongside the other basic cloud resources their mutual customers use, such as compute and storage.

That presents another way to think about the current situation: most grocery stores offer an array of brand-name products alongside store-label equivalents, which are generally cheaper. Sometimes the cheaper product is good enough, and sometimes it’s not; it all depends on the needs of the shopper.

A lot of managed services offered by cloud providers are simply “good enough.” AWS is a formidable company that offers a huge array of internally developed services, but there are still lots of customers who are willing to pay extra for better versions, as the performance of MongoDB’s Atlas cloud database over the last quarter shows.

MongoDB actually highlights this when comparing Atlas against AWS’s DocumentDB and Microsoft’s Azure Cosmos DB: if you choose the “strip-mined” versions of the MongoDB open-source project offered by the two cloud providers, you get much more basic features than if you choose MongoDB’s service.

Several of the companies listed in The Times’ story signed these kinds of revenue-sharing deals earlier this year with Google, citing the much simpler billing process as one of the benefits from such an arrangement and a non-compete vow from Google, which was far later to the cloud market than AWS and offers fewer overall services. While it’s not clear how far they’ve gotten in talks with AWS, several of those companies have indicated to me in the past that they would sign similar deals with AWS if the price was right.

And this is the biggest problem with the article: AWS has a solid chance to eventually become the most powerful enterprise computing vendor this world has ever seen, and it deserves scrutiny. The cloud has changed nearly everything about enterprise computing, and the time will come when regulators might need to think differently about how a dominant cloud vendor exerts power over the market.

The relationship between AWS and the open-source community has also been strained because the cloud leader hasn’t always been the most enthusiastic participant in the open-source community, using a lot of open-source software but contributing little back. This has nothing to do with business-model decisions made around those projects, but it has rubbed a lot of people the wrong way with some justification. (AWS seems to have realized its error here, elevating open-source contributions as a corporate priority over the last couple of years.)

But in flubbing its interpretation of the underlying issues, the Times only gives fuel to another narrative enjoyed by thin-skinned Big Tech companies: that “the media” is out to get them. It offers no solution to these issues (other than vague talk of antitrust), and fails to note that cloud computing has sparked an explosion of competition and choice within enterprise software compared to earlier eras.

AWS has less than 50 percent of the cloud infrastructure market. The cloud infrastructure market still represents a small fraction of the $3.7 trillion (yes, trillion) that will be spent on information technology services worldwide this year. And while AWS is much bigger than Microsoft and Google, the latter two are growing more quickly than the cloud leader.

Informed analysis of the current situation would have noted all of that, as well as the fact that independent software vendors are under no obligation to offer their work under permissive open-source licenses if they are worried about Big Cloud. If the Times wants to cover enterprise computing I’m all for it, but parachuting into this world with a take that off-balance prompted a lot of head-scratching with the tech community Monday.