Charlie Hull
Category: Elasticsearch
Ironically, I’ve been writing recently about how search teams can gain better control of their search engine – but I’ve put that blog on hold for a few days to write about a different form of control, as Elastic, the company formed by the creators of open source search engine Elasticsearch, have now changed the licensing terms for future releases of this software.
An Elasticsearch timeline
- Elasticsearch was originally released as open source by Shay Banon in 2010 under the standard Apache 2 open source license, as a search server built on the Lucene library (also Apache 2 licensed and hosted by the Apache Foundation). In 2012 he formed Elastic as a business to sell subscriptions, hosting and training around Elasticsearch. This company has been hugely successful, attracting over $162m in funding and went public in 2018 with a valuation between $1.5bn and $3bn. Elastic employs some Lucene committers and has contributed significantly to the development of Lucene and Elasticsearch.
- Subsequently, Elastic released some enhancements under their own ‘source available’ license, the ‘Elastic license’. Note that any contributions to Elasticsearch (under either license) are managed by Elastic: if you’re not an Elastic employee you don’t get to decide what gets committed. Occasionally they have been criticized for putting some new features under this license that started as open, community discussions and ideas, and it was clear that this license was not an approved Open Source license. This model is ‘open code’ – but for a certain value of ‘open’, with centralised ontrol over the roadmap and direction of the product.
- Amazon and some smaller companies began to offer hosted Elasticsearch in a Software as a Service model (SaaS), competing directly with Elastic’s offering. Elastic began to intermingle the parts of their code licensed as Apache 2 and Elastic License to make it difficult to just use the open source parts, and Amazon went further by effectively forking Elasticsearch to offer a totally Apache 2 licensed version they named Elasticsearch OpenDistro. Elastic also sued Floragunn GmbH, who offer a security plugin that forms part of OpenDistro, and separately AWS for trademark infringement.
- Yesterday we heard that Elastic plan to release all future versions of Elasticsearch under a dual-licensing model, using their Elastic license and the SSPL – a license created by MongoDB specifically to prevent third parties such as Amazon from offering hosted versions of their software.
So is Elasticsearch still open source?
Neither of the new licenses are approved by the Open Source Initiative (OSI), the generally accepted arbiters. The new licenses restrict certain freedoms that were permitted under the Apache 2 license – in short, if you want to provide Elasticsearch on a SaaS basis, you have to release any code that you use to do this: in Amazon’s case this could mean all the management layers that go into providing Elasticsearch on Amazon Web Services (AWS), so I doubt this is going to happen. The SSPL itself was apparently withdrawn by the originators from consideration by the OSI, so we can assume they didn’t think it would be approved as open source.
Smaller SaaS providers are probably tearing their hair out over this move, and suffice it to say that the reaction on social media hasn’t been very positive, especially from contributors to Elasticsearch.
What is important here is to realise that if a piece of software is controlled by one company, then that company can always change the game if they feel there are commercial reasons to do so (and fighting off a giant competitor such as Amazon is a pretty solid reason). Shay Banon has written that they “believe deeply in the principles of free and open products, and of transparency with the community” but the company also said as late as December 2020 that their core code would be Apache 2 licensed ‘now and always’, so this repeated messaging from the founder is beginning to feel a little disingenuous.
So by the official definition, the answer is no. Personally, I feel a little saddened by this, although at OSC we also have some sympathy with Elastic.
What does this mean for users?
This will depend on which sort of user you are. Amazon will probably continue with their fork of OpenDistro, perhaps renaming it to get away from trademark issues, although they will be unable to pull in future versions of Elasticsearch core code, meaning their version will diverge from Elastic’s. Smaller SaaS providers without Amazon’s resources will have to decide whether to do a deal with Elastic or Amazon to continue to offer a hosted Elasticsearch. Those using Elasticsearch to provide any kind of online service will have to instruct their lawyers to consider what the SSPL/Elastic licenses allow them to do (unfortunately I agree with Vicky Brasseur that this is a potential business risk). If you consider the risks too high, or your trust in Elastic or Amazon to not move the goalposts any further has been lost, perhaps you’ll be considering migrating to Apache Solr or Vespa. If you’re someone who has or wants to contribute to Elasticsearch you will have to consider whether you will continue to support the project under the new terms. If you’re just starting out in the search game you’ll have to consider how important true open source – as opposed to just ‘open code’ – is to you.
At OSC we have long experience with both Elasticsearch and Solr – so do get in touch if you need an expert guide through this new landscape of open (source?) search.
Image from Lock Vectors by Vecteezy