Charlie Hull
Category: Elasticsearch
As I wrote only a few days ago, Elastic, the company formed by the creators of Elasticsearch, will be releasing future versions of this software under a non open source license, to prevent competition in paid hosting from Amazon Web Services. As many have predicted, AWS announced that they will be forking the codebase and releasing their own version of Elasticsearch under an open source license. Encouragingly, other companies are promising similar efforts including Logz.io, a major user of Elasticsearch for log monitoring applications (I also hear rumours about some other big companies coming on board).
The announcements are reassuring in tone and make much of the preservation of the open source ideals that Elastic have been accused by some of abandoning. However, and I cannot stress this enough, open source is not just about releasing software under a particular license. An open source project without an active and healthy community is simply casting ideas and ideals into the wind and hoping they’ll stick – and far too often this leads to unclear project direction, lack of innovation and even ‘abandonware’, projects that sit there sadly with less than one active maintainer and no upgrades for the last few years (but hey! it’s open source!).
Commitment to community
So what would it take to build this healthy community? I’m going to suggest some ways that Amazon and others can use some of their significant revenue from Elasticsearch services or products to support this movement. Actually, I’m going to challenge these companies, who have written that they are standing up for an open source ideal in the face of a protectionist move by Elastic, to make it happen, and to do this they will need to put their hands in their often very deep pockets! A truly open Elasticsearch community will benefit everyone.
Open and neutral governance
Let’s hear how the project will be managed in the future. There are existing models that can be copied (Apache’s Project Management Committee is one good example). The key here is to ensure that no one organisation gains effective control over the project and that decisions on future roadmaps etc. are made to benefit the community, not just special interests. Those who manage the project should accept criticism while maintaining polite discourse – develop a code of conduct. I’d love to see this software eventually become an Apache project as Logz.io have suggested, joining its ancestor Lucene and sibling Solr – why not set out that kind of ambition for open discussion now, with a potential timeline?
Build the ecosystem
Create lists of third party plugins and accessory products and make it as easy as possible to integrate these with the core software (fund any support required by third-party creators, who may not know much about the internals). Solr.cool is a good example of how to list useful add-ons for a search engine. A search engine doesn’t exist in isolation: for example, without good client libraries the adoption by a particular community using a particular language (say .NET or Python) will be harder. Promote third-party services like consulting or training – it’s easy to make a simple alphabetical list of these that doesn’t promote one provider over another. The more people who can make a living from this ecosystem the better, be they one-person shops or large integrators – they will contribute to ongoing development and can be great advocates.
More documentation
We need more blogs, articles, examples and indeed whole books on Elasticsearch. Manning’s last book on the subject was released over 5 years ago (which is more like 500 years in the fast-moving world of software). Find some independent experts and pay them for their time: writing a whole book can be a thankless and difficult task (as some of us at OSC know to our cost), but also we need short-form content, how-to examples etc. The core documentation of the software itself must be kept up to date, fund technical writers to oversee it and a process to refresh it. Find out who within your own organisation uses this software and help them write about it. Pay staff to monitor forums (not just your on own site – Stackoverflow or our Relevance Slack for example) and answer questions.
Meet the people
Search Meetups are a highly effective way to network with peers and communicate best practice and solutions, but work best if they operate in a spirit of openness. Support existing events with sponsored food, drink or a free venue (there’s quite a few Search Meetups, here’s a list I created some years ago – note that this isn’t being actively updated) but don’t expect anything in return – organisers are however always looking for interesting talks. If you’re running an event, accept any relevant talk, even if it’s from your competitor (but make sure speakers know not to pitch, this is a chance to educate, not to sell). Keep the event generally unbranded (but we all like some free swag occasionally) and don’t just schedule talks from your own team – at the Meetups I’ve run we actively avoided scheduling events if we were the only company with a talk ready to give.
Conferences
Many conferences exist that cover search topics, on a sliding scale of business models from ‘community event, no sponsors, no vendors’ (like our Haystack events) to fully ‘pay to play’ where speakers mainly come from sponsor organisations. Fund developers working on this software to attend and speak. Pay for experts to go and answer questions in ‘Ask me Anything’ sessions. Sponsor those who couldn’t afford to attend otherwise, like students or disadvantaged groups. Lead the charge – for a good example look to Bloomberg, who after adopting Apache Solr have consistently supported events such as Activate and Berlin Buzzwords and gained a great deal of credibility in the open source search world as a result (they’ve also contributed a lot of code). Remember that after this last year a lot of conference organisers are struggling, so your funds could make a real difference.
Doing it right
I really hope that after the dust has settled, we have the foundations for a truly open source Elasticsearch community, but this will take commitment from all concerned – and by commitment, I mean time and money! At OSC we’ll continue to contribute how and where we can, writing plugins, patches, blogs and more, and we’re always willing to help those navigating this rapidly changing landscape.
Over to you, AWS and others!
image by Key Vectors by Vecteezy