Elasticsearch

Shay Banon created the precursor to Elasticsearch, called Compass, in 2004.[6] While thinking about the third version of Compass he realized that it would be necessary to rewrite big parts of Compass to "create a scalable search solution".[6] So he created "a solution built from the ground up to be distributed" and used a common interface, JSON over HTTP, suitable for programming languages other than Java as well.[6] Shay Banon released the first version of Elasticsearch in February 2010.[7]

Elastic NV was founded in 2012 to provide commercial services and products around Elasticsearch and related software.[8] In June 2014, the company announced raising $70 million in a Series C funding round, just 18 months after forming the company. The round was led by New Enterprise Associates (NEA). Additional funders include Benchmark Capital and Index Ventures. This round brought total funding to $104M.[9]

In March 2015, the company Elasticsearch changed their name to Elastic.[10]

In June 2018, Elastic filed for an initial public offering with an estimated valuation of between 1.5 and 3 billion dollars.[11] On 5 October 2018, Elastic was listed on the New York Stock Exchange.[12]

Elasticsearch can be used to search all kinds of documents. It provides scalable search, has near real-time search, and supports multitenancy.[4] "Elasticsearch is distributed, which means that indices can be divided into shards and each shard can have zero or more replicas. Each node hosts one or more shards, and acts as a coordinator to delegate operations to the correct shard(s). Rebalancing and routing are done automatically".[4] Related data is often stored in the same index, which consists of one or more primary shards, and zero or more replica shards. Once an index has been created, the number of primary shards cannot be changed.[13]

Elasticsearch is developed alongside a data collection and log-parsing engine called Logstash, an analytics and visualisation platform called Kibana, and Beats, a collection of lightweight data shippers. The four products are designed for use as an integrated solution, referred to as the "Elastic Stack" (formerly the "ELK stack").[14]

Elasticsearch uses Lucene and tries to make all its features available through the JSON and Java API. It supports facetting and percolating,[15] [16] which can be useful for notifying if new documents match for registered queries. Another feature is called "gateway" and handles the long-term persistence of the index;[17] for example, an index can be recovered from the gateway in the event of a server crash. Elasticsearch supports real-time GET requests, which makes it suitable as a NoSQL datastore,[18] but it lacks distributed transactions.[19]

On 20 May 2019, Elastic made the core security features of the Elastic Stack available free of charge, including TLS for encrypted communications, file and native realm for creating and managing users, and role-based access control for controlling user access to cluster APIs and indexes.[20] The corresponding source code is available under the “Elastic License”, a source-available license.[21] In addition, Elasticsearch now offers SIEM [22] and Machine Learning [23] as part of its offered services.

Developed from the Found acquisition by Elastic in 2015,[24] Elastic Cloud is a family of Elasticsearch-powered SaaS offerings which include the Elasticsearch Service, as well as Elastic App Search Service, and Elastic Site Search Service which were developed from Elastic’s acquisition of Swiftype.[25] In late 2017, Elastic formed partnerships with Google to offer Elastic Cloud in GCP, and Alibaba to offer Elasticsearch and Kibana in Alibaba Cloud.

Elasticsearch Service on Elastic Cloud is the official hosted and managed Elasticsearch and Kibana offering from the creators of the project since August 2018[26][27] Elasticsearch Service users can create secure deployments with partners, Google Cloud Platform (GCP)  and Alibaba Cloud.[28][29]

AWS offers Elasticsearch as a managed service since 2015.[30][31][32] Such managed services provide hosting, deployment, backup and other support.[33] Most managed services also include support for Kibana.

Elasticsearch is the basis of Pangeanic's contribution to the EU's Marie Curie research project "EXPERT"[34] called ActivaTM. Pangeanic built a bilingual database compatible with Computer-Assisted Translation tools, which could offer real-time access via API from a variety of tools. The project received further funding from the EU as the National and European Central Translation Memory project[35] under the Connecting Europe Facility (CEF) programme. NEC TM aims to centralise national translation assets in all the EU's Member States so countries can re-use bilingual translation data produced as a result of public procurement contracts.

• 2018-11-15 AWS Elasticsearch database belonging to VoxOx exposed tens of millions of text messages, including password reset links, two-factor codes, shipping notifications and more.[36]
• 2018-11-27 Elasticsearch database belonging to Urban Massage exposed more than 309,000 user records, including names, email addresses and phone numbers.[37]
• 2019-01-12 Elasticsearch server belonging to do-it-yourself chain, B&Q exposed personal details of individuals caught or suspected of stealing goods from stores.[38][39]
• 2019-01-21 Elasticsearch database belonging to Youth-run agency AIESEC exposed over 4 million intern applications including the applicant’s name, gender, date of birth, and the reasons why the person was applying for the internship.[40]
• 2019-01-23 Elasticsearch database belonging to Ascension Data and Analytics exposed 24 million financial and banking documents, representing tens of thousands of loans and mortgages from some of the biggest banks in the U.S.[41]
• 2019-09-13 Elasticsearch database belonging to Dealer Leads exposed 198 million car buying records which contained the personal information of customers.[42]
• 2019-10-26 Elasticsearch database belonging to Adobe exposed 7.5 million customer records which contained email addresses, Adobe member IDs (usernames), country of origin, and what Adobe products they were using.[43]
• 2019-11-19 Elasticsearch database belonging to Conrad Electronic exposed 14 million customer records which contained postal addresses, in parts fax- and telephone numbers as well as IBANs on a fifth of the exposed data-records.[44]

• List of information retrieval libraries
• Information extraction

• Official website