Apache Kylin

Apache Kylin is an open source distributed analytics engine designed to provide a SQL interface and multi-dimensional analysis (OLAP) on Hadoop and Alluxio supporting extremely large datasets.

Apache Kylin
Developer(s)Apache Kylin Committee
Initial releaseJune 10, 2015 (2015-06-10)[1]
Stable release
2.6.3 / July 6, 2019 (2019-07-06)[2]
Preview release
3.0.0-alpha2 / July 31, 2019 (2019-07-31)[3]
RepositoryKylin Repository
Written inJava
LicenseApache License 2.0
Websitekylin.apache.org

It was originally developed by eBay, and is now a project of the Apache Software Foundation.[4]

History

The Kylin project was started in 2013, in eBay's R&D in Shanghai, China. In Oct 2014, Kylin v0.6 was open sourced on github.com with the name "KylinOLAP".[5]

In Nov 2014, Kylin joined Apache Software Foundation incubator.

In December 2015, Apache Kylin graduated to be a Top Level Project.[4]

Kyligence Inc, a Chinese company, provides commercial support for Apache Kylin and consists primarily of the original developers of the open source code.[6]

Architecture

Apache Kylin is built on top of Apache Hadoop, Apache Hive, Apache HBase, Apache Calcite, Apache Spark and other technologies. These technologies are very powerful and matured, making Kylin able to support massive data load, and easily scale.

Kylin has the following core components:

  • REST Server: Receive and response user or API requests
  • Metadata: Persistent and manage system, especially the cube metadata;
  • Query Engine: Parse SQL queries to execution plan, and then talk with storage engine;
  • Storage Engine: Pushdown and scan underlying cube storage (default in HBase);
  • Job Engine: Generate and execute MapReduce or Spark job to build source data into cube;

Users

Apache Kylin has been adopted by many companies as their OLAP platform in production. Typical users includes eBay, Meituan, XiaoMi, NetEase, Beike, Yahoo! Japan.

Roadmap

Apache Kylin roadmap (from Kylin website[7]):

  • Hadoop 3.0 support (Erasure Coding)
  • Fully on Spark Cube engine
  • Connect more data sources (MySQL, Oracle, SparkSQL, etc)
  • Ad-hoc queries without Cubing
  • Better storage (Druid, Kudu, etc)
  • Real-time analytics with Lambda Architecture

References
  1. "Previous Release". v0.7.1-incubating (First Apache Release). Retrieved 15 June 2019.
  2. "Previous Release". v2.6.3. Retrieved 15 July 2019.
  3. "Apache Kylin - Release Notes". v3.0.0-alpha2 - 2019-07-31. Retrieved 5 August 2019.
  4. Apache Software Foundation. "The Apache Software Foundation Announces Apache™ Kylin™ as a Top-Level Project", 8 December 2015
  5. "Announcing Kylin: Extreme OLAP Engine for Big Data". www.ebayinc.com. 2014-10-20. Retrieved 2018-11-08.
  6. "Kyligence announces funding building enterprise software around Apache Kylin".
  7. Kylin, Apache. "Apache Kylin | Development Quick Guide". kylin.apache.org. Retrieved 2018-11-08.


This article is issued from Wikipedia. The text is licensed under Creative Commons - Attribution - Sharealike. Additional terms may apply for the media files.