Automated machine learning
Automated machine learning (AutoML) is the process of automating the process of applying machine learning to real-world problems. AutoML covers the complete pipeline from the raw dataset to the deployable machine learning model. AutoML was proposed as an artificial intelligence-based solution to the ever-growing challenge of applying machine learning.[1][2] The high degree of automation in AutoML allows non-experts to make use of machine learning models and techniques without requiring to become an expert in this field first.
| Part of a series on |
| Machine learning and data mining |
|---|
 |
|
Theory |
|
Machine-learning venues |
Automating the process of applying machine learning end-to-end additionally offers the advantages of producing simpler solutions, faster creation of those solutions, and models that often outperform hand-designed models.
Comparison to the standard approach
In a typical machine learning application, practitioners have a set of input data points to train on. The raw data may not be in a form that all algorithms can be applied to it. To make the data amenable for machine learning, an expert may have to apply appropriate data pre-processing, feature engineering, feature extraction, and feature selection methods. After these steps, practitioners must then perform algorithm selection and hyperparameter optimization to maximize the predictive performance of their model. All of these steps induce challenges, accumulating to a significant hurdle to get started with machine learning.
AutoML dramatically simplifies these steps for non-experts. Automation is not perfect because AutoML tools still have hyperparameters, and setting them may require expertise.
Targets of automation
Automated machine learning can target various stages of the machine learning process.[2] Steps to automate are:
- Data preparation and ingestion (from raw data and miscellaneous formats)
- Column type detection; e.g., boolean, discrete numerical, continuous numerical, or text
- Column intent detection; e.g., target/label, stratification field, numerical feature, categorical text feature, or free text feature
- Task detection; e.g., binary classification, regression, clustering, or ranking
- Feature engineering
- Feature selection
- Feature extraction
- Meta learning and transfer learning
- Detection and handling of skewed data and/or missing values
- Model selection
- Hyperparameter optimization of the learning algorithm and featurization
- Pipeline selection under time, memory, and complexity constraints
- Selection of evaluation metrics and validation procedures
- Problem checking
- Leakage detection
- Misconfiguration detection
- Analysis of results obtained
- User interfaces and visualizations for automated machine learning
References
- Thornton C, Hutter F, Hoos HH, Leyton-Brown K (2013). Auto-WEKA: Combined Selection and Hyperparameter Optimization of Classification Algorithms. KDD '13 Proceedings of the 19th ACM SIGKDD international conference on Knowledge discovery and data mining. pp. 847–855.
- Hutter F, Caruana R, Bardenet R, Bilenko M, Guyon I, Kegl B, and Larochelle H. "AutoML 2014 @ ICML". AutoML 2014 Workshop @ ICML. Retrieved 2018-03-28.