Automated machine learning

Machine learning and data mining
Problems Classification Clustering Regression Anomaly detection AutoML Association rules Reinforcement learning Structured prediction Feature engineering Feature learning Online learning Semi-supervised learning Unsupervised learning Learning to rank Grammar induction
Supervised learning (classification • regression) Decision trees Ensembles Bagging Boosting Random forest k-NN Linear regression Naive Bayes Neural networks Logistic regression Perceptron Relevance vector machine (RVM) Support vector machine (SVM)
Clustering BIRCH CURE Hierarchical k-means Expectation–maximization (EM) DBSCAN OPTICS Mean-shift
Dimensionality reduction Factor analysis CCA ICA LDA NMF PCA t-SNE
Structured prediction Graphical models Bayes net Conditional random field Hidden Markov
Anomaly detection k-NN Local outlier factor
Neural nets Autoencoder Deep learning Multilayer perceptron RNN LSTM GRU) Restricted Boltzmann machine SOM Convolutional neural network U-Net
Reinforcement learning Q-learning SARSA Temporal difference (TD)
Theory Bias-variance dilemma Computational learning theory Empirical risk minimization Occam learning PAC learning Statistical learning VC theory
Machine-learning venues NIPS ICML ML JMLR ArXiv:cs.LG
Glossary of artificial intelligence Glossary of artificial intelligence
Related articles List of datasets for machine-learning research Outline of machine learning
Machine learning portal

Automated machine learning (AutoML) is the process of automating the end-to-end process of applying machine learning to real-world problems. In a typical machine learning application, practitioners must apply the appropriate data pre-processing, feature engineering, feature extraction, and feature selection methods that make the dataset amenable for machine learning. Following those preprocessing steps, practitioners must then perform algorithm selection and hyperparameter optimization to maximize the predictive performance of their final machine learning model. As many of these steps are often beyond the abilities of non-experts, AutoML was proposed as an artificial intelligence-based solution to the ever-growing challenge of applying machine learning.^[1][2] Automating the end-to-end process of applying machine learning offers the advantages of producing simpler solutions, faster creation of those solutions, and models that often outperform models that were designed by hand.

Targets of automation

Automated machine learning can target various stages of the machine learning process:^[2]

• Automated data preparation and ingestion (from raw data and miscellaneous formats)
  • Automated column type detection; e.g., boolean, discrete numerical, continuous numerical, or text
  • Automated column intent detection; e.g., target/label, stratification field, numerical feature, categorical text feature, or free text feature
  • Automated task detection; e.g., binary classification, regression, clustering, or ranking
• Automated feature engineering
  • Feature selection
  • Feature extraction
  • Meta learning and transfer learning
  • Detection and handling of skewed data and/or missing values
• Automated model selection
• Hyperparameter optimization of the learning algorithm and featurization
• Automated pipeline selection under time, memory, and complexity constraints
• Automated selection of evaluation metics / validation procedures
• Automated problem checking
  • Leakage detection
  • Misconfiguration detection
• Automated analysis of results obtained
• User interfaces and visualizations for automated machine learning

Examples

Software tackling various stages of AutoML:

See also

• Hyperparameter optimization
• Model selection
• Neuroevolution
• Self-tuning

References