Automated Machine Learning (AutoML)

Tackling a predictive modeling task with machine learning requires the design of a suitable "machine learning pipeline”, i.e., the selection and parameterization of machine learning algorithms for specific subtasks and their combination into an overall solution. Doing this manually is difficult and often cumbersome because the space of candidate pipelines is huge. For example, most machine learning algorithms have parameters themselves, called hyperparameters (to distinguish them from the parameters of models learned by the algorithm). These may have a strong influence on an algorithm’s performance, i.e., the quality of models induced by the algorithm, but systematically searching for optimal hyperparameter configurations is a tedious and time-consuming task.

In response to this, and in light of the increasing need for practical ML solutions, automated machine learning (AutoML) has recently emerged as a new branch of ML research. AutoML is commonly understood as the task of automating the process of engineering a machine learning pipeline specifically tailored to a problem at hand. Thus, compared to “basic” machine learning algorithms such as neural networks, which solve a concrete learning task, an AutoML tool can be seen as solving a “learning to learn” problem. For the standard problem classes such as classification and regression, several AutoML tools have been proposed in the last couple of years, and their performance has been demonstrated quite impressively in several experimental studies. In particular, this includes methods for algorithm selection(i.e., given a set of candidate algorithms, selecting the one that is best suited for the problem at hand) and algorithm configuration (i.e., selecting optimal hyperparameters of an ML algorithm).

Selected Publications