Document Type
Lecture
Publication Date
2-17-2017
Abstract
We compare the performance of five model average predictors—stacking, Bayesian model averaging, bagging, random forests, and boosting—to the components used to form them. In all five cases we provide conditions under which the model average predictor performs as well or better than any of its components. This is well known empirically, especially for complex problems, although few theoretical results seem to be available. Moreover, all five of the model averages can be regarded as Bayesian. Stacking is the Bayes optimal action in an asymptotic sense under several loss functions. Bayesian model averaging is known to be the Bayes action under squared error. We show that bagging can be regarded as a special case of Bayesian model averaging in an asymptotic sense. Random forests are a special case of bagging and hence likewise Bayesian. Boosted regression is a limit of Bayes optimal boosting classifiers. We have limited our attention to the regression context since that is where model averaging techniques differ most often from current practice. This is joint work with Bertrand Clarke.
Relational Format
presentation
Recommended Citation
Le, Tri, "Model average versus model selection: a Bayes perspective" (2017). Colloquium. 10.
https://egrove.olemiss.edu/math_colloquium/10
