Bayesian nonlinear modeling for the prediction competition

Conference · Sat Dec 31 04:00:00 EST 1994

OSTI ID:33309

MacKay, D J.C. ^[1]

---

1. Cavendish Lab., Cambridge (United Kingdom)

The 1993 energy prediction competition involved the prediction of a series of building energy loads from a series of environmental input variables. Nonlinear regression using neural networks is a popular technique for such modeling tasks. Since it is not obvious how large the input time-window should be or what preprocessing of inputs is best, this can be viewed as a regression problem in which there are many possible input variables, some of which may actually be irrelevant to the prediction of the output variable. Because a finite data set will show random correlations between the irrelevant inputs and the output, any conventional neural network (even with regularization or ``weight decay``) will not set the coefficients for these junk inputs to zero. Thus, the irrelevant variables will hurt the model`s performance. The automatic relevance determination (ARD) model puts a prior probability distribution over the regression parameters that embodies the concept of relevance. This is done in a simple and ``soft`` way by introducing multiple regularization constants -- one associated with each input. Using Bayesian methods, the regularization constants for junk inputs are automatically inferred to be large, preventing those inputs from causing significant over fitting. An entry using the ARD model won the competition by a significant margin.

• OSTI ID: 33309
• Report Number(s): CONF-9406105--
• Country of Publication: United States
• Language: English

Related Subjects

32 ENERGY CONSERVATION, CONSUMPTION, AND UTILIZATION
BUILDINGS
CLIMATES
COMPUTERIZED SIMULATION
ENERGY DEMAND
FORECASTING
NEURAL NETWORKS
SEASONS
WEATHER