Evaluation of the Predictive Accuracy of Five Whole-Building Baseline Models

This report documents the relative and absolute performance of five baseline models used to characterize whole-building energy consumption. The Pulse Adaptive Model¹, multi-parameter change-point, mean-week, day-time-temperature, and LBNL models were evaluated according to a number of statistical 'goodness of fit' metrics, to determine their accuracy in characterizing the energy consumption of a set of 29 buildings. The baseline training period, prediction horizon, and predicted energy quantity (daily, weekly, and monthly energy consumption) were varied, and model predictions were compared to interval meter data to determine the accuracy of each model. Three combinations of baseline training periods and prediction horizons were considered: 6 months of training to generate a 12-month prediction; 9 months of training to generate a 7-month prediction; and 12 months of training to generate a 6- month prediction.

Although there was no single best performer, the study results showed that the LBNL model, the Pulse Adaptive Model, and the day-time-temperature model consistently outperformed the mean-week, and industry standard change-point models. In aggregate, across the three training and prediction periods that were considered, and across the three energy quantities that were predicted, the median absolute percent errors for these models ranged from 3-6% of the actual total metered energy consumption. When considering the normalized root mean squared error, monthly energy use was predicted with the least error, and daily energy was predicted with the most error. For the LBNL, day-time-temperature, and Pulse Adaptive Model, these errors ranged from 8-18%. As the training period increased and the prediction horizon decreased, the model performance improved a few percent for any given model. Other statistical metrics such as correlation, root mean squared error, and relative bias, were included in the study, in addition to an assessment of the accuracy of Pulse Energy's reported 90% confidence intervals.

The methodology developed and applied for this study addressed baseline accuracy, which comprises the first step in determining the uncertainty in the measurement and verification (M&V) of gross, whole-building energy savings. Future work will expand the analyses to account for uncertainty in the degree to which, excluding all effects but the EEMs, the building's operation and energy use during the baseline period is equal to that of the post-measure period. This will require data spanning a longer period of time, from buildings where efficiency measures have been implemented. Since the dataset for this study was limited to a small number of buildings, and not fully representative, widely generalizable conclusions cannot be established. Therefore, future work will focus on expansion of the number of datasets, and diversity of buildings to enable more generalizable findings, disaggregation of different building types, and correlation of errors with load variability.

¹ As noted in the Disclaimer, this work does not comprise a product endorsement or recommendation by the United States Government or any agency thereof, by The Regents of the University of California, by the Lawrence Berkeley National Laboratory, or by the authors.