Forecasting ERCOT demand one hour ahead
Electricity demand changes continuously with weather, time of day, and human activity. Accurate short-term forecasts help grid operators schedule generation, balance supply and demand, and respond to changing conditions without carrying unnecessary operational cost.
This project explores that problem using an XGBoost regressor trained on historical ERCOT demand and Central Texas weather data. The final model produced a one-hour-ahead forecast with a mean absolute percentage error of 0.93% on the out-of-sample test period.
The objective was not simply to fit a high-performing model, but to build the forecast in a way that resembles its eventual use. Every input needed to be available at the moment of prediction, the test period needed to come strictly after the training period, and performance needed to be compared with forecasts simple enough to expose whether the added complexity was worthwhile.
Data and feature engineering
The demand dataset covers hourly ERCOT system load from 2017 through mid-2025. Weather observations came from Meteostat and included temperature, dew point, perceived temperature, and related measurements from a Central Texas station.
Raw observations were expanded into features designed to capture both immediate movement and repeating seasonal behavior:
- Hour, weekday, month, season, weekend, and school-day indicators.
- Demand and weather lags at 1, 2, 3, 22, 23, and 24 hours.
- Rolling means and standard deviations over 6, 12, and 24-hour windows.
- Heating and cooling degree days, along with nonlinear weather terms.
Time variables were encoded so the model could distinguish working days from weekends and capture recurring hourly and monthly behavior. Lag intervals clustered around the most recent hours and the same time on the previous day, reflecting the strong autocorrelation visible in electricity demand.
Rolling statistics added another view of the system. A single lag describes one earlier observation, while rolling means and standard deviations describe the local level and volatility leading into a prediction. The weather-derived variables were intended to represent the nonlinear way heating and cooling requirements respond as temperatures move away from a comfortable range.
Modeling without looking into the future
The train-test split was chronological rather than random. Data from 2017 through the end of 2023 was used for training, while the most recent 19 months were held out for testing. This better represents real forecasting and prevents future observations from leaking into the model.
XGBoost was selected because boosted trees work well with engineered time, weather, and categorical features. Training used a learning rate of 0.1, 50 estimators, and early stopping to limit overfitting while retaining useful predictive capacity.
The workflow followed four deliberate stages: load and clean the source data, divide it chronologically, engineer features independently within the training and test periods, and then train and evaluate the model. Temporal shifts were checked carefully so a row could not accidentally inherit information from a future hour.
Hyperparameters were tuned manually rather than through a full grid search. That limits how confidently the chosen settings can be called optimal, but it kept the experiment focused on establishing a leakage-safe baseline. Validation performance began to flatten after roughly 50 boosting rounds, making the smaller model a reasonable choice for this iteration.
Results against simple baselines
On the unseen test period, the XGBoost model achieved a MAPE of 0.93% and an RMSE of 675 MW. Two deliberately simple forecasts provided context: carrying forward the previous hour produced a 2.67% MAPE, while a 24-hour rolling average produced a 9.47% MAPE.
A weeklong comparison shows the model following the daily demand curve closely, including steep morning ramps and evening peaks. The benchmark models either react one step late or smooth away too much short-term movement.
The previous-hour baseline is difficult to beat casually because electricity demand usually changes gradually. Its weakness becomes visible around ramps and turning points, where repeating the last value introduces a consistent delay. The rolling-average baseline has the opposite problem: it is stable, but suppresses the daily peaks and valleys that matter to short-term operations.
Viewed in that context, the XGBoost result suggests that the engineered features are doing more than reproducing the latest observation. The model tracks both the level and changing direction of demand while maintaining a relatively small error across a long test window that includes different seasons.
Where the model still struggles
Residuals were centered near zero without an obvious directional bias, but their variance increased when demand rose above roughly 60,000 MW. These extreme periods appear less frequently in the training data, so the model has fewer examples from which to learn.
Lagged demand—especially the most recent hour—was the strongest predictor. Weather and calendar features offered smaller gains, but together improved MAPE by about 10% compared with a model based only on lagged demand.
A model trained only on lagged target variables reached approximately 1.03% MAPE. The difference between that result and 0.93% is modest in absolute terms, but it indicates that weather and time features still contribute information beyond demand history. Their value may be most important near changes in temperature or daily operating patterns rather than during ordinary hours.
The widening residual spread at high demand is an important qualification to the aggregate score. ERCOT load is concentrated nearer 40,000 MW and has a thinner upper tail, so extreme conditions receive less representation during training. Those are also the hours when forecast reliability can matter most, making performance by demand regime as important as overall average error.
Limitations and next steps
The current model is designed specifically for one-hour forecasting. A quick 24-hour test reached approximately 4.75% MAPE, but performed similarly to using demand from the same hour one day earlier. Meaningful day-ahead forecasting would require a different feature and validation strategy.
The weather data also comes from a single station. A future version could combine observations from multiple locations and use zonal ERCOT demand to build more geographically specific forecasts. That would provide a clearer view of how localized weather patterns contribute to system-wide load.
Longer horizons also change which information is available. A one-hour model can lean heavily on current demand, while a day-ahead model must rely more on weather forecasts, calendar structure, and expected operating conditions. Extending the existing model without redesigning those inputs would risk presenting a precise-looking result that adds little beyond a seasonal baseline.
A stronger next iteration would use weather stations distributed across Texas, evaluate errors separately for extreme temperatures, and compare zonal models with the system-wide forecast. Formal time-series cross-validation and broader parameter search could then test whether the current result remains stable across multiple historical windows.
Conclusion
The project demonstrates how careful time-series design can turn a familiar tree-based model into a strong short-term forecaster. More importantly, it highlights the practical details behind the headline metric: chronological validation, leakage-safe features, meaningful baselines, and honest attention to the conditions where performance begins to weaken.
The 0.93% test error is encouraging, but the more useful outcome is a repeatable framework for asking better forecasting questions. It separates the parts of performance explained by recent demand from the smaller contribution of weather and calendar information, and it makes the remaining weaknesses visible enough to guide the next version.