Content uploaded by Nadeem Javaid

Author content

All content in this area was uploaded by Nadeem Javaid on Mar 22, 2019

Content may be subject to copyright.

Short Term Load Forecasting

Using XGBoost

Raza Abid Abbasi1, Nadeem Javaid1(B

), Muhammad Nauman Javid Ghuman2,

Zahoor Ali Khan3, Shujat Ur Rehman2, and Amanullah2

1COMSATS Institute of Information Technology, Islamabad 44000, Pakistan

nadeemjavaidqau@gmail.com

2Quaid-i-Azam University, Islamabad 44000, Pakistan

3Computer Information Science, Higher Colleges of Technology,

Fujairah 4114, UAE

http://www.njavaid.com/

Abstract. For eﬃcient use of smart grid, exact prediction about the in-

future coming load is of great importance to the utility. In this proposed

scheme initially we converted daily Australian energy market operator

load data to weekly data time series. Furthermore, we used eXtreme Gra-

dient Boosting (XGBoost) for extracting features from the data. After

feature selection we used XGBoost for the purpose of forecasting the elec-

tricity load for single time lag. XGBoost perform extremely well for time

series prediction with eﬃcient computing time and memmory resources

usage. Our proposed scheme outperformed other schemes for mean aver-

age percentage error metric.

1 Introduction

Energy production and consumption diﬀerence minimization is a challenging

task these days. Eﬃcient consumption of electricity is one good solution to this

problem. Researchers has done a lot of work in for introducing cost eﬀective

and eﬃcient energy utilization systems. Next generation Smart Grid (SG) is

the most attractive solution so far. SG is the integration of information and

communication technology in traditional grid which makes it intelligent power

grid supporting real-time information exchange between producer and consumer.

SG enables the energy eﬃciency optimization. More precisely, the SG needs an

accurate forecasting of the energy load for more productive application.

AN increasing attempt of deregulating strength markets to shape a more

dependable, green, and price-eﬀective system with the aid of improving com-

petitions has been witnessed in world’s important economies [1,2]. In the lib-

eralized markets, the strength is commoditized and consequently its price is

dynamic. Due to the fee variant, pricing power correctly will become important

to generate proﬁts, schedule strength productions, and plan load responses [3–

7]. The accurate power charge forecasting is helpful to decide the strength rate

and accordingly is precious. As liberalized electricity markets include types, day-

ahead and real-time [8], it’s miles meaningful to discuss both of the day-ahead

c

Springer Nature Switzerland AG 2019

L. Barolli et al. (Eds.): WAINA 2019, AISC 927, pp. 1120–1131, 2019.

https://doi.org/10.1007/978-3-030-15035-8_108

STLF Using XGBoost 1121

and online forecasting of the power fee. The energy rate forecasting has been

vigorously studied inside the literature. From the application aspect, the fore-

casting of the energy rate in unique deregulated markets of essential economies

around the world has been stated [9–15].

The Rest of the paper is structured as deﬁned next. The mostly used practi-

cle techniques for load forecasting are discussed in Sect. 2. The Sect. 3enlightens

the diﬀerent problems associated with the load prediction. The proposed model

for load prediction and the evaluation metrices are explained in Sect.4. Exper-

imental results are depicted and highlighted in Sect.5. Conclusion about the

work done in this paper is expressed in Sect. 6at the end.

2 Related Work

Authors in [16] proposed a novel practical methodology using quantile regression

mean on a set of sisters point forecasts. Data from GEFCom2014 probabilistic

load track was used to forecast for developing probabilistic load forecasts. The

suggested scheme has dual advantages, where It will strength the advancement in

the ﬁeld of point load forecasting, it is not dependent on the high quality expert

predictions. Scheme proposed in this work produces better results as compared to

the benchmark methods. “Recency eﬀect” a psychology term is used by authors

in [17]. They exploited the fact that power consumption demand is inﬂuenced by

the temperature of the earlier hours. Authors produced a ample study to show

the eﬀect of recency with the help of big data. Modern computing power is used

in order to decide how many lagged temperature are required for catching the

recency eﬀect completely without aﬀecting the predicting accuracy.

In [18] authors proposed a scheme for big data analytics in smart grid which

aim at reducing the electricity cost for users. Moreover they explored the individ-

ual components needed for an improved decision support system for the purpose

of energy saving. The presented framework has four diﬀerent layers in its archi-

tecture, i.e., smart grid, data accumulation, an analysis counter and supporting

web portal. Future power consumption is fore-casted and optimized through a

innovative composite nature inspired meta heuristic prediction scheme. A versa-

tile optimization algorithm works as a backbone for the analytics counter that

helps in achieving accurate results. The proposed novel framework is the major

contribution of this work, which supports the energy saving decision process.

This contribution is the basis for full scale, Smart Decision Support System

(SDSS). SDSS can identify the usage pattern of an individual user which helps

in enhancing the eﬃciency of energy usage where improving the accuracy of fore-

casted energy demands. Authors in [19] used forecasting analytics while focusing

the extraction of related external features. More explicitly the proposed scheme

predicts the spot prices in German energy market in relation to the historical

data of prices and weather features. Least Absolute Shrinkage Selection Opera-

tion (LASSO) ﬁnds the related weather stations where implicit variable selection

is executed by Random Forest (RF). This work enhanced the prediction accuracy

with respect to Mean Average Error (MAE) by 16.9%.

1122 R. A. Abbasi et al.

A novel modeling scheme for electricity price prediction is introduced in [20].

Four diﬀerent deep learning models are suggested by Lago et al. for forecast-

ing electricity prices that lead to advancement in forecasting accuracy. Authors

proposed despite the presence of a good number of electricity price forecast-

ing methods, still a benchmark is missing. This work compared and evaluated

27 diﬀerent common techniques used for electricity price prediction and then

proved how the proposed models outperform the state of the art techniques

those are signiﬁcant. Wang et al. used Stacked De-noising Auto-encoder and

Random Samples RS-SDA for live and next day hourly price prediction. In [21]

short term forecasting of the electricity price is performed using data driven

scheme. Deep Neural Networks type, SDA and its extended version RS-SDA are

used to forecast the electricity price hourly for the data collected from diﬀerent

states of United States. This research is focused on next day hourly prediction

and the live hourly prediction. SDA deﬁned models are assessed in comparison

with conventional neural network and support vector machine, where next day

prediction SDA models accuracy is assessed in comparison industrial model.

In [22] Lagoa et al. introduced two distinct schemes for combining market

incorporation in energy price prediction and to enhance the forecasting depic-

tion. First scheme suggested a DNN that examines features from linked markets

to enhance the forecasting results in a community market. Features importance

is calculated using a innovative feature selection scheme that contains the opti-

mization and functional analysis of variance. Second scheme forecasts the prices

from two adjacent markets simultaneously which bring the accuracy metric Sym-

metric Mean Absolute Error (SMAPE) even further lower. Raviv et al. worked

on predicting next day energy prices while utilizing hourly prices in [23]. This

work exhibit that dismantled hourly rates include handy forecasting facts for the

daily typical prices in the Nord pool market. It is evaluated that the multivari-

ate patterns for the complete group of hourly prices considerably go better than

univariate patterns of the daily normal price. Multivariate models reduce RMSE

upto 16%. In [24] authors worked on electrical load forecasting on the basis of

pre analysis and weight coeﬃcients optimization. A novel scheme is introduced

exploiting the features of electrical load data i.e., capacity to eﬀectively calcu-

late the seasonality and nonlinearity. The proposed new scheme can use up the

advantages stay away from disadvantages of the individual schemes. In suggested

combined scheme the data fore analyzation is adapted so that conﬂicts can be

minimized in the data, where weight factors are adjusted using cuckoo search

in the combined model. The newly proposed scheme outperforms the individual

forecasting models regarding forecast performance.

Singh et al. worked on the amount of power consumed prediction in [25].

An intellectual data mining scheme is proposed that can evaluate, predict and

reﬂect electricity time series to disclose numerous temporary energy using pat-

terns. These patterns help to identify appliance usage relationship with time

i.e., hour of day, week, month e.t.c, and appliance usage relationship with other

appliances. This identiﬁcation basis for the understanding the customer usage

behavior, energy load forecasting and the price forecasting. Authors proposed

STLF Using XGBoost 1123

Bayesian network forecasting, constant analysis of data through data mining and

unsupervised data accumulating for electricity consumption prediction. In [26]

authors proposed a short-lived electricity load prediction scheme for academic

buildings. This work used 2-stage forecasting analysis for the productive working

of their energy system. Energy consumption data is collected from diﬀerent uni-

versities and moving average method is used for ﬁnding the energy load pattern

according to week day. Random Forest (RF) technique is used for forecasting

the daily energy load. RF performance is assessed using cross-validation on time

series.

Gonz´alez et al. predicted electricity price adopting functional time series

using a New Hilbertian ARMAX model in [27]. Suggested scheme has a lin-

ear regression structure, where functional variables are operated by functional

parameters. Where functional parameters are fundamental entities with linearly

combined kernels as sigmoid operations. Quasi-Newton model is used for param-

eters optimization in sigmoid which minimizes the sum of squared error. Data

integrity attacks aﬀect the results of load prediction models i.e., artiﬁcial neural

network, multiple linear regression, support vector regression and fuzzy interac-

tion regression). Authors in [28] worked on exposing the consequences of these

attacks. We begin by simulating some knowledge integrity attacks through the

random injection of some multipliers that follow a traditional or uniform distri-

bution into the load series. Then, the four same load prognostication models are

used to generate one-year-ahead ex post purpose forecasts so as to supply a com-

parison of their forecast errors. The results show that the support vector regres-

sion model is most robust, followed closely by the multiple rectilinear regression

model, whereas the fuzzy interaction regression model is that the least sturdy of

the four. withal, all four models fail to supply satisfying forecasts once the size

of the info integrity attacks becomes giant. This presents a serious challenge to

each load forecasters and therefore the broader prognostication community: the

generation of correct forecasts beneath knowledge integrity attacks.

Dong et al. worked on the energy management in a microgrid. Bayesian-

optimization-algorithm (BOA) is used for a single SG using house. Authors

in [29] articulates the enhancement beyond the closed form equitable function

equation, and work out on it using BOA based data-driven technique. We can

consider the suggested technique as a black box function improving technique

as a whole. Furthermore, it has the ability to handle the microgrid working and

argument forecasting ambiguity.

3 Motivation and Problem Statement

Electricity load prediction is an important part of advanced power systems

i.e., SG, eﬀective power controlling, and improved energy operation engineering.

Therefore, highly accurate prediction is needed for diﬀerent perspectives, that

are related to control, forwarding, planning and unit responsibility in a grid.

Artiﬁcial Intelligence (AI) centered schemes has high competency to manipu-

late complicated mathematical problems, therefore, these techniques are widely

employed in number of research areas.

1124 R. A. Abbasi et al.

The Artiﬁcial Neural Network (ANN) outperforms statistical schemes, as

ANN is more eﬃcient in mapping inputs to the outputs beyond complicated

mathematical designs. Diverse learning structures are used by ANN for exploit-

ing the linear association among the inputs [30]. ANN schemes has better depic-

tion than analytical and time series techniques for prediction problems. The

prediction performance in neural network is enhanced by the pre-processing of

training data, high equivalence impact, optimal network structure and better

learning algorithm. Moreover, ANN brings rapid conﬂuence, minimized comput-

ing complexity, minimal training period and improved generalization [31].

Given a time series of 30 mins electricity loads, up to the time t, X1,.....Xt,

our goal is to predict load at time t+1, i.e., Xt+1.

4 Proposed System Model

Our proposed system forecasts the electricity load. In our proposed model we

used dataset from Australian Energy Market Operator (AEMO). In this dataset

electricity load recordings are taken after every 30 min, therefore producing 48

recordings for 48 time lags in a single day. Now if we want to predict the load of

a time lag, we have just 48 features to use. To increase the number of features

and better prediction we combine the records of week days from same week to

form a single record yielding 336 recordings for a single record, i.e we have 48 ×7

features. The proposed system model is visualized in Fig. 1.

Now we calculate the importance of each feature in updated dataset using

XGBoost feature selection technique. It help us to select the most appropriate

features for selection. We consider 35–40 features with highest importance values

for training and testing of our proposed scheme.

We have 1 year of data for 12 diﬀerent regions, bringing 12×365 records for

365 days in a year. We divide the data into training and testing data as 75%

training and 25% testing data. We train our proposed model using training data

and perform testing using testing data. Algorithmic steps followed during the

process of load prediction using XGBoost are deﬁned in Algorithm 1.

Fig. 1. System model

STLF Using XGBoost 1125

Algorithm 1. Proposed Scheme Algorithm for Load Prediction.

1: Load daily records data

2: Convert daily records data to weekly records data

3: for i←1tosize(features)do

4: Calculate feature importance for feature

5: end for

6: Select features with importance value greater then threshold

7: Divide data into training and testing data

8: Train model over training data

9: Predict load using trained model over testing data

10: Calculate accuracy

11: Calculate MAPE

12: Calculate MAE

4.1 Evaluation Metrices

We used various standards for the evaluation of our proposed prediction model

eﬃciency. The two most commonly used metrices for the measurement of predic-

tion accuracy are Mean Absolute Percentage Error (MAPE) and Mean Absolute

Error (MAE).

4.1.1 MAPE

The MAPE may be a live of prediction accuracy of a forecasting methodology for

constructing ﬁtted statistic values in statistics, speciﬁcally in trend estimation.

it always expresses accuracy as a proportion of the error. as a result of this range

may be a percentage, it may be easier to know than the opposite statistics. The

MAPE is outlined as shown in (1). Here, and area unit the actual worth and

therefore the forecast worth, severally. Also, is the number of times discovered.

MAPE =100

n

n

t=1

At−Ft

At

(1)

4.1.2 MAE

In statistics, the MAE is employed to measure however shut forecasts or predic-

tions area unit to the particular outcomes. It’s calculated by making a mean of

absolutely the variations between the prediction values and therefore the actual

ascertained values. The MAE is deﬁned as shown in (2). Wherever nine is that

the prediction price, is the actual price.

MAE =1

n

n

i=1

|fi−yi|=1

n

n

i=1

|ei|(2)

1126 R. A. Abbasi et al.

5 Simulation Results and Discussion

We evaluated the performance of our proposed forecasting technique for predict-

ing the load. We did perform a lot of experiments. The results obtained from

extensive simulation are discussed here in this section.

5.1 Data Set Description

We used AEMO load data for the year 2017. The dataset records electricity load

after every 30 mins making 48 lags daily. The considered dataset has records

about 12 diﬀerent areas for the same year 2017. We considered 365 days in a

year in the provided dataset. Here Fig.2plots the load proﬁle for individual

week days. i.e., each weekday has its own line. From the graph we can see that

Saturday has an overall highest load with respect to the other week days in the

selected week. Furthermore, we can see that Wednesday has lowest load with

respect to the other weekdays in the selected week. Where Fig.3is displaying

the load of two consecutive weeks. It is clear from Figs. 2and 3that the electricity

load data has daily as well as weekly cycles. The load at a speciﬁc time with

respect to the other day is more or less same and it rises and falls more or less

like the previous and next day. Similarly for the week, we can see that the load

at one day in a week is more or less to the same day in previous and next week.

These cycles are due to the cycles in daily human activities. For performing the

load prediction of a speciﬁc time lag we combined the daily loads for same week

to form a weekly dataset. i.e We combined the data from same weekdays to form

single row. While conversion we neglected the weekdays that were not forming

a complete week at the end of the year. This conversion provides more features

for forecasting the load of a time lag.

Fig. 2. Daily load by weekdays

STLF Using XGBoost 1127

Fig. 3. Two weeks load

5.2 Prediction Model Conﬁgurations

We used XGBoost a gradient boosting framework, introduced back in 2014.

XGBoost can be used as a forecasting technique for feature selection and load

prediction of a time lag. From prediction to classiﬁcation XGBoost has proved

its worth in terms of performance.

When we convert the dataset to form a weekly data we have 48×7 recordings

for a week. One row in the modiﬁed dataset represent a week with 336 features.

To understand the importance of features, we used XGBoost to calculate the

Feature Importance (FI) of all these features.

Figure 4is depicting the FI of all these features. The greater the FI values

means the feature will more eﬀect the load prediction. It is evident from Fig. 4

that features with hight importance values are less in number, where most of

the features have low importance value. We can see that the features close to

the predicting lag have high importance as well. Also it is evident that days

having same weekday for which we are predicting the load i.e., sunday have

high importance value. We will only use the features with high FI values for the

purpose of prediction and eliminate all other features from dataset. For selecting

the features, we set a threshold for feature importance and we set this threshold

by repeating experiment multiple times, and trying diﬀerent threshold value.

The value with best results in considered as threshold. The Fig. 4shows that the

features with high FI value are less in number, it will save the running time.

5.3 Forecasting Results

We used XGBoost for forecasting the load for a speciﬁc time lag in a week using

weekly data. Figure 5shows the real load in the dataset and the XGBoost fore-

casted load. Here x-axis is depicting the time lags where y-axis is the load at

1128 R. A. Abbasi et al.

Fig. 4. XGB feature importance

that speciﬁc time lag. Actual load is represented by the blue graph, where fore

casted load is represented by the green graph. We can see that the XGBoost

load prediction follows the real load at most of the time, however at some high

load instances XGBoost is not exactly following the real load. We can see that

XGBoost is not predicting well at the high loads.

Fig. 5. XGB predictions

The XGBoost load forecasting results for a time lag are displayed in Fig.6.

We can see that XGBoost forecasting technique results in a low Mean Average

Percentage Error (MAPE), high accuracy and high Mean Average Error (MAE).

XGBoost load prediction resulted in a 10.08% MAPE, 97.21% accuracy and

88.90% MAE.

STLF Using XGBoost 1129

Fig. 6. XGB results

6 Conclusion

In this paper, we proposed a new scheme for electricity load forecasting. We con-

verted daily electricity load information into weekly load information. It increases

number of features available for predicting load for a lag variable. Then, we used

XGBoost, a recently dominant machine learning technique for time series pre-

diction, for feature selection from converted data. Once features are extracted

we train the model using XGBoost. After training we use trained model for load

prediction.

References

1. Mathaba, T., Xia, X., Zhang, J.: Analysing the economic beneﬁt of electricity price

forecast in industrial load scheduling. Electr. Power Syst. Res. 116, 158–165 (2014)

2. Sarada, K., Bapira ju, V.: Comparison of day-ahead price forecasting in energy

market using Neural Network and Genetic Algorithm. In: Proceeding of the Inter-

national Conference on Smart Electric Grid, pp. 1–5 (2014)

3. Shaﬁe-Khah, M., Moghaddam, M.P., Sheikh-El-Eslami, M.: Price forecasting of

day-ahead electricity markets using a hybrid forecast method. Energy Convers.

Manag. 52(5), 2165–2169 (2011)

4. Garcia, R.C., Contreras, J., Van Akkeren, M., Garcia, J.B.C.: A GARCH forecast-

ing model to predict day-ahead electricity prices. IEEE Trans. Power Syst. 20(2),

867–874 (2005)

5. Shahidehpour, M., Yamin, H., Li, Z.: Market overview in electric power systems. In:

Market Operations in Electric Power Systems, pp. 1–20. Wiley, New York (2002)

6. Ci-wei, G., Bompard, E., Napoli, R., Cheng, H.: Price forecast in the competitive

electricity market by support vector machine. Phys. A: Stat. Mech. Appl. 382(1),

98–113 (2007)

1130 R. A. Abbasi et al.

7. Cai, Y., Lin, J., Wan, C., Song, Y.: A stochastic Bi-level trading model for an

active distribution company with distributed generation and interruptible loads.

IET Renew. Power Gener. 11(2), 278–288 (2017)

8. Weron, R.: Electricity price forecasting: a review of the state-of-the-art with a look

into the future. Int. J. Forecast. 30(4), 1030–1081 (2014)

9. Hu, L., Taylor, G.: A novel hybrid technique for short-term electricity price fore-

casting in UK electricity markets. J. Int. Counc. Electr. Eng. 4(2), 114–120 (2014)

10. Voronin, S., Partanen, J.: Forecasting electricity price and demand using a hybrid

approach based on wavelet transform, ARIMA and neural networks. Int. J. Energy

Res. 38(5), 626–637 (2014)

11. Kou, P., Liang, D., Gao, L., Lou, J.: Probabilistic electricity price forecasting with

variational heteroscedastic Gaussian process and active learning. Energy Convers.

Manag. 89, 298–308 (2015)

12. Shrivastava, N.A., Panigrahi, B.K.: A hybrid wavelet-ELM based short term price

forecasting for electricity markets. Int. J. Electr. Power Energy Syst. 55, 41–50

(2014)

13. He, K., Xu, Y., Zou, Y., Tang, L.: Electricity price forecasts using a curvelet

denoising based approach. Phys. A: Stat. Mech. Appl. 425, 1–9 (2015)

14. Wan, C., Xu, Z., Wang, Y., Dong, Z.Y., Wong, K.P.: A hybrid approach for prob-

abilistic forecasting of electricity price. IEEE Trans. Smart Grid 5(1), 463–470

(2014)

15. Wan, C., Niu, M., Song, Y., Xu, Z.: Pareto optimal prediction intervals of electricity

price. IEEE Trans. Power Syst. 32(1), 817–819 (2017)

16. Liu, B., Nowotarski, J., Hong, T., Weron, R.: Probabilistic load forecasting via

quantile regression averaging on sister forecasts. IEEE Trans. Smart Grid 8(2), 1

(2017). https://doi.org/10.1109/tsg.2015.2437877

17. Wanga, P., Liu, B., Hongb, T.: Electric load forecasting with recency eﬀect: a big

data approach. Int. J. Forecast. 32, 585–597 (2016)

18. Chou, J.-S., Ngo, N.-T.: Smart grid data analytics framework for increasing energy

savings in residential buildings. Autom. Constr. 72, 247–257 (2016)

19. Ludwig, N., Feuerriegel, S., Neumann, D.: Putting big data analytics to work:

feature selection for forecasting electricity prices using the LASSO and random

forests. ISSN 1246-0125 (Print) 2116-7052

20. Lago, J., De Ridder, F., De Schutter, B.: Forecasting spot electricity prices: deep

learning approaches and empirical comparison of traditional algorithms. Appl.

Energy 221, 386–405 (2018)

21. Wang, L., Zhang, Z., Chen, J.: Short-term electricity price forecasting with stacked

denoising autoencoders. IEEE Trans. Power Syst. 32(4), 2673–2681 (2017)

22. Lagoa, J., De Ridder, F., Vrancx, P., De Schutter, B.: Forecasting day-ahead elec-

tricity prices in Europe: the importance of considering market integration. Appl.

Energy 211(1), 890–903 (2018)

23. Raviv, E., Bouwman, K.E., van Dijk, D.: Forecasting day-ahead electricity prices:

utilizing hourly prices. Energy Econ. 50, 227–239 (2015)

24. Xiao, L., Jianzhou Wang, R., Hou, J.W.: A combined model based on data pre-

analysis and weight coeﬃcients optimization for electrical load forecasting. Energy

82(15), 524–549 (2015)

25. Singh, S., Yassine, A.: Big data mining of energy time series for behavioral analytics

and energy consumption forecasting. Energies 11, 452 (2018)

26. Moon, J., Kim, K.-H., Kim, Y., Hwang, E.: A short-term electric load forecasting

scheme using 2-stage predictive analytics. In: 2018 IEEE International Conference

on Big Data and Smart Computing (2018)

STLF Using XGBoost 1131

27. Gonz´alez, J.P., San Roque, A.M., Perez, E.A.: Forecasting functional time series

with a new Hilbertian ARMAX model: application to electricity price forecasting.

IEEE Trans. Power Syst. 33(1), 545–556 (2018)

28. Luo, J., Hong, T., Fang, S.-C.: Benchmarking robustness of load forecasting models

under data integrity attacks. Int. J. Forecast. 34, 89–104 (2018)

29. Dong, G., Chen, Z.: Data driven energy management in a home microgrid based

on Bayesian optimal algorithm. IEEE Trans. Ind. Inform

30. Fausett, L.: Fundamentals of Neural Networks: Architectures, Algorithms, and

Applications. Pearson Education, Delhi (2006)

31. Shekhar, S., Amin, M.B.: Generalization by neural networks. IEEE Trans. Knowl.

Data Eng. 4, 177–185 (1992)