No full-text available
To read the full-text of this research,
you can request a copy directly from the authors.
Residential Short-Term Load Forecasting Using Convolutional Neural Networks
... To illustrate, consider the example in Fig. 10.12. Any estimates outside of the observed domain [1,10] have the same fixed constant values and are unlikely to be accurate. ...
... Note, that by adding the activation to the values prior in time before feeding it into the tanh activation to compute the next activation, this is related to residual skip connections described in Sect. 10 ...
... Note, that in the literature and in libraries, details in the implementation of what is referred to as TCN may vary. An architecture based on WaveNet has been used for residential load forecasting in [10]. In [11], TCNs are used for Fig. 10.31 ...
The traditional statistical and benchmark methods presented in Sect. 9.1 often assume some relatively simple relationship between the dependent and independent variables, be that linear trends, particular seasonalities or autoregressive behaviours. They have performed quite successfully for load forecasting, being quite accurate, even with low amounts of data, and can easily be interpreted by practitioners. However, the methods described in Sect. 9.1 may be less suitable for modelling more complex and highly nonlinear relationships. As data has become more ubiquitous due to increased monitoring, machine learning methods are becoming increasingly common as they can find complicated and subtle patterns in the data.
... Deep learning methods have also been applied to point forecasts at low aggregation levels, e.g., LSTM [12]- [15], deep belief networks (DBN) [16]- [18], and recently convolutional neural networks (CNN) [19], [20] which are easier to train than DBN as they are a supervised method and more efficient than LSTM on sequential data as they can be trained in parallel. Further, it has been shown that simple CNNs may actually be preferable for sequence modeling [21]. ...
... Related approaches typical use one-dimensional convolutions on the time series input (e.g., [19]) or time-seriesspecific convolutions such as causal convolutions (see [20]). Load data have an inherit daily and weekly seasonality and it has been shown that interaction with temperature is a relevant feature [25]. ...
... The two-and three-dimensional encoding make it interesting for using other more recent CNN architectures such as capsule networks [30]. But also timeseries specific architectures as in our previous work [20] may applicable for probabilistic load forecasting. The literature on lowly aggregated forecasts is still comparatively small and approaches hard to compare. ...
Lowly aggregated load profiles such as of individual households or buildings are more fluctuating and relative forecast errors are comparatively high. Therefore, the prevalent point forecasts are not sufficiently capable of optimally capturing uncertainty and hence lead to non-optimal decisions in different operational tasks. We propose an approach for short-term load quantile forecasting based on convolutional neural networks (STLQF-CNN). Historical load and temperature are encoded in a three-dimensional input to enforce locality of seasonal data. The model efficiently minimizes the pinball loss over all desired quantiles and the forecast horizon at once. We evaluate our approach for day-ahead and intra-day predictions on 222 households and different aggregations from the Pecan Street dataset. The evaluation shows that our model consistently outperforms a naive and an established linear quantile regression benchmark model, e.g., between 21 and 29 % better than the best benchmark on aggregations of 10, 20 and 50 households from Austin.
... This requires accurate forecasts on various aggregation levels, up to fine-grained low-voltage level load forecasts (Haben et al. 2021;Ordiano et al. 2018). Such fine-grained load forecasts can be used for demand-side management, energy management systems, distribution grid state estimation, grid management, storage optimization, peer-to-peer trading, peak shaving, smart electrical vehicle charging, dispatchable feeders, provision of feedback to customers, anomaly detection and intervention evaluation (Haben et al. 2021;Yildiz et al. 2017;Voß et al. 2018;Grabner et al. 2023;Werling et al. 2022). Moreover, the aggregation of fine-grained load forecasts can result in a more accurate forecast of the aggregated load (Hong et al. 2020). ...
... Global load forecasting models are already used with convolutional neural networks (Voß et al. 2018) (2022). In contrast to these works, our transfer learning approach is to train a generalized model on the data from many clients, without fine-tuning for a target time series. ...
In the smart grid of the future, accurate load forecasts on the level of individual clients can help to balance supply and demand locally and to prevent grid outages. While the number of monitored clients will increase with the ongoing smart meter rollout, the amount of data per client will always be limited. We evaluate whether a Transformer load forecasting model benefits from a transfer learning strategy, where a global univariate model is trained on the load time series from multiple clients. In experiments with two datasets containing load time series from several hundred clients, we find that the global training strategy is superior to the multivariate and local training strategies used in related work. On average, the global training strategy results in 21.8% and 12.8% lower forecasting errors than the two other strategies, measured across forecasting horizons from one day to one month into the future. A comparison to linear models, multi-layer perceptrons and LSTMs shows that Transformers are effective for load forecasting when they are trained with the global training strategy.
... There are an increasing number of studies on load forecasting [7,8]; however, most contributions are limited either to system-level loads or to individual household loads, which rely on smart meter data [4]. Recent studies on low-level aggregation load forecasting have focused on forecasting enhancement by (1) assessing and modifying traditional load forecasting techniques [9,10]; (2) building various models using machine learning, ensemble and hybrid methods, and probabilistic forecasting [11][12][13][14][15][16][17][18][19][20]; and (3) other methods such as data processing and clustering [4,9,21]. ...
... A framework based on a long short-term memory (LSTM) recurrent neural network (RNN) is proposed by Kong et al. [11] to address STLF problems for residential households, which shows the best forecasting performance compared with multiple benchmarks. Voß et al. [12] demonstrate that convolutional neural networks (CNN) called WaveNet outperforms the benchmarks for the aggregation of 10~200 households. ...
In the effort to achieve carbon neutrality through a decentralized electricity market, accurate short-term load forecasting at low aggregation levels has become increasingly crucial for various market participants' strategies. Accurate probabilistic forecasts at low aggregation levels can improve peer-to-peer energy sharing, demand response, and the operation of reliable distribution networks. However, these applications require not only probabilistic demand forecasts, which involve quantification of the forecast uncertainty, but also determining which consumers to include in the aggregation to meet electricity supply at the forecast lead time. While research papers have been proposed on the supply side, no similar research has been conducted on the demand side. This paper presents a method for creating a portfolio that optimally aggregates demand for a given energy demand, minimizing forecast inaccuracy of overall low-level aggregation. Using probabilistic load forecasts produced by either ARMA-GARCH models or kernel density estimation (KDE), we propose three approaches to creating a portfolio of residential households' demand: Forecast Validated, Seasonal Residual, and Seasonal Similarity. An evaluation of probabilistic load forecasts demonstrates that all three approaches enhance the accuracy of forecasts produced by random portfolios, with the Seasonal Residual approach for Korea and Ireland outperforming the others in terms of both accuracy and computational efficiency.
... At the same time, they are simple to use and have light computational costs [2]. AI-based methods, in turn, are well suited to identifying non-linear patterns and work well with individual (i.e., residential level) and aggregated data (i.e., substation level) [5,52]. ...
... The studies presented on FL-based STLF use a range of different NN architectures ( Table 1). Overall, the architectures have become deeper (i.e., multi-layered) over time as depth is typically associated with more accurate results [52]. In terms of layer design, we found Fully Connected layers (FCL), Long Short-term Memory (LSTM) Layers [53] and Convolutional Neural Networks (CNN). ...
With high levels of intermittent power generation and dynamic demand patterns, accurate forecasts for residential loads have become essential. Smart meters can play an important role when making these forecasts as they provide detailed load data. However, using smart meter data for load forecasting is challenging due to data privacy requirements. This paper investigates how these requirements can be addressed through a combination of federated learning and privacy preserving techniques such as differential privacy and secure aggregation. For our analysis, we employ a large set of residential load data and simulate how different federated learning models and privacy preserving techniques affect performance and privacy. Our simulations reveal that combining federated learning and privacy preserving techniques can secure both high forecasting accuracy and near-complete privacy. Specifically, we find that such combinations enable a high level of information sharing while ensuring privacy of both the processed load data and forecasting models. Moreover, we identify and discuss challenges of applying federated learning, differential privacy and secure aggregation for residential short-term load forecasting.
... al. (2020),Wang et al. (2018b),Marino et al. (2016),Divina et al. (2019),Yan et al. (2019),Zhou et al. (2020),Amber et al. (2018),Wang et al. (2018a),Liu et al. (2020),Fan et al. (2019a),Mocanu et al. (2016),Li et al. (2017b),Fan et al. (2019b),Vos et al. (2018),Williams and Gomez (2016),Fan et al. (2020),Pham et al. (2020),Fan et al. (2017),Huang et al. (2019),Taik and Cherkaoui (2020),Lee and Choi (2020),Nichiforov et al. (2018),Chen and Tan (2017) and Skomski et al. (2020) MAPE Wen et al. (2020), He (2017), Wang et al. (2018b), Yan et al. (2019), Li et al. (2015), Amber et al. (2018), Massana et al. (2015), Wang et al. (2018a), Cai et al. (2019), Wang et al. (2019), González-Vidal et al. (2017), Sehovac et al. (2019), Pham et al. (2020), Taik and Cherkaoui (2020), Gao et al. (2019), Kong et al. (2019), Nichiforov et al. (2018), Dong et al. (2016), Chen and Tan (2017) and Shan et al. (2019) CV-RMSE Zeng et al. (2019), Fan et al. (2019a,b), Wang et al. (2020), González-Vidal et al. (2017), Fan et al. (2020, 2017), Naug and Biswas (2018), Chae et al. (2016), Platon et al. (2015) and Dong et al. (2016) MAE Wen et al. (2020), He (2017), Divina et al. (2019), Amber et al. (2018), Liu et al. (2020), Fan et al. (2019a), Li et al. (2017b), Fan et al. (2019b), Sehovac et al. (2019), Pham et al. (2020), Fan et al. (2017), Jihad and Tahiri (2018) and Huang et al. (2019) R-Squared Ma et al. (2019), Wang et al. (2018b), Mehar et al. (2018), Wang et al. (2018a), Deb et al. (2016) and Rahman et al. (2019) MSE Ma et al. (2019), Liu et al. (2015), Jaber et al. (2019), Sendra-Arranz and Gutiérrez (2020), Xypolytou et al. (2017), Güngör et al. (2019), Kim and Cho (2019), Nichiforov et al. (2018) and Shao et al. (2020) NRMSE Mlangeni et al. (2020), Amber et al. (2018), Magoulès et al. (2017), Vos et al. (2018) and Sendra-Arranz and Gutiérrez (2020) Adjusted R-Squared Zeng et al. (2019) and Mehar et al. (2018) MAD Zhou et al. (2020) MRE Li et al. (2017b) ...
... Domestic Wen et al. (2020), Yan et al. (2019), Mocanu et al. (2016), Vos et al. (2018), Jihad and Tahiri (2018), Huang et al. (2019), Rahman et al. (2018), Kim and Cho (2019) ...
The building sector accounts for 36 % of the total global energy usage and 40% of associated Carbon Dioxide emissions. Therefore, the forecasting of building energy consumption plays a key role for different building energy management applications (e.g., demand-side management and promoting energy efficiency measures), and implementing intelligent control strategies. Thanks to the advancement of Internet of Things in the last few years, this has led to an increase in the amount of buildings energy related-data. The accessibility of this data has inspired the interest of researchers to utilize different data-driven approaches to forecast building energy consumption. In this study, we first present state of-the-art Machine Learning, Deep Learning and Statistical Analysis models that have been used in the area of forecasting building energy consumption. In addition, we also introduce a comprehensive review of the existing research publications that have been published since 2015. The reviewed literature has been categorized according to the following scopes: (I) building type and location; (II) data components; (III) temporal granularity; (IV) data pre-processing methods; (V) features selection and extraction techniques; (VI) type of approaches; (VII) models used; and (VIII) key performance indicators. Finally, gaps and current challenges with respect to data-driven building energy consumption forecasting have been highlighted, and promising future research directions are also recommended.
... The elephant in the room is the undeniable impact of hyperparameter optimisation on forecast accuracy and model performance. For example, [25] introduced an innovative approach, leveraging a convolutional neural network (CNN) to perform short-term probabilistic load forecasting, especially targeting lower aggregation tiers. This avant-garde model was capable of assimilating variable input data to produce probabilistic forecasts, a potential game changer in power grid decision-making matrices. ...
Increasing the renewable energy penetration, especially photovoltaic systems, requires accurate and short-term load forecasting for every individual electricity customer. This can significantly help distributed network service providers (DNSP) plan and operate electrical networks and provide better quality and more reliable electricity services to their customers. Due to the strong volatility of the load data, an optimised deep-ensemble learning methodology is proposed to develop several deep learning models to forecast the load data in individual and aggregate load scenarios. The adaptive wind-driven optimisation (AWDO) algorithm is used to tune the hyperparameters of four of the developed deep learning models, significantly improving their performance. Tuning the deep learning models' hyperparameters shows significant improvements in the accuracy, as well as training, and testing times. In addition, a hybrid ensemble strategy that contains bagging, random Subspace, and boosting (BRSB) with ensemble pruning is adapted in the optimised deep learning-based models to extract deep features from multivariate data. In this context, the bidirectional long-short-term memory optimised by the AWDO algorithm (Bi-LSTM-AWDO) based hybrid ensemble learning forecasting model is compared exhaustively with several benchmark models, including the recently developed models in the state-of-the-art. The average root mean square error (RMSE), and the average mean absolute percentage error (MAPE) of the Bi-LSTM-AWDO model for individual loads are 0.121 kW, and 7.55%, respectively, while they are 0.025 kW, and 1.51% for aggregate loads, respectively. Accordingly, the Bi-LSTM-AWDO model outperforms other models in forecasting short-term load data for individual and aggregate households using actual smart meters' measurements.
... Global load forecasting models are already used with other deep learning models than Transformers, such as convolutional neural networks [15] and N-BEATS [16]. A mixture between a multivariate and a global model is investigated in [17], where a single recurrent neural network (RNN) model is trained on randomly pooled subsets of the time series. ...
Recent work uses Transformers for load forecasting, which are the state of the art for sequence modeling tasks in data-rich domains. In the smart grid of the future, accurate load forecasts must be provided on the level of individual clients of an energy supplier. While the total amount of electrical load data available to an energy supplier will increase with the ongoing smart meter rollout, the amount of data per client will always be limited. We test whether the Transformer benefits from a transfer learning strategy, where a global model is trained on the load time series data from multiple clients. We find that the global model is superior to two other training strategies commonly used in related work: multivariate models and local models. A comparison to linear models and multi-layer perceptrons shows that Transformers are effective for electrical load forecasting when they are trained with the right strategy.
... They examined that "Factored Conditional Restricted Boltzmann Machine (FCRBM)" performed well on energy consumption dataset with better energy consumption prediction accuracy and outperformed the other ML models for instance ANN, SVM, RNN. [30] developed CNN-based WaveNet technique for short term forecasting and handling noisy data of energy appliances. They concluded that WaveNet performed well on short term prediction of electric loads for households. ...
The prediction of energy consumption plays a significant role in energy conservation and reducing the cost of power generation, to improve energy sustainability and economic stability. Current studies show an increased interest in the application of Machine Learning algorithms to forecast energy utilisation in smart homes. The performance of these Machine Learning algorithms is evaluated using accuracy algorithms. The process of manually selecting best-performing Machine Learning algorithms is still very challenging for data analysts and decision makers because the algorithms might not work well in a different use case or data-set. To address this, we propose a decision algorithm model using machine learning based data mining and picture fuzzy operators. First, Machine Learning algorithms are trained and tested to predict energy consumption of smart home appliances with respect to the weather information. Second, score values of Lasso Regression are used to understand the patterns and features of weather information for smart house micro-climate. We then propose a decision matrix using fuzzy operators to aggregate Machine Learning algorithms, prior to ranking using a score function. Finally, the electricity consumption of appliances as well as total energy consumed in the smart home is provided in Kilowatts (KW).
... The early machine learning (ML) methods for load forecasting were based on Support Vector Machines (SVM) [Chen et al., 2004] and basic (shallow) Neural Networks (NNs) [Tsekouras et al., 2006;Sözen et al., 2007;Geem and Roper, 2009;Ardakani and Ardehali, 2014]. Later on, more sophisticated techniques such as Deep NNs (DNNs) [Din and Marnerides, 2017], Convolutional NNs (CNNs) [Voß et al., 2018], and Long Short-Term Memory (LSTM) [Marino et al., 2016;Kong et al., 2019] emerged. However, deep architectures are prone to overfitting, which hinders their forecasting accuracy. ...
Electricity load forecasting is crucial for effectively managing and optimizing power grids. Over the past few decades, various statistical and deep learning approaches have been used to develop load forecasting models. This paper presents an interpretable machine learning approach that identifies load dynamics using data-driven methods within an operator-theoretic framework. We represent the load data using the Koopman operator, which is inherent to the underlying dynamics. By computing the corresponding eigenfunctions, we decompose the load dynamics into coherent spatiotemporal patterns that are the most robust features of the dynamics. Each pattern evolves independently according to its single frequency, making its predictability based on linear dynamics. We emphasize that the load dynamics are constructed based on coherent spatiotemporal patterns that are intrinsic to the dynamics and are capable of encoding rich dynamical features at multiple time scales. These features are related to complex interactions over interconnected power grids and different exogenous effects. To implement the Koopman operator approach more efficiently, we cluster the load data using a modern kernel-based clustering approach and identify power stations with similar load patterns, particularly those with synchronized dynamics. We evaluate our approach using a large-scale dataset from a renewable electric power system within the continental European electricity system and show that the Koopman-based approach outperforms a deep learning (LSTM) architecture in terms of accuracy and computational efficiency. The code for this paper has been deposited in a GitHub repository, which can be accessed at the following address github.com/Shakeri-Lab/Power-Grids.
... Shi et al. [18], for example, proposed a pooling-based deep Recurrent Neural Network (RNN) model for household forecasting, where consumers were split into different groups randomly, and the electrical load of each group was forecasted separately. Voß et al. [19] compared two forecasting strategies, i.e., one local model for all consumers and one global model for each consumer, for individual consumer load forecasting and reported that using a single model for all consumers yielded superior performance. Wang et al. [20] proposed a transformer-based model for forecasting over different types of loads simultaneously and explored the impact of attention mechanisms for this task. ...
With the increasing numbers of smart meter installations, scalable and efficient load forecasting techniques are critically needed to ensure sustainable situation awareness within the distribution networks. Distribution networks include a large amount of different loads at various aggregation levels, such as individual consumers, low-voltage feeders, and transformer stations. It is impractical to develop individual (or so-called local) forecasting models for each load separately. Additionally, such local models also (i) (largely) ignore the strong dependencies between different loads that might be present due to their spatial proximity and the characteristics of the distribution network, (ii) require historical data for each load to be able to make forecasts, and (iii) are incapable of adjusting to changes in the load behavior without retraining. To address these issues, we propose a global modeling framework for load forecasting in distribution networks that, unlike its local competitors, relies on a single global model to generate forecasts for a large number of loads. The global nature of the framework, significantly reduces the computational burden typically required when training multiple local forecasting models, efficiently exploits the cross-series information shared among different loads, and facilitates forecasts even when historical data for a load is missing or the behavior of a load evolves over time. To further improve on the performance of the proposed framework, an unsupervised localization mechanism and optimal ensemble construction strategy are also proposed to localize/personalize the global forecasting model to different load characteristics. Our experimental results show that the proposed framework outperforms naive benchmarks by more than 25% (in terms of Mean Absolute Error) on real-world dataset while exhibiting highly desirable characteristics when compared to the local models that are predominantly used in the literature. All source code and data are made publicly available to enable reproducibility: https://github.com/mihagrabner/GlobalModelingFramework.
... For example, Satos et al. [52] day to a year. Generally, these papers use data-driven methods, such as Random Forests [53], 284 Convolution Neural Network [54][55][56], and Long Short-term Memory Neural Network [56,63]. ...
... External features such as weather data, other Transformer architectures [5,[7][8][9]14] and data augmentation [22] could improve the results. Another possible improvement is to make use of dependencies between buildings, e.g. with transfer learning [23], by clustering similar buildings [24], or by using the time series of other buildings as covariates. ...
Accurate electrical load forecasts of buildings are needed to optimize local energy storage and to make use of demand-side flexibility. We study the usage of Transformer neural networks for short-term electrical load forecasting of 296 buildings from a public dataset. Transformer neural networks trained on many buildings give the best forecasts on 115 buildings, and multi-layer perceptrons trained on a single building are better on 161 buildings. In addition, we evaluate the models on buildings that were not used for training, and find that Transformer neural networks generalize better than multi-layer perceptrons and our statistical baselines. This shows that the usage of Transformer neural networks for building load forecasting could reduce training resources due to the good generalization to unseen buildings, and they could be useful for cold-start scenarios.
... In case of buildings, this pre-training could be performed on public data for similar buildings (Hooshmand and Sharma 2019), data from buildings showing a high correlation with the target building (Ozer et al. 2021;Gomez-Rosero et al. 2021;Tian et al. 2019;Lin and Wu 2021), or information-rich buildings (Li et al. 2021). Instead of using separate data sets for TL, data from multiple buildings can also be combined for the pre-training and individual buildings can be used for the fine-tuning (Voß et al. 2018). Furthermore, to counteract negative transfer and improve learning performance, specific source selection algorithms (Moon et al. 2020;Zhang and Luo 2015) or time series decomposition (Xu and Meng 2020) can be applied. ...
Sustainable energy systems are characterised by an increased integration of renewable energy sources, which magnifies the fluctuations in energy supply. Methods to to cope with these magnified fluctuations, such as load shifting, typically require accurate short-term load forecasts. Although numerous machine learning models have been developed to improve short-term load forecasting (STLF), these models often require large amounts of training data. Unfortunately, such data is usually not available, for example, due to new users or privacy concerns. Therefore, obtaining accurate short-term load forecasts with little data is a major challenge. The present paper thus proposes the latent space-based forecast enhancer (LSFE), a method which combines transfer learning and data augmentation to enhance STLF when training data is limited. The LSFE first trains a generative model on source data similar to the target data before using the latent space data representation of the target data to generate seed noise. Finally, we use this seed noise to generate synthetic data, which we combine with real data to enhance STLF. We evaluate the LSFE on real-world electricity data by examining the influence of its components, analysing its influence on obtained forecasts, and comparing its performance to benchmark models. We show that the Latent Space-based Forecast Enhancer is generally capable of improving the forecast accuracy and thus helps to successfully meet the challenge of limited available training data.
... Established methods for scenario generation often utilize univariate, i.e., step-by-step prediction, approaches like classical autoregressive models (Sharma et al., 2013) or autoregressive neural networks (Vagropoulos et al., 2016;Voss et al., 2018). As opposed to univariate models, multivariate modeling techniques model a series of time steps in a single prediction step. ...
We present a specialized scenario generation method that utilizes forecast information to generate scenarios for the particular usage in day-ahead scheduling problems. In particular, we use normalizing flows to generate wind power generation scenarios by sampling from a conditional distribution that uses day-ahead wind speed forecasts to tailor the scenarios to the specific day. We apply the generated scenarios in a simple stochastic day-ahead bidding problem of a wind electricity producer and run a statistical analysis focusing on whether the scenarios yield profitable and reliable decisions. Compared to conditional scenarios generated from Gaussian copulas and Wasserstein-generative adversarial networks, the normalizing flow scenarios identify the daily trends more accurately and with a lower spread while maintaining a diverse variety. In the stochastic day-ahead bidding problem, the conditional scenarios from all methods lead to significantly more profitable and reliable results compared to an unconditional selection of historical scenarios. The obtained profits using the normalizing flow scenarios are consistently closest to the perfect foresight solution, in particular, for small sets of only five scenarios.
... Shi et al. [9], for example, proposed a pooling-based deep RNN model for household forecasting, where consumers were split into different groups randomly, and the electrical load of each group was forecasted separately. Voß et al. [10] compared two forecasting strategies, i.e., one arXiv:2204.00493v1 [cs. ...
Efficient load forecasting is needed to ensure better observability in the distribution networks, whereas such forecasting is made possible by an increasing number of smart meter installations. Because distribution networks include a large amount of different loads at various aggregation levels, such as individual consumers, transformer stations and feeders loads, it is impractical to develop individual (or so-called local) forecasting models for each load separately. Furthermore, such local models ignore the strong dependencies between different loads that might be present due to their spatial proximity and the characteristics of the distribution network. To address these issues, this paper proposes a global modeling approach based on deep learning for efficient forecasting of a large number of loads in distribution networks. In this way, the computational burden of training a large amount of local forecasting models can be largely reduced, and the cross-series information shared among different loads can be utilized. Additionally, an unsupervised localization mechanism and optimal ensemble construction strategy are also proposed to localize/personalize the forecasting model to different groups of loads and to improve the forecasting accuracy further. Comprehensive experiments are conducted on real-world smart meter data to demonstrate the superiority of the proposed approach compared to competing methods.
... Among the 77 papers reviewed, 57 used a parameter-based approach, representing the vast majority of the papers reviewed. Fig. 10 shows that parameter-based methods are mostly used in load prediction, with several architectures including: MLP ( [108] ) LSTM ( [109] ), GRU (gated recurrent units, [56] ), CNN (convolutional neural network, [58,110] ), CNN + LSTM ( [62] ), and LSTM-TLL ( [111] ). Other common applications are building dynamics and systems control, linked by the nonlinear nature of these topics, affected by stochastic variables or driven by physical laws that justify the wide adoption of neural networks. ...
Smart buildings play a crucial role toward decarbonizing society, as globally buildings emit about one-third of greenhouse gases. In the last few years, machine learning has achieved a notable momentum that, if properly harnessed, may unleash its potential for advanced analytics and control of smart buildings, enabling the technique to scale up for supporting the decarbonization of the building sector. In this perspective, transfer learning aims to improve the performance of a target learner exploiting knowledge in related environments. The present work provides a comprehensive overview of transfer learning applications in smart buildings, classifying and analyzing 77 papers according to their applications, algorithms, and adopted metrics. The study identified four main application areas of transfer learning: (1) building load prediction, (2) occupancy detection and activity recognition, (3) building dynamics modeling, and (4) energy systems control. Furthermore, the review highlighted the role of deep learning in transfer learning applications that has been used in more than half of the analyzed studies. The paper also discusses how to integrate transfer learning in a smart building’s ecosystem, identifying, for each application area, the research gaps and guidelines for future research directions.
... A 1D CNN is a CNN model that has a convolutional hidden layer operating over a 1D sequence. A typical 1D CNN model with input, one convolutional layer, one pooling layer and one fully connected layer is shown in Figure 2(a) [21][22][23]. Deep Learning libraries such as PyTorch, Keras, and TensorFlow can support well for a 1D, 2D, and 3D CNN [24]. In the paper, we used Keras, Python to implement a 1D CNN for time series prediction. ...
The Convolutional Neural Network (CNN) model is one of the most effective models for load forecasting with hyperparameters which can be used not only to determine the CNN structure and but also to train the CNN model. This paper proposes a framework for Grid Search hyperparameters of the CNN model. In a training process, the optimal models will specify conditions that satisfy requirement for minimum of accuracy scores of Root Mean Square Error (RMSE), Mean Absolute Percentage Error (MAPE) and Mean Absolute Error (MAE). In the testing process, these optimal models will be used to evaluate the results along with all other ones. The results indicated that the optimal models have accuracy scores near the minimum values. Load demand data of Queensland (Australia) and Ho Chi Minh City (Vietnam) were utilized to verify the accuracy and reliability of the Grid Search framework. © 2021. The Author(s). This is an open-access article distributed under the terms of the Creative Commons Attribution-NonCommercial-NoDerivatives License (CC BY-NC-ND 4.0, https://creativecommons.org/licenses/by-nc-nd/4.0/), which permits use, distribution, and reproduction in any medium, provided that the Article is properly cited, the use is non-commercial, and no modifications or adaptations are made.
... In addition to approaches based on recurrent models like LSTM and GRU, methods based on convolutional neural network (CNN) layers have been proposed. Voss et al.[215] apply a time series specic CNN, the WaveNET architecture based on causal and dilated convolutions at the household-level and show that it can outperform more straightforward approaches like MLR and ANN. Besides time series specic convolutions, two-and three-dimensional encodings of time series into images have also been applied. ...
The increased digitalisation and monitoring of the energy system opens up numerous opportunities % and solutions which can help to decarbonise the energy system. Applications on low voltage (LV), localised networks, such as community energy markets and smart storage will facilitate decarbonisation, but they will require advanced control and management. Reliable forecasting will be a necessary component of many of these systems to anticipate key features and uncertainties. Despite this urgent need, there has not yet been an extensive investigation into the current state-of-the-art of low voltage level forecasts, other than at the smart meter level. This paper aims to provide a comprehensive overview of the landscape, current approaches, core applications, challenges and recommendations. Another aim of this paper is to facilitate the continued improvement and advancement in this area. To this end, the paper also surveys some of the most relevant and promising trends. It establishes an open, community-driven list of the known LV level open datasets to encourage further research and development.
... Therefore depending on the hourly resolution, in case of SME, Residential and London datasets, this receptive field corresponds to 10 days. To attain a receptive field of 256, the CNN should have atleast 21 layers, which leads to a large number of parameters [27]. Each layer has 32 different kernels each with a width of 4. Two layers of dropout have been added in between the dilated convolutional layers. ...
Smart grids infrastructure is rapidly adopting the recent technology to optimize the power generation and energy saving. The load forecasting in smart grids has been one such technology integration and accurate load forecasting models has been a challenge. With the advent of advanced infrastructure, huge data is being generated at different time frequencies, that can be used to build accurate load forecasting models. Focusing on the state-of-the-art machine learning techniques, in this work, we propose a load forecasting model of stacked dilated convolutional layers. The dilations efficiently captures the local trend and seasonality from the time series for future predictions. Proposed model has been trained on multiple time series data with varying frequencies. Results show that the proposed model is an improvement to the existing state-of-the-art.
... The CNN was primarily designed for image recognition, while it has recently been employed for STLF [25]. In [26], a time-dependent CNN (TD-CNN) was developed to improve the forecasting accuracy of STLF based on only the historical electricity load data. ...
Precise and reliable forecasting of short-term electricity load is essential to the development of smart grids. Particularly, deep neural networks (DNNs) are widely utilized for the prediction of short-term electricity load due to their automatic feature extraction ability. However, these available stacked deep-learning models may lose some temporal features or spatial features of original input data. To capture more comprehensive information, in this article, we present an integration scheme based on empirical mode decomposition (EMD), similar day methods, and DNNs to perform short-term load forecasting. It is especially worth noting that the electricity price is also an important factor for load variation, which is considered in our proposed scheme. Specifically, there are two primary layers: a feature extraction layer and a forecasting layer. In the feature extraction layer, EMD is applied to decompose load time series into several components, which are arranged into the 2-D input matrix of the convolutional neural network (CNN). Both the output vectors of the CNN and the raw load sequences are fed into the long short-term memory (LSTM) layer. Therefore, the whole EMD based CNN-LSTM approach extracts multimodal spatial-temporal features from input data. Meanwhile, the electricity price data is utilized to obtain multimodal spatial-temporal features in the same way. Additionally, the day and hour information and loads of similar days are to augment extra features for prediction. In the forecasting layer, the forecasting task is accomplished through a fully-connected neural network based on the outputs of the feature extraction layer. Leveraging these techniques enables our proposed scheme to extract more latent features, which significantly improve the accuracy. In order to demonstrate the performance of our proposed scheme, related experiments are conducted on actual data from the electricity market in Singapore. Compared to other available models, our proposed scheme is superior in graphic and numerical results.
... Nikolay Laptev, Jiafan Yu, and Ram Rajagopal, 2018 have shown, that network-based TL with DNNs is suitable for time series forecasting and that TL models are capable of outperforming regularly trained DNNs in multiple domains. These results coincide with other case studies related to forecasting in which a network-based TL approach with DNNs was used (Hu, R. Zhang, and Zhou, 2016;Liang et al., 2018;Marcus Vos, Christian Bender-Saebelkampf, and Sahin Albayrak, 2018;Qureshi et al., 2017). Previous work is also focussing to utilize TL as a means for information exchange in business networks to overcome data confidentiality challenges (Hirt and Kühl, 2018). ...
Data-driven methods -- such as machine learning and time series forecasting -- are widely used for sales forecasting in the food retail domain. However, for newly introduced products insufficient training data is available to train accurate models. In this case, human expert systems are implemented to improve prediction performance. Human experts rely on their implicit and explicit domain knowledge and transfer knowledge about historical sales of similar products to forecast new product sales. By applying the concept of Transfer Learning, we propose an analytical approach to transfer knowledge between listed stock products and new products. A network-based Transfer Learning approach for deep neural networks is designed to investigate the efficiency of Transfer Learning in the domain of food sales forecasting. Furthermore, we examine how knowledge can be shared across different products and how to identify the products most suitable for transfer. To test the proposed approach, we conduct a comprehensive case study for a newly introduced product, based on data of an Austrian food retailing company. The experimental results show, that the prediction accuracy of deep neural networks for food sales forecasting can be effectively increased using the proposed approach.
... 5 Based on these customers' load data, a lot of load forecasting models are developed for individual customers. [6][7][8][9][10][11][12][13] Inspired by the success of these methods, similar forecasting models are developed to predict the customer's response behavior. 14 For example, the work in Paoletti et al 15 tries to first subtract the baseline load from the actual load and get residual load. ...
Demand response, in which customers can actively reduce their energy consumption in respond to the price signal or incentives, is an effective technology to mitigate the temporary electricity shortage in modern power system. However, incentive‐based demand response event rarely happened. Thus, the customers' historical response to the peak event is seldom recorded, which leads to the difficulty in training the demand response behavior forecasting model. To solve the insufficient training data problem, a transfer learning–based model is proposed to forecast the customers' response to incentives, where the forecasting accuracy of a certain customer (target customer) is improved by introducing the load data of related customers (source customers). The whole forecasting model can be divided into two parts. (a) In the first part, the two‐layer load forecasting model is built to characterize the relations between the target customer and source customers. At the same time, variant weights are assigned to data sets according to source customers and according to their correlation to the target customer. (b) In the second step, with the latent variables and weight known, the weight and the latent variables are directly assigned to the corresponding demand response behavior data set. Then, the demand response forecasting model is trained, and the forecasting result of target customer is generated. The case studies are developed based on the data from the Smart Grid Smart City Customer Research (SGSC). The result shows the proposed model can achieve higher accuracy in forecasting the customer's response behavior to incentives, when compared with other benchmark models.
... • Temporal convolutional network (TCN) is recently proposed to handle sequence and achieves the state-of-theart performance in many sequence modeling tasks [27]. It has been used to load forecasting on individual resident level [28]. In order to compare with our model, we also transform the model into a day-ahead load forecasting model. ...
Accurate day-ahead individual resident load forecasting is very important to various applications of smart grid. As a powerful machine learning technology, deep learning has shown great advantages in load forecasting task. However, deep learning is a computationally-hungry method, requires a plenty of training time and results in considerable energy consumed and a plenty of CO2 emitted. This aggravates the energy crisis and incurs a substantial cost to the environment. As a result, the deep learning methods are difficult to be popularized and applied in the real smart grid environment. In this paper, to reduce training time, energy consumed and CO2 emitted, we propose a efficient green model based on convolutional neural network, namely LoadCNN, for next-day load forecasting of individual resident. The training time, energy consumption, and CO2 emissions of LoadCNN are only approximately 1/70 of the corresponding indicators of other state-of-the-art models. Meanwhile, it achieves state-of-the-art performance in terms of prediction accuracy. LoadCNN is the first load forecasting model which simultaneously considers prediction accuracy, training time, energy efficiency and environment costs. It is a efficient green model that is able to be quickly, cost-effectively and environmental-friendly deployed in a realistic smart grid environment.
Accurate prediction of gas field production is an important task for reservoir engineers, which is challenging due to many unknown reservoir parameters. Aiming to have a low-cost, intelligent, and robust method to predict gas and water production for a given gas reservoir, this paper proposes a CNN-LSTM model to predict gas field production based on a gas field in southwest China. The convolutional neural network (CNN) has a feature extraction ability, and the long short-term memory network (LSTM) can learn sequence dependence. By the combination of the two abilities, the CNN-LSTM model can describe the changing trend of gas field production. A new prediction strategy named partly unknown recursive prediction strategy (PURPS) is proposed that some input features are estimated using the predicted gas and water production according to known equations. The results show that the CNN-LSTM model can effectively predict gas field production. A detailed performance comparison was conducted between CNN-LSTM and other models. The comparison shows that the proposed CNN-LSTM model outperforms the existing methods. The monthly gas production average MAPE errors of the three different stages are CNN-LSTM (7.7%), RNN (18%), Random Forest (23.17%), ARIMA (25.3%), DNN (28.3%), Support Vector Machine (28.3%), CNN (41%), and LSTM (46%).
More and more factories and commercial buildings are installing combined heat and power (CHP) systems that include various energy storage devices. To reduce the energy cost of CHPs, optimal operation plans to satisfy time‐varying energy demands with minimum energy cost are required. Conventional operation planning methods using optimized calculation have an issue with long computing time. To address this, we adapted a convolutional neural network (CNN) to learn the time‐series features of input–output pairs of optimized calculations. The architecture of the proposed CNN has the following key features: (i) the application of one‐dimensional convolutional filters to extract the time‐series features of input data and (ii) the elimination of pooling layers that compress the input data. We also designed rule‐based flows to remove constraint violations from the generated operation plans. Our evaluations showed that the proposed method could generate operation plans with the average error of 3.3%, which is much lower than the 8.0% average error when using the conventional CNN with pooling calculation. © 2021 Institute of Electrical Engineers of Japan. Published by Wiley Periodicals LLC.
Short-term load forecasting is integral to the energy planning sector. Various techniques have been employed to achieve effective operation of power systems and efficient market management. We present a scalable system for day-ahead household electrical energy consumption forecasting, named HousEEC. The proposed forecasting method is based on a deep residual neural network, and integrates multiple sources of information by extracting features from (i) contextual data (weather, calendar), and (ii) the historical load of the particular household and all households present in the dataset. Additionally, we compute novel domain-specific time-series features that allow the system to better model the pattern of energy consumption of the household. The experimental analysis and evaluation were performed on one of the most extensive datasets for household electrical energy consumption, Pecan Street, containing almost four years of data. Multiple test cases show that the proposed model provides accurate load forecasting results, achieving a root-mean-square error score of 0.44 kWh and mean absolute error score of 0.23 kWh, for short-term load forecasting for 300 households. The analysis showed that, for hourly forecasting, our model had 8% error (22 kWh), which is 4 percentage points better than the benchmark model. The daily analysis showed that our model had 2% error (131 kWh), which is significantly less compared to the benchmark model, with 6% error (360 kWh).
Energy load forecasting plays an important role in the smart grid, which can affect the promoting energy production and consumption decision‐making processes. In this paper, the state‐of‐the‐art deep learning (DL) neural models are used in the short‐term load forecasting, including the multilayer perceptron (MLP), the convolutional neural network (CNN), and the long short‐term memory (LSTM). A novel loss function is proposed for the load forecasting, and two commonly used benchmarks are used to verify the validity of the proposed function. The simulation results show that the mean absolute percentage error (MAPE) of the proposed loss function is 19.63% lower than cross‐entropy and 2.34% lower than mean absolute error (MAE). We compared the mentioned neural networks in different aspects, and the results show that in energy load forecasting, CNN has superior performance than MLP and LSTM in terms of high accuracy and robustness to weather changes.
This paper presents a recurrent neural network model to make medium-to-long term predictions, i.e. time horizon of ⩾1 week, of electricity consumption profiles in commercial and residential buildings at one-hour resolution. Residential and commercial buildings are responsible for a significant fraction of the overall energy consumption in the U.S. With advances in sensors and smart technologies, there is a need for medium to long-term prediction of electricity consumption in residential and commercial buildings at hourly intervals to support decision making pertaining to operations, demand response strategies, and installation of distributed generation systems. The modeler may have limited access to information about building's schedules and equipment, making data-driven machine learning models attractive. The energy consumption data that is available may also contain blocks of missing data, making time-series predictions difficult. Thus, the main objectives of this paper are: (a) Develop and optimize novel deep recurrent neural network (RNN) models aimed at medium to long term electric load prediction at one-hour resolution; (b) Analyze the relative performance of the model for different types of electricity consumption patterns; and (c) Use the deep NN to perform imputation on an electricity consumption dataset containing segments of missing values. The proposed models were used to predict hourly electricity consumption for the Public Safety Building in Salt Lake City, Utah, and for aggregated hourly electricity consumption in residential buildings in Austin, Texas. For predicting the commercial building's load profiles, the proposed RNN sequence-to-sequence models generally correspond to lower relative error when compared with the conventional multi-layered perceptron neural network. For predicting aggregate electricity consumption in residential buildings, the proposed model generally does not provide gains in accuracy compared to the multi-layered perceptron model.
The scheduling and operation of power system becomes prominently complex and uncertain, especially with the penetration of distributed power. Load forecasting matters to the effective operation of power system. This paper proposes a novel deep learning framework to forecast the short-term grid load. First, the load data is processed by Box-Cox transformation, and two parameters (electricity price and temperature) are investigated. Then, to quantify the tail-dependence of power load on the two parameters, parametric Copula models are fitted and the threshold of peak load are computed. Next, a deep belief network is built to forecast the hourly load of the power grid. One year grid load data collected from an urbanized area in Texas, United States is utilized in the case studies. Short-term load forecasting are examined in four seasons independently. Day-ahead and week-ahead load forecasting experiments are conducted in each season using the proposed framework. The proposed framework is compared with classical neural networks, support vector regression machine, extreme learning machine, and classical deep belief networks. The load forecasting performances are assessed by mean absolute percentage error, root mean square error, and hit rate. Computational results confirm the effectiveness of the proposed data-driven deep learning framework. The prediction accuracies of both day-ahead forecasting and week-ahead forecasting demonstrate that the proposed framework outperforms the tested algorithms.
As the power system is facing a transition towards a more intelligent, flexible and interactive system with higher penetration of renewable energy generation, load forecasting, especially short-term load forecasting for individual electric customers plays an increasingly essential role in the future grid planning and operation. Other than aggregated residential load in a large scale, forecasting an electric load of a single energy user is fairly challenging due to the high volatility and uncertainty involved. In this paper, we propose a long short-term memory (LSTM) recurrent neural network (RNN) based framework, which is the latest and one of the most popular techniques of deep learning, to tackle this tricky issue. The proposed framework is tested on a publicly available set of real residential smart meter data, of which the performance is comprehensively compared to various benchmarks including the state-of-the-arts in the field of load forecasting. As a result, the proposed LSTM approach outperforms the other listed rival algorithms in the task of short-term load forecasting for individual residential households.
We present a method for conditional time series forecasting based on an adaptation of the recent deep convolutional WaveNet architecture. The proposed network contains stacks of dilated convolutions that allow it to access a broad range of history when forecasting, a ReLU activation function and conditioning is performed by applying multiple convolutional filters in parallel to separate time series which allows for the fast processing of data and the exploitation of the correlation structure between the multivariate time series. We test and analyze the performance of the convolutional network both unconditionally as well as conditionally for financial time series forecasting using the S&P500, the volatility index, the CBOE interest rate and several exchange rates and extensively compare it to the performance of the well-known autoregressive model and a long-short term memory network. We show that a convolutional network is well-suited for regression-type problems and is able to effectively learn dependencies in and between the series without the need for long historical time series, is a time-efficient and easy to implement alternative to recurrent-type networks and tends to outperform linear and recurrent models.
Short-Term Load Forecasting (STLF) is a fundamental component in the efficient management of power systems, which has been studied intensively over the past 50 years. The emerging development of smart grid technologies is posing new challenges as well as opportunities to STLF. Load data, collected at higher geographical granularity and frequency through thousands of smart meters, allows us to build a more accurate local load forecasting model, which is essential for local optimization of power load through demand side management. With this paper, we show how several existing approaches for STLF are not applicable on local load forecasting, either because of long training time, unstable optimization process, or sensitivity to hyper-parameters. Accordingly, we select five models suitable for local STFL, which can be trained on different time-series with limited intervention from the user. The experiment, which consists of 40 time-series collected at different locations and aggregation levels, revealed that yearly pattern and temperature information are only useful for high aggregation level STLF. On local STLF task, the modified version of double seasonal Holt-Winter proposed in this paper performs relatively well with only 3 months of training data, compared to more complex methods.
In the smart grid, one of the most important research areas is load forecasting; it spans from traditional time series analyses to recent machine learning approaches and mostly focuses on forecasting aggregated electricity consumption. However, the importance of demand side energy management, including individual load forecasting, is becoming critical. In this paper, we propose deep neural network (DNN)-based load forecasting models and apply them to a demand side empirical load database. DNNs are trained in two different ways: a pre-training restricted Boltzmann machine and using the rectified linear unit without pre-training. DNN forecasting models are trained by individual customer's electricity consumption data and regional meteorological elements. To verify the performance of DNNs, forecasting results are compared with a shallow neural network (SNN), a double seasonal Holt-Winters (DSHW) model and the autoregressive integrated moving average (ARIMA). The mean absolute percentage error (MAPE) and relative root mean square error (RRMSE) are used for verification. Our results show that DNNs exhibit accurate and robust predictions compared to other forecasting models, e.g., MAPE and RRMSE are reduced by up to 17% and 22% compared to SNN and 9% and 29% compared to DSHW.
Ensuring sustainability demands more efficient energy management with minimized energy wastage. Therefore, the power grid of the future should provide an unprecedented level of flexibility in energy management. To that end, intelligent decision making requires accurate predictions of future energy demand/load, both at aggregate and individual site level. Thus, energy load forecasting have received increased attention in the recent past, however has proven to be a difficult problem. This paper presents a novel energy load forecasting methodology based on Deep Neural Networks, specifically Long Short Term Memory (LSTM) algorithms. The presented work investigates two variants of the LSTM: 1) standard LSTM and 2) LSTM-based Sequence to Sequence (S2S) architecture. Both methods were implemented on a benchmark data set of electricity consumption data from one residential customer. Both architectures where trained and tested on one hour and one-minute time-step resolution datasets. Experimental results showed that the standard LSTM failed at one-minute resolution data while performing well in one-hour resolution data. It was shown that S2S architecture performed well on both datasets. Further, it was shown that the presented methods produced comparable results with the other deep learning methods for energy forecasting in literature.
This paper introduces WaveNet, a deep neural network for generating raw audio waveforms. The model is fully probabilistic and autoregressive, with the predictive distribution for each audio sample conditioned on all previous ones; nonetheless we show that it can be efficiently trained on data with tens of thousands of samples per second of audio. When applied to text-to-speech, it yields state-of-the-art performance, with human listeners rating it as significantly more natural sounding than the best parametric and concatenative systems for both English and Mandarin. A single WaveNet can capture the characteristics of many different speakers with equal fidelity, and can switch between them by conditioning on the speaker identity. When trained to model music, we find that it generates novel and often highly realistic musical fragments. We also show that it can be employed as a discriminative model, returning promising results for phoneme recognition.
The electrical demand forecasting problem can be regarded as a non-linear time series prediction problem depending on many complex factors since it is required at various aggregation levels and at high resolution. To solve this challenging problem, various time series and machine learning approaches has been proposed in the literature. As an evolution of neural network-based prediction methods, deep learning techniques are expected to increase the prediction accuracy by being stochastic and allowing bi-directional connections between neurons. In this paper, we investigate a newly developed deep learning model for time series prediction, namely Factored Conditional Restricted Boltzmann Machine (FCRBM), and extend it for demand forecasting. The assessment is made on the EcoGrid EU dataset, consisting of aggregated electric power consumption, price and meteorological data collected from 1900 customers. The households are equipped with local generation and smart appliances capable of responding to real-time pricing signals. The results show that for the energy prediction problem solved here, FCRBM outperforms the benchmark machine learning approach, i.e. Support Vector Machine.
State-of-the-art models for semantic segmentation are based on adaptations of convolutional networks that had originally been designed for image classification. However, dense prediction problems such as semantic segmentation are structurally different from image classification. In this work, we develop a new convolutional network module that is specifically designed for dense prediction. The presented module uses dilated convolutions to systematically aggregate multi-scale contextual information without losing resolution. The architecture is based on the fact that dilated convolutions support exponential expansion of the receptive field without loss of resolution or coverage. We show that the presented context module increases the accuracy of state-of-the-art semantic segmentation systems. In addition, we examine the adaptation of image classification networks to dense prediction and show that simplifying the adapted network can increase accuracy.
The increasing use of renewable energy sources with variable output, such as
solar photovoltaic and wind power generation, calls for Smart Grids that
effectively manage flexible loads and energy storage. The ability to forecast
consumption at different locations in distribution systems will be a key
capability of Smart Grids. The goal of this paper is to benchmark
state-of-the-art methods for forecasting electricity demand on the household
level across different granularities and time scales in an explorative way,
thereby revealing potential shortcomings and find promising directions for
future research in this area. We apply a number of forecasting methods
including ARIMA, neural networks, and exponential smoothening using several
strategies for training data selection, in particular day type and sliding
window based strategies. We consider forecasting horizons ranging between 15
minutes and 24 hours. Our evaluation is based on two data sets containing the
power usage of individual appliances at second time granularity collected over
the course of several months. The results indicate that forecasting accuracy
varies significantly depending on the choice of forecasting methods/strategy
and the parameter configuration. Measured by the Mean Absolute Percentage Error
(MAPE), the considered state-of-the-art forecasting methods rarely beat
corresponding persistence forecasts. Overall, we observed MAPEs in the range
between 5 and >100%. The average MAPE for the first data set was ~30%, while it
was ~85% for the other data set. These results show big room for improvement.
Based on the identified trends and experiences from our experiments, we
contribute a detailed discussion of promising future research.
As low carbon technologies become more pervasive, distribution network operators are looking to support the expected changes in the demands on the low voltage networks through the smarter control of storage devices. Accurate forecasts of demand at the individual household-level, or of small aggregations of households, can improve the peak demand reduction brought about through such devices by helping to plan the most appropriate charging and discharging cycles. However, before such methods can be developed, validation measures which can assess the accuracy and usefulness of forecasts of the volatile and noisy household-level demand are required. In this paper we introduce a new forecast verification error measure that reduces the so-called “double penalty” effect, incurred by forecasts whose features are displaced in space or time, compared to traditional point-wise metrics, such as the Mean Absolute Error, and pp-norms in general. The measure that we propose is based on finding a restricted permutation of the original forecast that minimises the point-wise error, according to a given metric. We illustrate the advantages of our error measure using half-hourly domestic household electrical energy usage data recorded by smart meters, and discuss the effect of the permutation restriction.
INTRODUCTIONThe ability of multilayer back-propagation networks to learn complex, high-dimensional, nonlinearmappings from large collections of examples makes them obvious candidates for imagerecognition or speech recognition tasks (see PATTERN RECOGNITION AND NEURALNETWORKS). In the traditional model of pattern recognition, a hand-designed featureextractor gathers relevant information from the input and eliminates irrelevant variabilities.A trainable classifier then categorizes the...
Scikit-learn is a Python module integrating a wide range of state-of-the-art
machine learning algorithms for medium-scale supervised and unsupervised
problems. This package focuses on bringing machine learning to non-specialists
using a general-purpose high-level language. Emphasis is put on ease of use,
performance, documentation, and API consistency. It has minimal dependencies
and is distributed under the simplified BSD license, encouraging its use in
both academic and commercial settings. Source code, binaries, and documentation
can be downloaded from http://scikit-learn.sourceforge.net.
This paper proposes a simple empirical scaling law that describes load forecasting accuracy at varying levels of aggregation. We show that for many forecasting methods, aggregating more customers improves the relative forecasting performance up to specific point. Beyond this point, no more improvement in relative performance can be obtained. A benchmarking procedure for applying the scaling law to different forecasting models is presented. The aggregation model is evaluated with year long consumption profiles of over 180 thousand Pacific Gas & Electric customers. A theoretical model based on a bias variance decomposition of the forecast error is used to model the Aggregation Error Curves (AECs) that are empirically explored.
Load forecasting is an important electric utility task for planning resources in Smart grid. This function also aids in predicting the behavior of energy systems in reducing dynamic uncertainties. The efficiency of the entire grid operation depends on accurate load forecasting. This paper proposes and investigates the application of a multi-layered deep neural network to the Iberian electric market (MIBEL) forecasting task. Ninety days of energy demand data are used to train the proposed model. The ninety-day period is treated as a historical dataset to train and predict the demand for day-ahead markets. The network structure is implemented using Google's machine learning Tensor-flow platform. Various combinations of activation functions were tested to achieve a better Mean Absolute percentage error (MAPE) considering the weekday and weekend variations. The tested functions include Sigmoid, Rectifier linear unit (ReLU), and Exponential linear unit (ELU). The preliminary results are promising. and show significant savings in the MAPE values using the ELU function over the other activation functions.
The key challenge for household load forecasting lies in the high volatility and uncertainty of load profiles. Traditional methods tend to avoid such uncertainty by load aggregation (to offset uncertainties), customer classification (to cluster uncertainties) and spectral analysis (to filter out uncertainties). This paper, for the first time, aims to directly learn the uncertainty by applying a new breed of machine learning algorithms – deep learning. However simply adding layers in neural networks will cap the forecasting performance due to the occurrence of overfitting. A novel pooling-based deep recurrent neural network (PDRNN) is proposed in this paper which batches a group of customers’ load profiles into a pool of inputs. Essentially the model could address the over-fitting issue by increasing data diversity and volume. This work reports the first attempts to develop a bespoke deep learning application for household load forecasting and achieved preliminary success. The developed method is implemented on Tensorflow deep learning platform and tested on 920 smart metered customers from Ireland. Compared with the state-of-art techniques in household load forecasting, the proposed method outperforms ARIMA by 19.5%, SVR by 13.1% and classical deep RNN by 6.5% in terms of RMSE.
Energy consumption forecasting for buildings has immense value in energy efficiency and sustainability research. Accurate energy forecasting models have numerous implications in planning and energy optimization of buildings and campuses. For new buildings, where past recorded data is unavailable, computer simulation methods are used for energy analysis and forecasting future scenarios. However, for existing buildings with historically recorded time series energy data, statistical and machine learning techniques have proved to be more accurate and quick. This study presents a comprehensive review of the existing machine learning techniques for forecasting time series energy consumption. Although the emphasis is given to a single time series data analysis, the review is not just limited to it since energy data is often co-analyzed with other time series variables like outdoor weather and indoor environmental conditions. The nine most popular forecasting techniques that are based on the machine learning platform are analyzed. An in-depth review and analysis of the ‘hybrid model’, that combines two or more forecasting techniques is also presented. The various combinations of the hybrid model are found to be the most effective in time series energy forecasting for building.
Electric load profile forecasting is among the most important tasks in power system operation. This task has been performed reasonably well with minimal disruption from renewables over the past decades. The expected high penetration of renewables in the system is challenging this classical task. Moreover, the blossom of demand side management programs warrants the necessity of load profile forecasting even for individual (residential as well as industrial) consumers. In this paper, we seek to define predictability in a rigorous way from an information theoretic perspective, which shed light on comparing the predictability between different kinds of loads. Then, we identify two separable components of predictability - constancy and contingency. We further exploit the key parameters in deciding the predictability, which challenge the commonly-held beliefs. For example, does a finer granularity imply a lower predictability? To what degree does the law of large numbers help improve the predictability?
A number of recent studies use deep belief networks (DBN) with a great success in various applications such as image classification and speech recognition. In this paper, a DBN made up from multiple layers of restricted Boltzmann machines is used for electricity load forecasting. The layer-by-layer unsupervised training procedure is followed by fine-tuning of the parameters by using a supervised back-propagation training method. Our DBN model was applied to short-term electricity load forecasting based on the Macedonian hourly electricity consumption data in the period 2008–2014. The obtained results are not only compared with the latest actual data, but furthermore, they are compared with the predicted data obtained from a typical feed-forward multi-layer perceptron neural network and with the forecasted data provided by the Macedonian system operator (MEPSO). The comparisons show that the applied model is not only suitable for hourly electricity load forecasting of the Macedonian electric power system, it actually provides superior results than the ones obtained using traditional methods. The mean absolute percentage error (MAPE) is reduced by up to 8.6% when using DBN, compared to the MEPSO data for the 24-h ahead forecasting, and the MAPE for daily peak forecasting is reduced by up to 21%.
The energy industry has been going through a significant modernization process over the last decade. Its infrastructure is being upgraded rapidly. The supply, demand and prices are becoming more volatile and less predictable than ever before. Even its business model is being challenged fundamentally. In this competitive and dynamic environment, many decision-making processes rely on probabilistic forecasts to quantify the uncertain future. Although most of the papers in the energy forecasting literature focus on point or single-valued forecasts, the research interest in probabilistic energy forecasting research has taken off rapidly in recent years. In this paper, we summarize the recent research progress on probabilistic energy forecasting. A major portion of the paper is devoted to introducing the Global Energy Forecasting Competition 2014 (GEFCom2014), a probabilistic energy forecasting competition with four tracks on load, price, wind and solar forecasting, which attracted 581 participants from 61 countries. We conclude the paper with 12 predictions for the next decade of energy forecasting.
Deep neural nets with a large number of parameters are very powerful machine learning systems. However, overfitting is a serious problem in such networks. Large networks are also slow to use, making it difficult to deal with overfitting by combining the predictions of many different large neural nets at test time. Dropout is a technique for addressing this problem. The key idea is to randomly drop units (along with their connections) from the neural network during training. This prevents units from co-adapting too much. During training, dropout samples from an exponential number of different "thinned" networks. At test time, it is easy to approximate the effect of averaging the predictions of all these thinned networks by simply using a single unthinned network that has smaller weights. This significantly reduces overfitting and gives major improvements over other regularization methods. We show that dropout improves the performance of neural networks on supervised learning tasks in vision, speech recognition, document classification and computational biology, obtaining state-of-the-art results on many benchmark data sets. © 2014 Nitish Srivastava, Geoffrey Hinton, Alex Krizhevsky, Ilya Sutskever and Ruslan Salakhutdinov.
State-of-the-art models for semantic segmentation are based on adaptations of
convolutional networks that had originally been designed for image
classification. However, dense prediction and image classification are
structurally different. In this work, we develop a new convolutional network
module that is specifically designed for dense prediction. The presented module
uses dilated convolutions to systematically aggregate multi-scale contextual
information without losing resolution. The architecture is based on the fact
that dilated convolutions support exponential expansion of the receptive field
without loss of resolution or coverage. We show that the presented context
module increases the accuracy of state-of-the-art semantic segmentation
systems. In addition, we examine the adaptation of image classification
networks to dense prediction and show that simplifying the adapted network can
increase accuracy.
Accurate Short Term Load Forecasting is an essential step towards load balancing methods in energy systems. With the recent introduction of Smart Meters for residential buildings, load forecasting and shifting methods can be implemented for individual households. The high variance of the load demand on the household level requires specific forecasting methods. This paper provides an overview of the methods which have been applied and points out what results are comparable. Therefore a structured literature review is carried out. In the process, 375 papers are analyzed and categorized via a concept matrix. Based on this review it is pointed out, which methods achieve good results for which purpose and which publicly available datasets can be used for evaluation.
Data analytics in smart grids can be leveraged to channel the data downpour from individual meters into knowledge valuable to electric power utilities and end-consumers. Short-term load forecasting (STLF) can address issues vital to a utility but it has traditionally been done mostly at system (city or country) level. In this case study, we exploit rich, multi-year, and high-frequency annotated data collected via a metering infrastructure to perform STLF on aggregates of power meters in a mid-sized city. For smart meter aggregates complemented with geo-specific weather data, we benchmark several state-of-the-art forecasting algorithms, including kernel methods for nonlinear regression, seasonal and temperature-adjusted auto-regressive models, exponential smoothing and state-space models. We show how STLF accuracy improves at larger meter aggregation (at feeder, substation, and system-wide level). We provide an overview of our algorithms for load prediction and discuss system performance issues that impact real time STLF. ® 2014 Alcatel-Lucent.
The recent development of smart meters has allowed the analysis of household electricity consumption in real time. Predicting electricity consumption at such very low scales should help to increase the efficiency of distribution networks and energy pricing. However, this is by no means a trivial task since household-level consumption is much more irregular than at the transmission or distribution levels. In this work, we address the problem of improving consumption forecasting by using the statistical relations between consumption series. This is done both at the household and district scales (hundreds of houses), using various machine learning techniques, such as support vector machine for regression (SVR) and multilayer perceptron (MLP). First, we determine which algorithm is best adapted to each scale, then, we try to find leaders among the time series, to help short-term forecasting. We also improve the forecasting for district consumption by clustering houses according to their consumption profiles.
Benchmarking issue in short term load forecasting has not received as much attention as it deserves. Although dozens of techniques have been reported to be applied to short term load forecasting, most of them are still on the theoretical level with insignificant practical value. None of them has been established to produce benchmarking models for comparative assessment. This paper proposes a naïve multiple linear regression benchmark for short term load forecasting, which is from the experience of helping a US utility develop the first in- house short term load forecasts. The proposed model has been served as a benchmark for this utility since 2009, and was in production use for a year with satisfying performance before a major upgrade. It has also been used for a Canadian utility for load forecasting purposes. In addition, it was reproduced by a group of graduate students from a creditable US university following the documented procedure.
This paper proposes a recurrent neural network based approach to short-term load forecasting in power systems. Recurrent neural networks in multilayer perceptrons have an advantage that the context layer is able to cope with historical data. As a result, it is expected that recurrent neural networks give better solutions than the conventional feedforward multilayer perceptrons in term of accuracy. Also, the differential equation form of the time series is utilized to deal with the nonstationarity of the daily load time series. Furthermore, this paper proposes the diffusion learning method for determining weights between units in a recurrent network. The method is capable of escaping from local minima with stochastic noise. A comparison is made between conventional multilayer perceptrons and the proposed method for actual data.
A naïve multiple linear regression benchmark for short term load forecasting
- T Hong
- P Wang
- H L Willis
T. Hong, P. Wang, and H. L. Willis, "A naïve multiple linear regression
benchmark for short term load forecasting," in Power and Energy Society
General Meeting, 2011 IEEE. IEEE, 2011, pp. 1-6.