Conference Paper

Residential Short-Term Load Forecasting Using Convolutional Neural Networks

Authors:
To read the full-text of this research, you can request a copy directly from the authors.

No full-text available

Request Full-text Paper PDF

To read the full-text of this research,
you can request a copy directly from the authors.

... Deep learning methods have also been applied to point forecasts at low aggregation levels, e.g., LSTM [12]- [15], deep belief networks (DBN) [16]- [18], and recently convolutional neural networks (CNN) [19], [20] which are easier to train than DBN as they are a supervised method and more efficient than LSTM on sequential data as they can be trained in parallel. Further, it has been shown that simple CNNs may actually be preferable for sequence modeling [21]. ...
... Related approaches typical use one-dimensional convolutions on the time series input (e.g., [19]) or time-seriesspecific convolutions such as causal convolutions (see [20]). Load data have an inherit daily and weekly seasonality and it has been shown that interaction with temperature is a relevant feature [25]. ...
... The two-and three-dimensional encoding make it interesting for using other more recent CNN architectures such as capsule networks [30]. But also timeseries specific architectures as in our previous work [20] may applicable for probabilistic load forecasting. The literature on lowly aggregated forecasts is still comparatively small and approaches hard to compare. ...
Conference Paper
Full-text available
Lowly aggregated load profiles such as of individual households or buildings are more fluctuating and relative forecast errors are comparatively high. Therefore, the prevalent point forecasts are not sufficiently capable of optimally capturing uncertainty and hence lead to non-optimal decisions in different operational tasks. We propose an approach for short-term load quantile forecasting based on convolutional neural networks (STLQF-CNN). Historical load and temperature are encoded in a three-dimensional input to enforce locality of seasonal data. The model efficiently minimizes the pinball loss over all desired quantiles and the forecast horizon at once. We evaluate our approach for day-ahead and intra-day predictions on 222 households and different aggregations from the Pecan Street dataset. The evaluation shows that our model consistently outperforms a naive and an established linear quantile regression benchmark model, e.g., between 21 and 29 % better than the best benchmark on aggregations of 10, 20 and 50 households from Austin.
... Therefore depending on the hourly resolution, in case of SME, Residential and London datasets, this receptive field corresponds to 10 days. To attain a receptive field of 256, the CNN should have atleast 21 layers, which leads to a large number of parameters [27]. Each layer has 32 different kernels each with a width of 4. Two layers of dropout have been added in between the dilated convolutional layers. ...
Article
Full-text available
Smart grids infrastructure is rapidly adopting the recent technology to optimize the power generation and energy saving. The load forecasting in smart grids has been one such technology integration and accurate load forecasting models has been a challenge. With the advent of advanced infrastructure, huge data is being generated at different time frequencies, that can be used to build accurate load forecasting models. Focusing on the state-of-the-art machine learning techniques, in this work, we propose a load forecasting model of stacked dilated convolutional layers. The dilations efficiently captures the local trend and seasonality from the time series for future predictions. Proposed model has been trained on multiple time series data with varying frequencies. Results show that the proposed model is an improvement to the existing state-of-the-art.
... • Temporal convolutional network (TCN) is recently proposed to handle sequence and achieves the state-of-theart performance in many sequence modeling tasks [27]. It has been used to load forecasting on individual resident level [28]. In order to compare with our model, we also transform the model into a day-ahead load forecasting model. ...
Preprint
Accurate day-ahead individual resident load forecasting is very important to various applications of smart grid. As a powerful machine learning technology, deep learning has shown great advantages in load forecasting task. However, deep learning is a computationally-hungry method, requires a plenty of training time and results in considerable energy consumed and a plenty of CO2 emitted. This aggravates the energy crisis and incurs a substantial cost to the environment. As a result, the deep learning methods are difficult to be popularized and applied in the real smart grid environment. In this paper, to reduce training time, energy consumed and CO2 emitted, we propose a efficient green model based on convolutional neural network, namely LoadCNN, for next-day load forecasting of individual resident. The training time, energy consumption, and CO2 emissions of LoadCNN are only approximately 1/70 of the corresponding indicators of other state-of-the-art models. Meanwhile, it achieves state-of-the-art performance in terms of prediction accuracy. LoadCNN is the first load forecasting model which simultaneously considers prediction accuracy, training time, energy efficiency and environment costs. It is a efficient green model that is able to be quickly, cost-effectively and environmental-friendly deployed in a realistic smart grid environment.
... Nikolay Laptev, Jiafan Yu, and Ram Rajagopal, 2018 have shown, that network-based TL with DNNs is suitable for time series forecasting and that TL models are capable of outperforming regularly trained DNNs in multiple domains. These results coincide with other case studies related to forecasting in which a network-based TL approach with DNNs was used (Hu, R. Zhang, and Zhou, 2016;Liang et al., 2018;Marcus Vos, Christian Bender-Saebelkampf, and Sahin Albayrak, 2018;Qureshi et al., 2017). Previous work is also focussing to utilize TL as a means for information exchange in business networks to overcome data confidentiality challenges (Hirt and Kühl, 2018). ...
Preprint
Full-text available
Data-driven methods -- such as machine learning and time series forecasting -- are widely used for sales forecasting in the food retail domain. However, for newly introduced products insufficient training data is available to train accurate models. In this case, human expert systems are implemented to improve prediction performance. Human experts rely on their implicit and explicit domain knowledge and transfer knowledge about historical sales of similar products to forecast new product sales. By applying the concept of Transfer Learning, we propose an analytical approach to transfer knowledge between listed stock products and new products. A network-based Transfer Learning approach for deep neural networks is designed to investigate the efficiency of Transfer Learning in the domain of food sales forecasting. Furthermore, we examine how knowledge can be shared across different products and how to identify the products most suitable for transfer. To test the proposed approach, we conduct a comprehensive case study for a newly introduced product, based on data of an Austrian food retailing company. The experimental results show, that the prediction accuracy of deep neural networks for food sales forecasting can be effectively increased using the proposed approach.
... Shi et al. [9], for example, proposed a pooling-based deep RNN model for household forecasting, where consumers were split into different groups randomly, and the electrical load of each group was forecasted separately. Voß et al. [10] compared two forecasting strategies, i.e., one arXiv:2204.00493v1 [cs. ...
Preprint
Full-text available
Efficient load forecasting is needed to ensure better observability in the distribution networks, whereas such forecasting is made possible by an increasing number of smart meter installations. Because distribution networks include a large amount of different loads at various aggregation levels, such as individual consumers, transformer stations and feeders loads, it is impractical to develop individual (or so-called local) forecasting models for each load separately. Furthermore, such local models ignore the strong dependencies between different loads that might be present due to their spatial proximity and the characteristics of the distribution network. To address these issues, this paper proposes a global modeling approach based on deep learning for efficient forecasting of a large number of loads in distribution networks. In this way, the computational burden of training a large amount of local forecasting models can be largely reduced, and the cross-series information shared among different loads can be utilized. Additionally, an unsupervised localization mechanism and optimal ensemble construction strategy are also proposed to localize/personalize the forecasting model to different groups of loads and to improve the forecasting accuracy further. Comprehensive experiments are conducted on real-world smart meter data to demonstrate the superiority of the proposed approach compared to competing methods.
... Established methods for scenario generation often utilize univariate, i.e., step-by-step prediction, approaches like classical autoregressive models (Sharma et al., 2013) or autoregressive neural networks (Vagropoulos et al., 2016;Voss et al., 2018). As opposed to univariate models, multivariate modeling techniques model a series of time steps in a single prediction step. ...
Preprint
Full-text available
We present a specialized scenario generation method that utilizes forecast information to generate scenarios for the particular usage in day-ahead scheduling problems. In particular, we use normalizing flows to generate wind power generation scenarios by sampling from a conditional distribution that uses day-ahead wind speed forecasts to tailor the scenarios to the specific day. We apply the generated scenarios in a simple stochastic day-ahead bidding problem of a wind electricity producer and run a statistical analysis focusing on whether the scenarios yield profitable and reliable decisions. Compared to conditional scenarios generated from Gaussian copulas and Wasserstein-generative adversarial networks, the normalizing flow scenarios identify the daily trends more accurately and with a lower spread while maintaining a diverse variety. In the stochastic day-ahead bidding problem, the conditional scenarios from all methods lead to significantly more profitable and reliable results compared to an unconditional selection of historical scenarios. The obtained profits using the normalizing flow scenarios are consistently closest to the perfect foresight solution, in particular, for small sets of only five scenarios.
... In addition to approaches based on recurrent models like LSTM and GRU, methods based on convolutional neural network (CNN) layers have been proposed. Voss et al.[215] apply a time series specic CNN, the WaveNET architecture based on causal and dilated convolutions at the household-level and show that it can outperform more straightforward approaches like MLR and ANN. Besides time series specic convolutions, two-and three-dimensional encodings of time series into images have also been applied. ...
Preprint
Full-text available
The increased digitalisation and monitoring of the energy system opens up numerous opportunities % and solutions which can help to decarbonise the energy system. Applications on low voltage (LV), localised networks, such as community energy markets and smart storage will facilitate decarbonisation, but they will require advanced control and management. Reliable forecasting will be a necessary component of many of these systems to anticipate key features and uncertainties. Despite this urgent need, there has not yet been an extensive investigation into the current state-of-the-art of low voltage level forecasts, other than at the smart meter level. This paper aims to provide a comprehensive overview of the landscape, current approaches, core applications, challenges and recommendations. Another aim of this paper is to facilitate the continued improvement and advancement in this area. To this end, the paper also surveys some of the most relevant and promising trends. It establishes an open, community-driven list of the known LV level open datasets to encourage further research and development.
... A 1D CNN is a CNN model that has a convolutional hidden layer operating over a 1D sequence. A typical 1D CNN model with input, one convolutional layer, one pooling layer and one fully connected layer is shown in Figure 2(a) [21][22][23]. Deep Learning libraries such as PyTorch, Keras, and TensorFlow can support well for a 1D, 2D, and 3D CNN [24]. In the paper, we used Keras, Python to implement a 1D CNN for time series prediction. ...
Article
Full-text available
The Convolutional Neural Network (CNN) model is one of the most effective models for load forecasting with hyperparameters which can be used not only to determine the CNN structure and but also to train the CNN model. This paper proposes a framework for Grid Search hyperparameters of the CNN model. In a training process, the optimal models will specify conditions that satisfy requirement for minimum of accuracy scores of Root Mean Square Error (RMSE), Mean Absolute Percentage Error (MAPE) and Mean Absolute Error (MAE). In the testing process, these optimal models will be used to evaluate the results along with all other ones. The results indicated that the optimal models have accuracy scores near the minimum values. Load demand data of Queensland (Australia) and Ho Chi Minh City (Vietnam) were utilized to verify the accuracy and reliability of the Grid Search framework. © 2021. The Author(s). This is an open-access article distributed under the terms of the Creative Commons Attribution-NonCommercial-NoDerivatives License (CC BY-NC-ND 4.0, https://creativecommons.org/licenses/by-nc-nd/4.0/), which permits use, distribution, and reproduction in any medium, provided that the Article is properly cited, the use is non-commercial, and no modifications or adaptations are made.
... Among the 77 papers reviewed, 57 used a parameter-based approach, representing the vast majority of the papers reviewed. Fig. 10 shows that parameter-based methods are mostly used in load prediction, with several architectures including: MLP ( [108] ) LSTM ( [109] ), GRU (gated recurrent units, [56] ), CNN (convolutional neural network, [58,110] ), CNN + LSTM ( [62] ), and LSTM-TLL ( [111] ). Other common applications are building dynamics and systems control, linked by the nonlinear nature of these topics, affected by stochastic variables or driven by physical laws that justify the wide adoption of neural networks. ...
Article
Full-text available
Smart buildings play a crucial role toward decarbonizing society, as globally buildings emit about one-third of greenhouse gases. In the last few years, machine learning has achieved a notable momentum that, if properly harnessed, may unleash its potential for advanced analytics and control of smart buildings, enabling the technique to scale up for supporting the decarbonization of the building sector. In this perspective, transfer learning aims to improve the performance of a target learner exploiting knowledge in related environments. The present work provides a comprehensive overview of transfer learning applications in smart buildings, classifying and analyzing 77 papers according to their applications, algorithms, and adopted metrics. The study identified four main application areas of transfer learning: (1) building load prediction, (2) occupancy detection and activity recognition, (3) building dynamics modeling, and (4) energy systems control. Furthermore, the review highlighted the role of deep learning in transfer learning applications that has been used in more than half of the analyzed studies. The paper also discusses how to integrate transfer learning in a smart building’s ecosystem, identifying, for each application area, the research gaps and guidelines for future research directions.
... The CNN was primarily designed for image recognition, while it has recently been employed for STLF [25]. In [26], a time-dependent CNN (TD-CNN) was developed to improve the forecasting accuracy of STLF based on only the historical electricity load data. ...
Article
Full-text available
Precise and reliable forecasting of short-term electricity load is essential to the development of smart grids. Particularly, deep neural networks (DNNs) are widely utilized for the prediction of short-term electricity load due to their automatic feature extraction ability. However, these available stacked deep-learning models may lose some temporal features or spatial features of original input data. To capture more comprehensive information, in this article, we present an integration scheme based on empirical mode decomposition (EMD), similar day methods, and DNNs to perform short-term load forecasting. It is especially worth noting that the electricity price is also an important factor for load variation, which is considered in our proposed scheme. Specifically, there are two primary layers: a feature extraction layer and a forecasting layer. In the feature extraction layer, EMD is applied to decompose load time series into several components, which are arranged into the 2-D input matrix of the convolutional neural network (CNN). Both the output vectors of the CNN and the raw load sequences are fed into the long short-term memory (LSTM) layer. Therefore, the whole EMD based CNN-LSTM approach extracts multimodal spatial-temporal features from input data. Meanwhile, the electricity price data is utilized to obtain multimodal spatial-temporal features in the same way. Additionally, the day and hour information and loads of similar days are to augment extra features for prediction. In the forecasting layer, the forecasting task is accomplished through a fully-connected neural network based on the outputs of the feature extraction layer. Leveraging these techniques enables our proposed scheme to extract more latent features, which significantly improve the accuracy. In order to demonstrate the performance of our proposed scheme, related experiments are conducted on actual data from the electricity market in Singapore. Compared to other available models, our proposed scheme is superior in graphic and numerical results.
Article
More and more factories and commercial buildings are installing combined heat and power (CHP) systems that include various energy storage devices. To reduce the energy cost of CHPs, optimal operation plans to satisfy time‐varying energy demands with minimum energy cost are required. Conventional operation planning methods using optimized calculation have an issue with long computing time. To address this, we adapted a convolutional neural network (CNN) to learn the time‐series features of input–output pairs of optimized calculations. The architecture of the proposed CNN has the following key features: (i) the application of one‐dimensional convolutional filters to extract the time‐series features of input data and (ii) the elimination of pooling layers that compress the input data. We also designed rule‐based flows to remove constraint violations from the generated operation plans. Our evaluations showed that the proposed method could generate operation plans with the average error of 3.3%, which is much lower than the 8.0% average error when using the conventional CNN with pooling calculation. © 2021 Institute of Electrical Engineers of Japan. Published by Wiley Periodicals LLC.
Article
Full-text available
Short-term load forecasting is integral to the energy planning sector. Various techniques have been employed to achieve effective operation of power systems and efficient market management. We present a scalable system for day-ahead household electrical energy consumption forecasting, named HousEEC. The proposed forecasting method is based on a deep residual neural network, and integrates multiple sources of information by extracting features from (i) contextual data (weather, calendar), and (ii) the historical load of the particular household and all households present in the dataset. Additionally, we compute novel domain-specific time-series features that allow the system to better model the pattern of energy consumption of the household. The experimental analysis and evaluation were performed on one of the most extensive datasets for household electrical energy consumption, Pecan Street, containing almost four years of data. Multiple test cases show that the proposed model provides accurate load forecasting results, achieving a root-mean-square error score of 0.44 kWh and mean absolute error score of 0.23 kWh, for short-term load forecasting for 300 households. The analysis showed that, for hourly forecasting, our model had 8% error (22 kWh), which is 4 percentage points better than the benchmark model. The daily analysis showed that our model had 2% error (131 kWh), which is significantly less compared to the benchmark model, with 6% error (360 kWh).
Article
Energy load forecasting plays an important role in the smart grid, which can affect the promoting energy production and consumption decision‐making processes. In this paper, the state‐of‐the‐art deep learning (DL) neural models are used in the short‐term load forecasting, including the multilayer perceptron (MLP), the convolutional neural network (CNN), and the long short‐term memory (LSTM). A novel loss function is proposed for the load forecasting, and two commonly used benchmarks are used to verify the validity of the proposed function. The simulation results show that the mean absolute percentage error (MAPE) of the proposed loss function is 19.63% lower than cross‐entropy and 2.34% lower than mean absolute error (MAE). We compared the mentioned neural networks in different aspects, and the results show that in energy load forecasting, CNN has superior performance than MLP and LSTM in terms of high accuracy and robustness to weather changes.
Article
Demand response, in which customers can actively reduce their energy consumption in respond to the price signal or incentives, is an effective technology to mitigate the temporary electricity shortage in modern power system. However, incentive‐based demand response event rarely happened. Thus, the customers' historical response to the peak event is seldom recorded, which leads to the difficulty in training the demand response behavior forecasting model. To solve the insufficient training data problem, a transfer learning–based model is proposed to forecast the customers' response to incentives, where the forecasting accuracy of a certain customer (target customer) is improved by introducing the load data of related customers (source customers). The whole forecasting model can be divided into two parts. (a) In the first part, the two‐layer load forecasting model is built to characterize the relations between the target customer and source customers. At the same time, variant weights are assigned to data sets according to source customers and according to their correlation to the target customer. (b) In the second step, with the latent variables and weight known, the weight and the latent variables are directly assigned to the corresponding demand response behavior data set. Then, the demand response forecasting model is trained, and the forecasting result of target customer is generated. The case studies are developed based on the data from the Smart Grid Smart City Customer Research (SGSC). The result shows the proposed model can achieve higher accuracy in forecasting the customer's response behavior to incentives, when compared with other benchmark models.
Article
Full-text available
This paper presents a recurrent neural network model to make medium-to-long term predictions, i.e. time horizon of ⩾1 week, of electricity consumption profiles in commercial and residential buildings at one-hour resolution. Residential and commercial buildings are responsible for a significant fraction of the overall energy consumption in the U.S. With advances in sensors and smart technologies, there is a need for medium to long-term prediction of electricity consumption in residential and commercial buildings at hourly intervals to support decision making pertaining to operations, demand response strategies, and installation of distributed generation systems. The modeler may have limited access to information about building's schedules and equipment, making data-driven machine learning models attractive. The energy consumption data that is available may also contain blocks of missing data, making time-series predictions difficult. Thus, the main objectives of this paper are: (a) Develop and optimize novel deep recurrent neural network (RNN) models aimed at medium to long term electric load prediction at one-hour resolution; (b) Analyze the relative performance of the model for different types of electricity consumption patterns; and (c) Use the deep NN to perform imputation on an electricity consumption dataset containing segments of missing values. The proposed models were used to predict hourly electricity consumption for the Public Safety Building in Salt Lake City, Utah, and for aggregated hourly electricity consumption in residential buildings in Austin, Texas. For predicting the commercial building's load profiles, the proposed RNN sequence-to-sequence models generally correspond to lower relative error when compared with the conventional multi-layered perceptron neural network. For predicting aggregate electricity consumption in residential buildings, the proposed model generally does not provide gains in accuracy compared to the multi-layered perceptron model.
Article
Full-text available
The scheduling and operation of power system becomes prominently complex and uncertain, especially with the penetration of distributed power. Load forecasting matters to the effective operation of power system. This paper proposes a novel deep learning framework to forecast the short-term grid load. First, the load data is processed by Box-Cox transformation, and two parameters (electricity price and temperature) are investigated. Then, to quantify the tail-dependence of power load on the two parameters, parametric Copula models are fitted and the threshold of peak load are computed. Next, a deep belief network is built to forecast the hourly load of the power grid. One year grid load data collected from an urbanized area in Texas, United States is utilized in the case studies. Short-term load forecasting are examined in four seasons independently. Day-ahead and week-ahead load forecasting experiments are conducted in each season using the proposed framework. The proposed framework is compared with classical neural networks, support vector regression machine, extreme learning machine, and classical deep belief networks. The load forecasting performances are assessed by mean absolute percentage error, root mean square error, and hit rate. Computational results confirm the effectiveness of the proposed data-driven deep learning framework. The prediction accuracies of both day-ahead forecasting and week-ahead forecasting demonstrate that the proposed framework outperforms the tested algorithms.
Article
Full-text available
As the power system is facing a transition towards a more intelligent, flexible and interactive system with higher penetration of renewable energy generation, load forecasting, especially short-term load forecasting for individual electric customers plays an increasingly essential role in the future grid planning and operation. Other than aggregated residential load in a large scale, forecasting an electric load of a single energy user is fairly challenging due to the high volatility and uncertainty involved. In this paper, we propose a long short-term memory (LSTM) recurrent neural network (RNN) based framework, which is the latest and one of the most popular techniques of deep learning, to tackle this tricky issue. The proposed framework is tested on a publicly available set of real residential smart meter data, of which the performance is comprehensively compared to various benchmarks including the state-of-the-arts in the field of load forecasting. As a result, the proposed LSTM approach outperforms the other listed rival algorithms in the task of short-term load forecasting for individual residential households.
Article
Full-text available
We present a method for conditional time series forecasting based on an adaptation of the recent deep convolutional WaveNet architecture. The proposed network contains stacks of dilated convolutions that allow it to access a broad range of history when forecasting, a ReLU activation function and conditioning is performed by applying multiple convolutional filters in parallel to separate time series which allows for the fast processing of data and the exploitation of the correlation structure between the multivariate time series. We test and analyze the performance of the convolutional network both unconditionally as well as conditionally for financial time series forecasting using the S&P500, the volatility index, the CBOE interest rate and several exchange rates and extensively compare it to the performance of the well-known autoregressive model and a long-short term memory network. We show that a convolutional network is well-suited for regression-type problems and is able to effectively learn dependencies in and between the series without the need for long historical time series, is a time-efficient and easy to implement alternative to recurrent-type networks and tends to outperform linear and recurrent models.
Article
Full-text available
Short-Term Load Forecasting (STLF) is a fundamental component in the efficient management of power systems, which has been studied intensively over the past 50 years. The emerging development of smart grid technologies is posing new challenges as well as opportunities to STLF. Load data, collected at higher geographical granularity and frequency through thousands of smart meters, allows us to build a more accurate local load forecasting model, which is essential for local optimization of power load through demand side management. With this paper, we show how several existing approaches for STLF are not applicable on local load forecasting, either because of long training time, unstable optimization process, or sensitivity to hyper-parameters. Accordingly, we select five models suitable for local STFL, which can be trained on different time-series with limited intervention from the user. The experiment, which consists of 40 time-series collected at different locations and aggregation levels, revealed that yearly pattern and temperature information are only useful for high aggregation level STLF. On local STLF task, the modified version of double seasonal Holt-Winter proposed in this paper performs relatively well with only 3 months of training data, compared to more complex methods.
Article
Full-text available
In the smart grid, one of the most important research areas is load forecasting; it spans from traditional time series analyses to recent machine learning approaches and mostly focuses on forecasting aggregated electricity consumption. However, the importance of demand side energy management, including individual load forecasting, is becoming critical. In this paper, we propose deep neural network (DNN)-based load forecasting models and apply them to a demand side empirical load database. DNNs are trained in two different ways: a pre-training restricted Boltzmann machine and using the rectified linear unit without pre-training. DNN forecasting models are trained by individual customer's electricity consumption data and regional meteorological elements. To verify the performance of DNNs, forecasting results are compared with a shallow neural network (SNN), a double seasonal Holt-Winters (DSHW) model and the autoregressive integrated moving average (ARIMA). The mean absolute percentage error (MAPE) and relative root mean square error (RRMSE) are used for verification. Our results show that DNNs exhibit accurate and robust predictions compared to other forecasting models, e.g., MAPE and RRMSE are reduced by up to 17% and 22% compared to SNN and 9% and 29% compared to DSHW.
Conference Paper
Full-text available
Ensuring sustainability demands more efficient energy management with minimized energy wastage. Therefore, the power grid of the future should provide an unprecedented level of flexibility in energy management. To that end, intelligent decision making requires accurate predictions of future energy demand/load, both at aggregate and individual site level. Thus, energy load forecasting have received increased attention in the recent past, however has proven to be a difficult problem. This paper presents a novel energy load forecasting methodology based on Deep Neural Networks, specifically Long Short Term Memory (LSTM) algorithms. The presented work investigates two variants of the LSTM: 1) standard LSTM and 2) LSTM-based Sequence to Sequence (S2S) architecture. Both methods were implemented on a benchmark data set of electricity consumption data from one residential customer. Both architectures where trained and tested on one hour and one-minute time-step resolution datasets. Experimental results showed that the standard LSTM failed at one-minute resolution data while performing well in one-hour resolution data. It was shown that S2S architecture performed well on both datasets. Further, it was shown that the presented methods produced comparable results with the other deep learning methods for energy forecasting in literature.
Conference Paper
Full-text available
The electrical demand forecasting problem can be regarded as a non-linear time series prediction problem depending on many complex factors since it is required at various aggregation levels and at high resolution. To solve this challenging problem, various time series and machine learning approaches has been proposed in the literature. As an evolution of neural network-based prediction methods, deep learning techniques are expected to increase the prediction accuracy by being stochastic and allowing bi-directional connections between neurons. In this paper, we investigate a newly developed deep learning model for time series prediction, namely Factored Conditional Restricted Boltzmann Machine (FCRBM), and extend it for demand forecasting. The assessment is made on the EcoGrid EU dataset, consisting of aggregated electric power consumption, price and meteorological data collected from 1900 customers. The households are equipped with local generation and smart appliances capable of responding to real-time pricing signals. The results show that for the energy prediction problem solved here, FCRBM outperforms the benchmark machine learning approach, i.e. Support Vector Machine.
Conference Paper
Full-text available
State-of-the-art models for semantic segmentation are based on adaptations of convolutional networks that had originally been designed for image classification. However, dense prediction problems such as semantic segmentation are structurally different from image classification. In this work, we develop a new convolutional network module that is specifically designed for dense prediction. The presented module uses dilated convolutions to systematically aggregate multi-scale contextual information without losing resolution. The architecture is based on the fact that dilated convolutions support exponential expansion of the receptive field without loss of resolution or coverage. We show that the presented context module increases the accuracy of state-of-the-art semantic segmentation systems. In addition, we examine the adaptation of image classification networks to dense prediction and show that simplifying the adapted network can increase accuracy.
Article
Full-text available
The increasing use of renewable energy sources with variable output, such as solar photovoltaic and wind power generation, calls for Smart Grids that effectively manage flexible loads and energy storage. The ability to forecast consumption at different locations in distribution systems will be a key capability of Smart Grids. The goal of this paper is to benchmark state-of-the-art methods for forecasting electricity demand on the household level across different granularities and time scales in an explorative way, thereby revealing potential shortcomings and find promising directions for future research in this area. We apply a number of forecasting methods including ARIMA, neural networks, and exponential smoothening using several strategies for training data selection, in particular day type and sliding window based strategies. We consider forecasting horizons ranging between 15 minutes and 24 hours. Our evaluation is based on two data sets containing the power usage of individual appliances at second time granularity collected over the course of several months. The results indicate that forecasting accuracy varies significantly depending on the choice of forecasting methods/strategy and the parameter configuration. Measured by the Mean Absolute Percentage Error (MAPE), the considered state-of-the-art forecasting methods rarely beat corresponding persistence forecasts. Overall, we observed MAPEs in the range between 5 and >100%. The average MAPE for the first data set was ~30%, while it was ~85% for the other data set. These results show big room for improvement. Based on the identified trends and experiences from our experiments, we contribute a detailed discussion of promising future research.
Article
Full-text available
As low carbon technologies become more pervasive, distribution network operators are looking to support the expected changes in the demands on the low voltage networks through the smarter control of storage devices. Accurate forecasts of demand at the individual household-level, or of small aggregations of households, can improve the peak demand reduction brought about through such devices by helping to plan the most appropriate charging and discharging cycles. However, before such methods can be developed, validation measures which can assess the accuracy and usefulness of forecasts of the volatile and noisy household-level demand are required. In this paper we introduce a new forecast verification error measure that reduces the so-called “double penalty” effect, incurred by forecasts whose features are displaced in space or time, compared to traditional point-wise metrics, such as the Mean Absolute Error, and pp-norms in general. The measure that we propose is based on finding a restricted permutation of the original forecast that minimises the point-wise error, according to a given metric. We illustrate the advantages of our error measure using half-hourly domestic household electrical energy usage data recorded by smart meters, and discuss the effect of the permutation restriction.
Conference Paper
Full-text available
INTRODUCTIONThe ability of multilayer back-propagation networks to learn complex, high-dimensional, nonlinearmappings from large collections of examples makes them obvious candidates for imagerecognition or speech recognition tasks (see PATTERN RECOGNITION AND NEURALNETWORKS). In the traditional model of pattern recognition, a hand-designed featureextractor gathers relevant information from the input and eliminates irrelevant variabilities.A trainable classifier then categorizes the...
Article
Full-text available
Scikit-learn is a Python module integrating a wide range of state-of-the-art machine learning algorithms for medium-scale supervised and unsupervised problems. This package focuses on bringing machine learning to non-specialists using a general-purpose high-level language. Emphasis is put on ease of use, performance, documentation, and API consistency. It has minimal dependencies and is distributed under the simplified BSD license, encouraging its use in both academic and commercial settings. Source code, binaries, and documentation can be downloaded from http://scikit-learn.sourceforge.net.
Article
This paper proposes a simple empirical scaling law that describes load forecasting accuracy at varying levels of aggregation. We show that for many forecasting methods, aggregating more customers improves the relative forecasting performance up to specific point. Beyond this point, no more improvement in relative performance can be obtained. A benchmarking procedure for applying the scaling law to different forecasting models is presented. The aggregation model is evaluated with year long consumption profiles of over 180 thousand Pacific Gas & Electric customers. A theoretical model based on a bias variance decomposition of the forecast error is used to model the Aggregation Error Curves (AECs) that are empirically explored.
Conference Paper
Load forecasting is an important electric utility task for planning resources in Smart grid. This function also aids in predicting the behavior of energy systems in reducing dynamic uncertainties. The efficiency of the entire grid operation depends on accurate load forecasting. This paper proposes and investigates the application of a multi-layered deep neural network to the Iberian electric market (MIBEL) forecasting task. Ninety days of energy demand data are used to train the proposed model. The ninety-day period is treated as a historical dataset to train and predict the demand for day-ahead markets. The network structure is implemented using Google's machine learning Tensor-flow platform. Various combinations of activation functions were tested to achieve a better Mean Absolute percentage error (MAPE) considering the weekday and weekend variations. The tested functions include Sigmoid, Rectifier linear unit (ReLU), and Exponential linear unit (ELU). The preliminary results are promising. and show significant savings in the MAPE values using the ELU function over the other activation functions.
Article
The key challenge for household load forecasting lies in the high volatility and uncertainty of load profiles. Traditional methods tend to avoid such uncertainty by load aggregation (to offset uncertainties), customer classification (to cluster uncertainties) and spectral analysis (to filter out uncertainties). This paper, for the first time, aims to directly learn the uncertainty by applying a new breed of machine learning algorithms – deep learning. However simply adding layers in neural networks will cap the forecasting performance due to the occurrence of overfitting. A novel pooling-based deep recurrent neural network (PDRNN) is proposed in this paper which batches a group of customers’ load profiles into a pool of inputs. Essentially the model could address the over-fitting issue by increasing data diversity and volume. This work reports the first attempts to develop a bespoke deep learning application for household load forecasting and achieved preliminary success. The developed method is implemented on Tensorflow deep learning platform and tested on 920 smart metered customers from Ireland. Compared with the state-of-art techniques in household load forecasting, the proposed method outperforms ARIMA by 19.5%, SVR by 13.1% and classical deep RNN by 6.5% in terms of RMSE.
Article
Energy consumption forecasting for buildings has immense value in energy efficiency and sustainability research. Accurate energy forecasting models have numerous implications in planning and energy optimization of buildings and campuses. For new buildings, where past recorded data is unavailable, computer simulation methods are used for energy analysis and forecasting future scenarios. However, for existing buildings with historically recorded time series energy data, statistical and machine learning techniques have proved to be more accurate and quick. This study presents a comprehensive review of the existing machine learning techniques for forecasting time series energy consumption. Although the emphasis is given to a single time series data analysis, the review is not just limited to it since energy data is often co-analyzed with other time series variables like outdoor weather and indoor environmental conditions. The nine most popular forecasting techniques that are based on the machine learning platform are analyzed. An in-depth review and analysis of the ‘hybrid model’, that combines two or more forecasting techniques is also presented. The various combinations of the hybrid model are found to be the most effective in time series energy forecasting for building.
Conference Paper
Electric load profile forecasting is among the most important tasks in power system operation. This task has been performed reasonably well with minimal disruption from renewables over the past decades. The expected high penetration of renewables in the system is challenging this classical task. Moreover, the blossom of demand side management programs warrants the necessity of load profile forecasting even for individual (residential as well as industrial) consumers. In this paper, we seek to define predictability in a rigorous way from an information theoretic perspective, which shed light on comparing the predictability between different kinds of loads. Then, we identify two separable components of predictability - constancy and contingency. We further exploit the key parameters in deciding the predictability, which challenge the commonly-held beliefs. For example, does a finer granularity imply a lower predictability? To what degree does the law of large numbers help improve the predictability?
Article
A number of recent studies use deep belief networks (DBN) with a great success in various applications such as image classification and speech recognition. In this paper, a DBN made up from multiple layers of restricted Boltzmann machines is used for electricity load forecasting. The layer-by-layer unsupervised training procedure is followed by fine-tuning of the parameters by using a supervised back-propagation training method. Our DBN model was applied to short-term electricity load forecasting based on the Macedonian hourly electricity consumption data in the period 2008–2014. The obtained results are not only compared with the latest actual data, but furthermore, they are compared with the predicted data obtained from a typical feed-forward multi-layer perceptron neural network and with the forecasted data provided by the Macedonian system operator (MEPSO). The comparisons show that the applied model is not only suitable for hourly electricity load forecasting of the Macedonian electric power system, it actually provides superior results than the ones obtained using traditional methods. The mean absolute percentage error (MAPE) is reduced by up to 8.6% when using DBN, compared to the MEPSO data for the 24-h ahead forecasting, and the MAPE for daily peak forecasting is reduced by up to 21%.
Article
The energy industry has been going through a significant modernization process over the last decade. Its infrastructure is being upgraded rapidly. The supply, demand and prices are becoming more volatile and less predictable than ever before. Even its business model is being challenged fundamentally. In this competitive and dynamic environment, many decision-making processes rely on probabilistic forecasts to quantify the uncertain future. Although most of the papers in the energy forecasting literature focus on point or single-valued forecasts, the research interest in probabilistic energy forecasting research has taken off rapidly in recent years. In this paper, we summarize the recent research progress on probabilistic energy forecasting. A major portion of the paper is devoted to introducing the Global Energy Forecasting Competition 2014 (GEFCom2014), a probabilistic energy forecasting competition with four tracks on load, price, wind and solar forecasting, which attracted 581 participants from 61 countries. We conclude the paper with 12 predictions for the next decade of energy forecasting.
Article
Deep neural nets with a large number of parameters are very powerful machine learning systems. However, overfitting is a serious problem in such networks. Large networks are also slow to use, making it difficult to deal with overfitting by combining the predictions of many different large neural nets at test time. Dropout is a technique for addressing this problem. The key idea is to randomly drop units (along with their connections) from the neural network during training. This prevents units from co-adapting too much. During training, dropout samples from an exponential number of different "thinned" networks. At test time, it is easy to approximate the effect of averaging the predictions of all these thinned networks by simply using a single unthinned network that has smaller weights. This significantly reduces overfitting and gives major improvements over other regularization methods. We show that dropout improves the performance of neural networks on supervised learning tasks in vision, speech recognition, document classification and computational biology, obtaining state-of-the-art results on many benchmark data sets. © 2014 Nitish Srivastava, Geoffrey Hinton, Alex Krizhevsky, Ilya Sutskever and Ruslan Salakhutdinov.
Article
State-of-the-art models for semantic segmentation are based on adaptations of convolutional networks that had originally been designed for image classification. However, dense prediction and image classification are structurally different. In this work, we develop a new convolutional network module that is specifically designed for dense prediction. The presented module uses dilated convolutions to systematically aggregate multi-scale contextual information without losing resolution. The architecture is based on the fact that dilated convolutions support exponential expansion of the receptive field without loss of resolution or coverage. We show that the presented context module increases the accuracy of state-of-the-art semantic segmentation systems. In addition, we examine the adaptation of image classification networks to dense prediction and show that simplifying the adapted network can increase accuracy.
Article
Accurate Short Term Load Forecasting is an essential step towards load balancing methods in energy systems. With the recent introduction of Smart Meters for residential buildings, load forecasting and shifting methods can be implemented for individual households. The high variance of the load demand on the household level requires specific forecasting methods. This paper provides an overview of the methods which have been applied and points out what results are comparable. Therefore a structured literature review is carried out. In the process, 375 papers are analyzed and categorized via a concept matrix. Based on this review it is pointed out, which methods achieve good results for which purpose and which publicly available datasets can be used for evaluation.
Article
Data analytics in smart grids can be leveraged to channel the data downpour from individual meters into knowledge valuable to electric power utilities and end-consumers. Short-term load forecasting (STLF) can address issues vital to a utility but it has traditionally been done mostly at system (city or country) level. In this case study, we exploit rich, multi-year, and high-frequency annotated data collected via a metering infrastructure to perform STLF on aggregates of power meters in a mid-sized city. For smart meter aggregates complemented with geo-specific weather data, we benchmark several state-of-the-art forecasting algorithms, including kernel methods for nonlinear regression, seasonal and temperature-adjusted auto-regressive models, exponential smoothing and state-space models. We show how STLF accuracy improves at larger meter aggregation (at feeder, substation, and system-wide level). We provide an overview of our algorithms for load prediction and discuss system performance issues that impact real time STLF. ® 2014 Alcatel-Lucent.
Conference Paper
The recent development of smart meters has allowed the analysis of household electricity consumption in real time. Predicting electricity consumption at such very low scales should help to increase the efficiency of distribution networks and energy pricing. However, this is by no means a trivial task since household-level consumption is much more irregular than at the transmission or distribution levels. In this work, we address the problem of improving consumption forecasting by using the statistical relations between consumption series. This is done both at the household and district scales (hundreds of houses), using various machine learning techniques, such as support vector machine for regression (SVR) and multilayer perceptron (MLP). First, we determine which algorithm is best adapted to each scale, then, we try to find leaders among the time series, to help short-term forecasting. We also improve the forecasting for district consumption by clustering houses according to their consumption profiles.
Article
Benchmarking issue in short term load forecasting has not received as much attention as it deserves. Although dozens of techniques have been reported to be applied to short term load forecasting, most of them are still on the theoretical level with insignificant practical value. None of them has been established to produce benchmarking models for comparative assessment. This paper proposes a naïve multiple linear regression benchmark for short term load forecasting, which is from the experience of helping a US utility develop the first in- house short term load forecasts. The proposed model has been served as a benchmark for this utility since 2009, and was in production use for a year with satisfying performance before a major upgrade. It has also been used for a Canadian utility for load forecasting purposes. In addition, it was reproduced by a group of graduate students from a creditable US university following the documented procedure.
Conference Paper
This paper proposes a recurrent neural network based approach to short-term load forecasting in power systems. Recurrent neural networks in multilayer perceptrons have an advantage that the context layer is able to cope with historical data. As a result, it is expected that recurrent neural networks give better solutions than the conventional feedforward multilayer perceptrons in term of accuracy. Also, the differential equation form of the time series is utilized to deal with the nonstationarity of the daily load time series. Furthermore, this paper proposes the diffusion learning method for determining weights between units in a recurrent network. The method is capable of escaping from local minima with stochastic noise. A comparison is made between conventional multilayer perceptrons and the proposed method for actual data.
Wavenet: A generative model for raw audio
  • A Van Den
  • S Oord
  • H Dieleman
  • K Zen
  • O Simonyan
  • A Vinyals
  • N Graves
  • A Kalchbrenner
  • K Senior
  • Kavukcuoglu
A. Van Den Oord, S. Dieleman, H. Zen, K. Simonyan, O. Vinyals, A. Graves, N. Kalchbrenner, A. Senior, and K. Kavukcuoglu, "Wavenet: A generative model for raw audio," arXiv preprint:1609.03499, 2016.
A naïve multiple linear regression benchmark for short term load forecasting
  • T Hong
  • P Wang
  • H L Willis
T. Hong, P. Wang, and H. L. Willis, "A naïve multiple linear regression benchmark for short term load forecasting," in Power and Energy Society General Meeting, 2011 IEEE. IEEE, 2011, pp. 1-6.