No full-text available
To read the full-text of this research,
you can request a copy directly from the authors.
Electricity Demand Forecasting Using Computational Intelligence and High Performance Computing
Abstract
This article presents the application of parallel computing for building different computational intelligence models applied to the forecast of the hourly electricity demand of the following day. The short-term forecast of electricity demand is a crucial problem to define the dispatch of generators. In turn, it is necessary to define demand response policies related with smart grids. Computational intelligence models have emerged as successful methods for prediction in recent years. The large amount of existing data from different sources and the great development of supercomputing allows to build models with adequate complexity to represent all the variables that improves the prediction. Parallel computing techniques are applied to obtain two artificial neural network architectures and its related parameters to forecast the total electricity demand of Uruguay for the next day. These techniques consists in train and evaluate models in parallel with different architectures and sets of parameters using grid search techniques. Furthermore each model is trained using Tensorflow with finite-grained GPU parallelism. Considering the high computing demands of the applied techniques, they are developed and executed on the high performance computing platform provided by National Supercomputing Center (Cluster-UY), Uruguay. Standard performance metrics are applied to evaluate the proposed models. The experimental evaluation of the best model reports excellent forecasting results. This model has a mean absolute percentage error of when applied to the prediction of unseen data.
ResearchGate has not been able to resolve any citations for this publication.
This article presents electricity demand forecasting models for industrial and residential facilities, developed using ensemble machine learning strategies. Short term electricity demand forecasting is beneficial for both consumers and suppliers, as it allows improving energy efficiency policies and the rational use of resources. Computational intelligence models are developed for day-ahead electricity demand forecasting. An ensemble strategy is applied to build the day-ahead forecasting model based on several one-hour models. Three steps of data preprocessing are carried out, including treating missing values, removing outliers, and standardization. Feature extraction is performed to reduce overfitting, reducing the training time and improving the accuracy. The best model is optimized using grid search strategies on hyperparameter space. Then, an ensemble of 24 instances is generated to build the complete day-ahead forecasting model. Considering the computational complexity of the applied techniques, they are developed and evaluated on the National Supercomputing Center (Cluster-UY), Uruguay. Three different real data sets are used for evaluation: an industrial park in Burgos (Spain), the total electricity demand for Uruguay, and demand from a distribution substation in Montevideo (Uruguay). Standard performance metrics are applied to evaluate the proposed models. The main results indicate that the best day ahead model based on ExtraTreesRegressor has a mean absolute percentage error of 2:55% on industrial data, 5:17% on total consumption data and 9:09% on substation data.
Non-intrusive load monitoring allows breaking down the aggregated household consumption into a detailed consumption per appliance, without installing extra hardware, apart of a smart meter. Breakdown information is very useful for both users and electric companies, to provide an accurate characterization of energy consumption, avoid peaks, and elaborate special tariffs to reduce the cost of the electricity bill. This article presents an approach for energy consumption disaggregation in residential households, based on detecting similar patterns of recorded consumption from labeled datasets. The proposed algorithm is evaluated using four different instances of the problem, which use synthetically generated data based on real energy consumption. Each generated dataset normalize the consumption values of the appliances to create complex scenarios. The nilmtk framework is used to process the results and to perform a comparison with two built-in algorithms provided by the framework, based on combinatorial optimization and factorial hidden Markov model. The proposed algorithm was able to achieve accurate results, despite the presence of ambiguity between the consumption of different appliances or the difference of consumption between training appliances and test appliances.
This article describes the Cloud Computing for Smart Energy Management (CC-SEM) project, a research effort focused on building an integrated platform for smart monitoring, controlling, and planning energy consumption and generation in urban scenarios. The project integrates cutting-edge technologies (Big Data analysis, computational intelligence, Internet of Things, High-Performance Computing and Cloud Computing), specific hardware for energy monitoring/controlling built within the project and efficient communication protocols. The proposed platform considers the point of view of both citizens and administrators, providing a set of tools for controlling home devices (for end-users), lanning/simulating scenarios of energy generation (for energy companies and administrators), and proposes advances in communication infrastructure for transmitting the generated data.
This article proposes a platform for distributed big data analysis in the context of smart cities. Extracting useful mobility information from large volumes of data is crucial to improve decision-making processes in smart cities. This article introduces a framework for mobility analysis in smart cities combining Intelligent Transportation Systems and socioeconomic data for the city of Montevideo, Uruguay. The efficiency of the proposed system is analyzed over a distributed computing infrastructure, demonstrating that the system scales properly for processing large volumes of data for both off-line and on-line scenarios. Applications of the proposed platform and case studies using real data are presented, as examples of the valuable information that can be offered to both citizens and authorities. The proposed model for big data processing can also be extended to allow using other distributed (e.g. grid, cloud, fog, edge) computing infrastructures.
We propose a novel methodology for high-dimensional time series prediction based on the kernel method extension of data-driven Koopman spectral analysis, via the following methodological advances: (a) a new numerical regularization method, (b) a natural ordering of Koopman modes which provides a fast alternative to the sparsity-promoting procedure, (c) a predictable Koopman modes selection technique which is equivalent to cross-validation in machine learning, (d) an optimization method for selected Koopman modes to improve prediction accuracy, (e) prediction model generation and selection based on historical error measures. The prediction accuracy of this methodology is excellent: for example, when it is used to predict clients’ order flow time series of foreign exchange, which is almost random, it can achieve more than 10% improvement on root-mean-square error over auto-regressive moving average. This methodology also opens up new possibilities for data-driven modeling and forecasting complex systems that generate the high-dimensional time series. We believe that this methodology will be of interest to the community of scientists and engineers working on quantitative finance, econometrics, system biology, neurosciences, meteorology, oceanography, system identification and control, data mining, machine learning, and many other fields involving high-dimensional time series and spatio-temporal data.
Any sufficiently complex system acts as a black box when it becomes easier to experiment with than to understand. Hence, black-box optimization has become increasingly important as systems have become more complex. In this paper we describe Google Vizier, a Google-internal service for performing black-box optimization that has become the de facto parameter tuning engine at Google. Google Vizier is used to optimize many of our machine learning models and other systems, and also provides core capabilities to Google's Cloud Machine Learning HyperTune subsystem. We discuss our requirements, infrastructure design, underlying algorithms, and advanced features such as transfer learning and automated early stopping that the service provides.
INTRODUCTIONThe ability of multilayer back-propagation networks to learn complex, high-dimensional, nonlinearmappings from large collections of examples makes them obvious candidates for imagerecognition or speech recognition tasks (see PATTERN RECOGNITION AND NEURALNETWORKS). In the traditional model of pattern recognition, a hand-designed featureextractor gathers relevant information from the input and eliminates irrelevant variabilities.A trainable classifier then categorizes the...
Learning to store information over extended time intervals by recurrent backpropagation takes a very long time, mostly because of insufficient, decaying error backflow. We briefly review Hochreiter's (1991) analysis of this problem, then address it by introducing a novel, efficient, gradient based method called long short-term memory (LSTM). Truncating the gradient where this does not do harm, LSTM can learn to bridge minimal time lags in excess of 1000 discrete-time steps by enforcing constant error flow through constant error carousels within special units. Multiplicative gate units learn to open and close access to the constant error flow. LSTM is local in space and time; its computational complexity per time step and weight is O. 1. Our experiments with artificial data involve local, distributed, real-valued, and noisy pattern representations. In comparisons with real-time recurrent learning, back propagation through time, recurrent cascade correlation, Elman nets, and neural sequence chunking, LSTM leads to many more successful runs, and learns much faster. LSTM also solves complex, artificial long-time-lag tasks that have never been solved by previous recurrent network algorithms.
Forecasting the day-ahead electricity load is beneficial for both suppliers and consumers. The reduction of electricity waste and the rational dispatch of electric generator units can be significantly improved with accurate load forecasts. This article is focused on studying and developing computational intelligence techniques for electricity load forecasting. Several models are developed to forecast the electricity load of the next hour using real data from an industrial pole in Spain. Feature selection and feature extraction are performed to reduce overfitting and therefore achieve better models, reducing the training time of the developed methods. The best of the implemented models is optimized using grid search strategies on hyperparameter space. Then, twenty four different instances of the optimal model are trained to forecast the next twenty four hours. Considering the computational complexity of the applied techniques, they are developed and evaluated on the computational platform of the National Supercomputing Center (Cluster-UY), Uruguay. Standard performance metrics are applied to evaluate the proposed models. The main results indicate that the best model based on ExtraTreesRegressor obtained has a mean absolute percentage error of 2.55% on day ahead hourly forecast which is a promising result.
This article describes the national initiative for installing and operating a collaborative scientific HPC infrastructure in Uruguay (Cluster-UY). The project was conceived as a mean to foster research and innovation projects that face complex problems with high computing demands. The main ideas and motivations of the Cluster-UY project are described. The technological decisions to install the platform are explained and the collaborative operation model to guarantee sustainability is introduced. In addition, the perspectives of the national scientific HPC initiative are highlighted and sample current projects are presented.
This paper proposes a novel causality analysis approach called the Causal Markov Elman Network (CMEN) to characterize the interdependency among heterogeneous time-series in multi-network systems. The CMEN performance, which comprises of inputs filtered by Markov property, successfully characterizes various multivariate dependencies in an urban environment. The paper also proposes a novel hypothesis of characterizing joint information between interconnected systems such as electricity and transportation networks. The proposed methodology and the hypotheses are then validated by Information Theory distance-based metrics. For cross-validation, the CMEN is applied to the electricity load forecasting problem using actual data from the City of Tallahassee, Florida. IEEE
Due to the explosion in restructuring of power markets within a deregulated economy, competitive power market needs to minimize their required generation reserve gaps. Efficient load forecasting for future demands can minimize the gap which will help in economic power generation, power operations, power construction planning and power distribution. Nowadays, neural networks are widely used for solving load forecasting problem due to its non-linear characteristics. Consequently, neural network is successfully combined with optimization techniques for finding optimal network parameters in order to reduce the forecasting error. In this paper, firstly a novel evolutionary algorithm based on follow the leader concept is developed and thereafter its performance is validated by COmparing Continuous Optimizers experimental framework on the set of 24 Black-Box Optimization Benchmarking functions with 12 state-of-art algorithms in 2-D, 3-D, 5-D, 10-D, and 20-D. The proposed algorithm outperformed all state-of-art algorithms in 20-D and ranked second in other dimensions. Further, the proposed algorithm is integrated with neural network for the proper tuning of network parameters to solve the real world problem of short term load forecasting. Through experiments on three real-world electricity load data sets namely New Pool England, New South Wales and Electric Reliability Council of Texas, we compared our proposed hybrid approach to baseline approaches and demonstrated its effectiveness in terms of predictive accuracy measures.
Neural networks dominate the modern machine learning landscape, but their training and success still suffer from sensitivity to empirical choices of hyperparameters such as model architecture, loss function, and optimisation algorithm. In this work we present \emph{Population Based Training (PBT)}, a simple asynchronous optimisation algorithm which effectively utilises a fixed computational budget to jointly optimise a population of models and their hyperparameters to maximise performance. Importantly, PBT discovers a schedule of hyperparameter settings rather than following the generally sub-optimal strategy of trying to find a single fixed set to use for the whole course of training. With just a small modification to a typical distributed hyperparameter training framework, our method allows robust and reliable training of models. We demonstrate the effectiveness of PBT on deep reinforcement learning problems, showing faster wall-clock convergence and higher final performance of agents by optimising over a suite of hyperparameters. In addition, we show the same method can be applied to supervised learning for machine translation, where PBT is used to maximise the BLEU score directly, and also to training of Generative Adversarial Networks to maximise the Inception score of generated images. In all cases PBT results in the automatic discovery of hyperparameter schedules and model selection which results in stable training and better final performance.
This paper presents an improved technique for short-term electric load forecasting making use of boosted neural networks (BooNN). The BooNN consist of combining a set of artificial neural networks (ANNs) trained iteratively. At each iteration, the error between the estimated output from the ANN model trained in the previous iteration and the target output is minimized. The final predicted result is the weighted sum of output from all the trained models. This process reduces the magnitude of forecasting errors and their variation compared to a single ANN and bagged neural networks (BNN). It further significantly lowers computational time compared to BNN. Results with real data further confirm that BooNN lead to improved load forecasting performance with respect to other existing techniques.
Traditionally, short-term electricity price forecasting has been essential for utilities and generation companies. However, the deregulation of electricity markets created a competitive environment and the introduction of new market participants, such as the retailers and aggregators, whose economic viability and profitability highly depends on the spot market price patterns. The aim of this study is to examine artificial neural network (ANN) based models for Day-ahead price forecasting. Specifically, the models refer to the sole application of ANNs or to hybrid models, where the ANN is combined with clustering algorithm. The training data are clustered in homogenous groups and for each cluster, a dedicated forecaster is employed. The proposed models are characterized by comprehensive operation and by high level of flexibility; different inputs can be taken under consideration and different ANN topologies can be examined. The models are tested on a data set that consists of atypical price patterns and many outliers. This approach makes the price forecasting problem a more challenging task, providing evidence that the proposed models can be considered as useful and robust forecasting tools to the actual needs of market participants, including the traditional generation companies and self-producers, but also the retailers/suppliers and aggregators.
Growth in electricity demand also gives a rise to the necessity of cheaper and safer electric supply and forecasting electricity load plays a key role in this goal. In this study recurrent extreme learning machine (RELM) was proposed as a novel approach to forecast electricity load more accurately. In RELM, extreme learning machine (ELM), which is a training method for single hidden layer feed forward neural network, was adapted to train a single hidden layer Jordan recurrent neural network. Electricity Load Diagrams 2011–2014 dataset was employed to evaluate and validate the proposed approach. Obtained results were compared with traditional ELM, linear regression, generalized regression neural network and some other popular machine learning methods. Achieved root mean square errors (RMSE) by RELM were nearly twice less than obtained results by other employed machine learning methods. The results showed that the recurrent type ANNs had extraordinary success in forecasting dynamic systems and also time-ordered datasets with comparison to feed forward ANNs. Also, used time in the training stage is similar to ELM and they are extremely fast than the others. This study showed that the proposed approach can be applied to forecast electricity load and RELM has high potential to be utilized in modeling dynamic systems effectively.
Deep neural nets with a large number of parameters are very powerful machine learning systems. However, overfitting is a serious problem in such networks. Large networks are also slow to use, making it difficult to deal with overfitting by combining the predictions of many different large neural nets at test time. Dropout is a technique for addressing this problem. The key idea is to randomly drop units (along with their connections) from the neural network during training. This prevents units from co-adapting too much. During training, dropout samples from an exponential number of different "thinned" networks. At test time, it is easy to approximate the effect of averaging the predictions of all these thinned networks by simply using a single unthinned network that has smaller weights. This significantly reduces overfitting and gives major improvements over other regularization methods. We show that dropout improves the performance of neural networks on supervised learning tasks in vision, speech recognition, document classification and computational biology, obtaining state-of-the-art results on many benchmark data sets. © 2014 Nitish Srivastava, Geoffrey Hinton, Alex Krizhevsky, Ilya Sutskever and Ruslan Salakhutdinov.
Although artificial neural networks have recently gained importance in time series applications, some methodological shortcomings still continue to exist. One of these shortcomings is the selection of the final neural network model to be used to evaluate its performance in test set among many neural networks. The general way to overcome this problem is to divide data sets into training, validation, and test sets and also to select a neural network model that provides the smallest error value in the validation set. However, it is likely that the selected neural network model would be overfitting the validation data. This paper proposes a new model selection strategy (IHTS) for forecasting with neural networks. The proposed selection strategy first determines the numbers of input and hidden units, and then, selects a neural network model from various trials caused by different initial weights by considering validation and training performances of each neural network model. It is observed that the proposed selection strategy improves the performance of the neural networks statistically as compared with the classic model selection method in the simulated and real data sets. Also, it exhibits some robustness against the size of the validation data.
In this paper, fuzzy (inaccuracy, vague, dispersion of individual interpretation) and random (incompleteness, noise and variability) uncertainties of electric load forecasting are modeled by random fuzzy variables (RFVs). Further integrating it into neural networks (NN) to formulate a novel integrated technique–Random Fuzzy NN (RFNN) for load forecasting is presented. The features of this methodology are as follows. (1) It is able to effectively and simultaneously model referred uncertainties occurred in load forecasting by one integrated technique, which existing techniques (e.g. fuzzified NN or Bayesian NN) tackle them separately. (2) Specially, historical data/information containing incompleteness, inaccuracy and vagueness can be modeled by the proposed representations of RFVs. No preprocessing algorithms such as data imputation or discard are required. (3) The proposed RFNN can make NN incorporating both types of uncertainties of inputs and network parameters so as to possess with better tackling uncertainties of load forecasting than other relevant methods. The proposed techniques are applied to electric load forecasting using a real operational data collected from Macau electric utility. Its application is promising in microgrid/small power system or in forecasting curves of individual customer where load curves would present a much higher variability and more noise than global curves of power grid of one country/region.
In this paper, a novel combination of wavelet and Elman network as a recurrent neural network is proposed to predict 1-day-ahead electrical power load under the influence of temperature. Using wavelet multi-resolution analysis, the load series are decomposed to different sub-series, showing different frequency characteristics of the load. Elman network (EN) is optimally designed and trained using static back propagation algorithm based on the optimization of performance measures such as mean square error, correlation coefficient and mean absolute percentage error on test prediction dataset. Feasibility of Daubechies wavelet at different scales with suitable number of decomposition levels is investigated to choose the best order for different seasonal load series. The estimated models are evaluated over different temperature and humidity in order to examine their impact on accurate load prediction. The reliability and consistency in prediction by the adopted technique is maintained even in the presence of controlled Gaussian noise to the predicted temperature series.
Information about the patterns that govern the energy demand and onsite generation can generate significant savings in the range of 15-30% in most cases and thus is essential for the management of commercial building energy systems. Predominantly, heating and cooling in a building as well as the availability of solar and wind energy are directly affected by variables such as temperature, humidity and solar radiation. This makes energy management decision making and planning sensitive to the prevalent and future weather conditions. Research attempts are being made using a variety of statistical or physical algorithms to predict the evolution of the building load or generation in order to optimise the building energy management The response of the building energy system to changes in weather conditions is inherently challenging to predict; nevertheless numerous methods in the literature describe and utilise weather predictions. Such methods are being reviewed in this study and their strengths, weaknesses and applications in commercial buildings at different prediction horizons are discussed. Furthermore, the importance of considering weather forecasting inputs in energy management systems is established by highlighting the dependencies of various building components on weather conditions. The issues of the difficulty in implementation of integrated weather forecasts at commercial building level and the potential added value through energy management optimisation are also addressed. Finally, a novel framework is proposed that utilises a range of weather variable predictions in order to optimise certain commercial building systems.
Time series prediction techniques have been used in many real-world applications such as financial market prediction, electric utility load forecasting , weather and environmental state prediction, and reliability forecasting. The underlying system models and time series data generating processes are generally complex for these applications and the models for these systems are usually not known a priori. Accurate and unbiased estimation of the time series data produced by these systems cannot always be achieved using well known linear techniques, and thus the estimation process requires more advanced time series prediction algorithms. This paper provides a survey of time series prediction applications using a novel machine learning approach: support vector machines (SVM). The underlying motivation for using SVMs is the ability of this methodology to accurately forecast time series data when the underlying system processes are typically nonlinear, non-stationary and not defined a-priori. SVMs have also been proven to outperform other non-linear techniques including neural-network based non-linear prediction techniques such as multi-layer perceptrons.The ultimate goal is to provide the reader with insight into the applications using SVM for time series prediction, to give a brief tutorial on SVMs for time series prediction, to outline some of the advantages and challenges in using SVMs for time series prediction, and to provide a source for the reader to locate books, technical journals, and other online SVM research resources.
Support vector machines have been very successful in pattern recognition and function estimation problems. In this paper we introduce the use of least squares support vector machines (LS-SVM's) for the optimal control of nonlinear systems. Linear and neural full static state feedback controllers are considered. The problem is formulated in such a way that it incorporates the N-stage optimal control problem as well as a least squares support vector machine approach for mapping the state space into the action space. The solution is characterized by a set of nonlinear equations. An alternative formulation as a constrained nonlinear optimization problem in less unknowns is given, together with a method for imposing local stability in the LS-SVM control scheme. The results are discussed for support vector machines with radial basis function kernel. Advantages of LS-SVM control are that no number of hidden units has to be determined for the controller and that no centers have to be specified for the Gaussian kernels when applying Mercer's condition. The curse of dimensionality is avoided in comparison with defining a regular grid for the centers in classical radial basis function networks. This is at the expense of taking the trajectory of state variables as additional unknowns in the optimization problem, while classical neural network approaches typically lead to parametric optimization problems. In the SVM methodology the number of unknowns equals the number of training data, while in the primal space the number of unknowns can be infinite dimensional. The method is illustrated both on stabilization and tracking problems including examples on swinging up an inverted pendulum with local stabilization at the endpoint and a tracking problem for a ball and beam system.