Content uploaded by Dazhi Yang
Author content
All content in this area was uploaded by Dazhi Yang on Mar 31, 2022
Content may be subject to copyright.
A review of solar forecasting, its dependence on atmospheric sciences and
implications for grid integration: Towards carbon neutrality
Dazhi Yanga,∗
, Wenting Wanga, Christian A. Gueymardb, Tao Hongc, Jan Kleissld, Jing Huange, Marc J. Perezf,
Richard Perezg, Jamie M. Brighth, Xiang’ao Xiai, Dennis van der Meerj, Ian Marius Petersk
aSchool of Electrical Engineering and Automation, Harbin Institute of Technology, Harbin, Heilongjiang, China
bSolar Consulting Services, Colebrook, NH, USA
cDepartment of Systems Engineering and Engineering Management, University of North Carolina at Charlotte, Charlotte, NC, USA
dCenter for Energy Research, Department of Mechanical and Aerospace Engineering, University of California, San Diego, CA, USA
eClean Power Research, Kirkland, WA, USA
fClean Power Research, Napa, CA, USA
gAtmospheric Sciences Research Center, University at Albany, SUNY, Albany, NY, USA
hUK Power Networks, London, UK
iKey Laboratory for Middle Atmosphere and Global Environment Observation (LAGEO), Institute of Atmospheric Physics, Chinese Academy of
Sciences, Beijing, China
jMINES ParisTech, PSL Research University, Sophia Antipolis, France
kForschungszentrum Jülich, IEK 11, Helmholtz Institute Erlangen Nuremberg for Renewable Energies (HI ERN), Erlangen, Germany
Abstract
The ability to forecast solar irradiance plays an indispensable role in solar power forecasting, which constitutes an
essential step in planning and operating power systems under high penetration of solar power generation. Since solar
radiation is an atmospheric process, solar irradiance forecasting, and thus solar power forecasting, can benefit from
the participation of atmospheric scientists. In this review, the two fields, namely, atmospheric science and power
system engineering are jointly discussed with respect to how solar forecasting plays a part. Firstly, the state of affairs
in solar forecasting is elaborated; some common misconceptions are clarified; and salient features of solar irradiance
are explained. Next, five technical aspects of solar forecasting: (1) base forecasting methods, (2) post-processing,
(3) irradiance-to-power conversion, (4) verification, and (5) grid-side implications, are reviewed. Following that, ten
potential research topics for atmospheric scientists are enumerated; they are related to (1) data and tools, (2) numerical
weather prediction, (3) forecast downscaling, (4) large eddy simulation, (5) dimming and brightening, (6) aerosols,
(7) spatial forecast verification, (8) multivariate probabilistic forecast verification, (9) predictability, and (10) extreme
weather events. Last but not least, a pathway towards ultra-high PV penetration is laid out, based on a recently
proposed concept of firm generation and forecasting. It is concluded that the collaboration between the atmospheric
science community and power engineering community is necessary if we are to further increase the solar penetration
while maintaining the stability and reliability of the power grid, and to achieve carbon neutrality in the long run.
Highlights
•Common pitfalls of solar forecasting research are given, with remedies proposed.
•Five technical aspects of solar forecasting are discussed, with a special focus on grid-integration implications.
•Ten potential research topics for atmospheric scientists are identified.
•A pathway towards ultra-high PV penetration is laid out.
1
Keywords: Review; Solar forecasting; Atmospheric Sciences; Power systems; Grid integration; Carbon neutrality
Word count: 17030.
1. Background
The pursuit of carbon neutrality has dramatically intensified over the past decade, and the range of communities and
scientists who are involved in this pursuit is unprecedentedly broad. Over the last few years, carbon neutrality has
rapidly entered the sphere of concern of the general public. Indeed, the discussions of achieving carbon neutrality
among the public have been commonplace since the 2015 Paris Agreement on climate change, in particular. Never-
theless, it is necessary to examine various issues scientifically in order to really limit the temperature increase to 1.5◦C
above pre-industrial levels by the end of this century. That is the goal put forward by the Intergovernmental Panel on
Climate Change (IPCC) as the most desirable outcome, under the stringent “very low greenhouse gas emissions and
CO2emissions declining to net zero around or after 2050, followed by varying levels of net negative CO2emissions”
scenario, or SSP1-1.9 for short [1]. This goal can only be achieved with radical and rapid changes to the forms of the
world’s energy generation and consumption.
Given the fact that climate change impacts mankind as a whole, everybody needs to play a part. Meanwhile, the
more parties are involved, the more difficult it is to reach agreements. To remedy this situation, policy makers and
scientists have a place at the forefront of tackling climate change. For instance, on July 14, 2021, the European
Commission passed a series of legislative proposals in regard to carbon neutrality. The grand plan to achieve carbon
neutrality in the European Union (EU) by 2050 has been set forth, which includes an intermediate target of reducing
emissions by 55% by 2030.1China has also pledged its plan of action. On March 15, 2021, China’s leadership has
promulgated the directive of “constructing a modern power system predominated by renewable energy” during the
Ninth Meeting of the Central Finance and Economics Committee, as a pathway to reach carbon peak by 2030 and to
achieve carbon neutrality by 2060—an ambitious program known as the “double-carbon objective” or “3060 target”
of China.2Besides EU and China, more than 110 countries have all committed to climate actions of various sorts.
The directives for low-carbon electricity generation consist of two closely interconnected parts: renewable energy
and power systems. The renewable energy part consists of generating a sufficient amount of electrical power through
carbon-neutral or carbon-negative technologies to serve the increasing energy demand. The power system part is
concerned with the delivery and consumption (including storage) of that power, which requires upgrading of the
current system in terms of both infrastructure and automation & control technology. Particularly, power system
planning and operations, which can be segregated by time scale, are key to the seamless combination of the two parts
of the low-carbon regime. Both planning and operations require information about the future renewable generation,
electricity price, and electric load. Hence, resource assessment and forecasting are highly relevant spheres of research.
Whereas resource assessment deals with the decade-long projection of energy supply, or demand potential, with the
∗Corresponding author. Tel.: +65 9159 0888.
Email address: yangdazhi.nus@gmail.com (Dazhi Yang)
Abbreviations: AGC, automatic generation control; AOD, aerosol optical depth; CAISO, California Independent System Operator; CAMS,
Copernicus Atmosphere Monitoring Service; CCN, cloud condensation nuclei; CMIP, Coupled Model Intercomparison Project Phase 5; CSP,
concentrating solar power; DNI, direct normal irradiance; ECMWF, European Centre for Medium-Range Weather Forecast; GBD, global dimming
and brightening; GHI, global horizontal irradiance; GEOS-5, Goddard Earth Observing System Version 5; GPI, graphical processing unit; GTI,
global tilted irradiance; HRRR, High Resolution Rapid Refresh; IFS, Integrated Forecasting System; IPCC, Intergovernmental Panel on Climate
Change; LES, large eddy simulation; MAE, mean absolute error; MAPE, mean absolute percentage error; MYNN, Mellor-Yamada Nakanishi
and Nino; NAM, North American Mesoscale; NASA, National Aeronautics and Space Administration; NOAA, National Oceanic and Atmospheric
Administration; NWP, numerical weather prediction; PBL, planetary boundary layer; PLF, probabilistic load flow; PMF, perfect forecast metric; PV,
photovoltaic; RAP, Rapid Refresh; RMSE, root mean square error; S2S, sub-seasonal to seasonal; SDS-WAS, WorldMeteorological Organization’s
Sand and Dust Storm Warning dvisory and Assessment System; T&D, transmission and distribution; WRF, Weather Research and Forecasting;
1https://www.cnbc.com/2021/07/14/whats-the- eu-plan- to-achieve- carbon-neutrality.html
2http://www.xinhuanet.com/politics/leaders/2021-03/15/c_1127214324.htm
Preprint submitted to Renewable &Sustainable Energy Reviews February 7, 2022
objective to understand the techno-economic viability of future power systems and to minimize the long-term risks,
forecasting makes predictions up to a few days ahead with the objective of scheduling various generation sources to
meet the changing demand and to mitigate the short-term variability. In fact, both topics here belong to the field of
energy meteorology, which seeks to serve the energy sector better with climatological and weather information and
knowledge by developing new scientific tools and models. At this stage, the motivation for bridging atmospheric
sciences and power system engineering should have been made clear.
2. Solar forecasting: An iconic area of research in energy meteorology
The central goal of this review is to promote interdisciplinary collaborations between atmospheric sciences and power
system engineering communities. Indeed, there is a wide spectrum of topics on which atmospheric scientists and
power system engineers can collaborate, and a few examples are listed next. First, extreme weather events, such as
prolonged heat waves or cold spells, would alter the electricity usage patterns, as well as the electricity generation
capacity [2]; early warnings and preparations for these extreme weather events are thus necessary to mitigate if not
avoid overburdening the power system. Second, remote-sensed surface solar irradiance retrievals are able to provide
long-term wide-area support for solar energy system design, simulation and performance evaluation, at locations
where ground-based radiometry is unavailable [3]. Third, temperature, as an essential meteorological variable, has a
profound impact on the electric load, as well as on utility-scale solar photovoltaic (PV) and wind power generation
[4]. Hence, obtaining long-term temperature projections under various warming scenarios is an absolute necessity
to systematically assess the renewable energy potential over the coming decades, during which a massive energy
transition is expected to take place.3While the aforementioned topics are of critical importance, energy forecasting,
particularly for solar and wind, is another area of research in energy meteorology [6].
Currently, solar forecasting and wind forecasting account for two of the four major areas of energy forecasting, with
electric load forecasting and electricity price forecasting being the other two. As mentioned earlier, operational energy
forecasting aims at estimating a future value of the quantity of interest, such as solar irradiance, wind speed, electric
load, or electricity price, over a horizon ranging from a few seconds to a few days [6].4The need for these energy
forecasts arises, primarily, to accommodate the various requirements and regulations affecting power system control
and operations. Price forecasting over various horizons is required for demand response and energy trading. Day-
ahead load, solar, and wind power forecasts, on the other hand, are necessary to predict the net load for the next
operating day, such that the power system operators could perform day-ahead unit commitment of thermal generators,
allowing them to optimally schedule the generators and balance the next-day electricity demand at a minimum cost,
while meeting all plant and system constraints [8]. In real-time markets, forecasts at a 5-, 15-, or 30-min resolution
out to several hours are needed for intra-day unit commitment and economic dispatch, which updates the day-ahead
schedules with the latest information, such that the coordination between renewables and conventional generation
can be further optimized [9]. Sub-minute solar and wind power forecasts, on the other hand, are required during
secondary frequency control of power systems—particularly the automatic generation control (AGC), in which the
power provided by pre-dispatched conventional thermal generators may not follow tightly enough the frequent and
steep ramps introduced by variable renewable generation [10,11]. Given the fact that both solar irradiance and wind
speed result from physical processes in the atmosphere, atmospheric science is bound to play a key role in advancing
solar and wind forecasting.
Over the last decade, there have been substantial skill improvements in wind speed forecasts [12]. As a consequence,
wind power forecasting, with the help of practical techniques for mapping wind speed to wind power (i.e., wind power
curve modeling), has been thriving [13]. Cloudiness, on the other hand, remains one of the most challenging phe-
nomena to forecast, particularly in terms of amount, depth, location (vertically and horizontally), and timing. Because
3Since various global climate change scenarios are still possible, an important question is how surface solar radiation and the solar resource of
various regions will vary in the coming decades as a result of potential variations in aerosol loading, atmospheric water vapor content, as well as
cloud and precipitation patterns. These variations and their effects are widely studied based on a number of climate models, including the Coupled
Model Intercomparison Project Phase 5 (CMIP5) and the latest CMIP6 model simulations [5].
4Forecasting with longer lead times is also needed in some energy areas, e.g., natural gas markets [7], but has limited relevance for grid
operations.
3
clouds constitute a principal influencing factor in determining the amount of surface solar radiation, solar forecast is
directly impaired by any inaccuracy in cloudiness forecasts. Moreover, weather models internally only require the
global horizontal irradiance (GHI) for the models’ energy budget, whereas the direct normal irradiance (DNI) and
diffuse horizontal irradiance (DIF) components, which are both needed for irradiance-to-solar-power conversion, are
not commonly provided to the user [14]. In this respect, it appears that solar forecasting would call for more atten-
tion in the near future, especially because the grid penetration level of solar power is expected to grow rapidly and
significantly, as a key component of the envisioned decarbonized society. Therefore, this review focuses on solar
forecasting, which now appears as an iconic area of energy meteorology that, just like wind forecasting, can benefit
from a strong collaboration between atmospheric sciences and power system engineering.
It should be noted that the classes of problems concerning atmospheric scientists and power system engineers, as
related to solar forecasting, are profoundly different. Take cloud prediction for instance: On the one hand, the former
group would be interested in clouds because they present challenges to sub-grid transfer of mass and heat, which is
confounded with microphysical and dynamical processes [14,15]. On the other hand, the latter group views clouds as
the main source of fluctuations in solar power generation, which can cause voltage variability on distribution feeders
and frequency variability in transmission systems [16–18]. It is thought that two domain-specific terms, such as
“boundary-layer meteorology” or “distribution feeders,” and their connection to weather, would be unquestionably
familiar to researchers in each domain, but not to those in the other domain. However, if fruitful collaborations are to
actually take place, it would be necessary to understand the terms and jargon in each other’s field. For this reason, the
important terms that are thought to be unfamiliar to atmospheric scientists are explained in footnotes throughout the
remaining part of this review.
A brief outline of the review is as follows. First and foremost, the state of affairs in the field of solar forecasting is
summarized in Section 3. The emphasis of that part is on the positioning of solar forecasting within the realm of
forecasting science. Three important questions addressed are: (1) what is the current maturity of solar forecasting, (2)
what has been overlooked by electrical engineers when performing solar forecasting, and (3) what distinguishes solar
forecasting from other forecasting tasks? Section 4takes a more technical look at solar forecasting. In that section, it
shall be made clear that numerical weather prediction, though vital, is but one class of solar forecasting methods, and
forecasting methods constitute but a fraction of research issues to be addressed in the field of solar forecasting. The
most important aspect of this review is presented in Section 5, that is, a list of potential areas of solar forecasting to
which atmospheric scientists are able to contribute. Section 6discusses two recently proposed concepts called “firm
generation” and “firm forecasting,” which can potentially contribute to transforming the present grid into one that has
an ultra-high solar penetration, as to meet our energy demand 24/365 with renewables. An outlook follows at the end.
3. Existing state of affairs of solar forecasting
Forecasting, as a scientific domain, has accumulated a collection of theories and knowledge that can be regarded
as general. For instance, the book by Armstrong [19] gathers many basic principles of forecasting, which are well
known to be beneficial for general-purpose forecasting when followed. Nevertheless, if one is to understand how solar
forecasting is distinct from other forecasting domains, knowing the current status of solar forecasting is a prerequisite,
and thus constitutes an essential motivation of this review.
3.1. Is solar forecasting an under-developed domain of energy forecasting?
Solar forecasting, as compared to other energy forecasting topics, has a much shorter history [20], and was even
described as the most immature energy forecasting domain by world-renowned energy forecasters only five years ago
[21]. Nevertheless, if one judges the level of maturity by the amount of publications and by their scientific impact
(e.g., via citations), it would not be surprising if the outcome is found to be contradictory to that earlier conclusion—
solar forecasting has experienced the fastest rate of expansion over the last decade, and has overtaken electricity price
forecasting in terms of publication number and citations [6]. The reason behind this observed disagreement has been
investigated by Yang [22], and was found to be two-fold.
Firstly, there was evidently a disconnect between solar forecasting and the other energy forecasting domains prior to
2018. Owing to the fact that, until then, the worldwide grid penetration level of solar energy had been low as compared
4
to that of wind energy [21], issues such as power grid stability or reliability caused by variable solar power injection
were only a minor concern. The attention that was given to solar forecasting by the energy forecasting community was
therefore marginal. For the top-tier power system, management science, and forecasting journals, the amount of solar
forecasting works was, and still is, only a small fraction of that of load, price, or wind forecasting [6]. In contrast,
the bulk of solar forecasting publications can be found in energy journals, which have a very different readership
than those aforementioned journals [20]. Additionally, it can be construed that those who work on solar forecasting
have heterogeneous backgrounds, whereas those who perform load forecasting are mostly electrical engineers, while
researchers from business and management sciences perform price forecasting, and statisticians and meteorologists
perform wind forecasting.5In summary, solar forecasting lacks an iconic community that can be classified into a
traditional subject of science or engineering. Since the body of forecasting knowledge has grown so much that a
lifetime is not sufficient to reach the frontiers of all forecasting areas,6it is not uncommon for researchers outside
solar forecasting to draw conclusions with a preconceived and restricted perspective.
Secondly, the current wave of development of solar forecasting started in the early 2010s, by which time load and
wind forecasting had already gained much attention. Furthermore, the past decade has also witnessed the rapid
popularization of statistical and machine-learning software environments, such as R or Python. Hence, anyone who
is interested in solar forecasting can easily find existing methodology and tools to support scientific investigation of
various kinds. One inevitable consequence of this is the large number of solar forecasting papers that use highly
replaceable statistical or machine-learning methods. A phenomenon, which is still seen in many journals, is that a
forecaster can simply apply a method, or a hybrid of several methods, and get published, as long as that method or
the hybrid has not been previously used for solar forecasting purposes; this pattern of “making scientific contribution
by exaggerating the novelty” was also seen in the load forecasting literature, but at an earlier time [23]. This situation
has led to coining the term “solar forecasting bubble” by Yang [22], analogous to an economic bubble. The solar
forecasting bubble is therefore thought to be the second reason why the domain was thought to be immature. In any
case, it seems attractive to investigate the nature of solar forecasting, and thus to identify whether there is any scientific
or engineering community who can take the leading role in this line of research.
3.2. Does solar forecasting belong to electrical engineering?
Grid-tied PV systems,7which constitute a major type of solar energy conversion technology, no doubt fall well within
the realm of electrical engineering. For that reason, many stakeholders deeply believe that solar forecasting, like
load forecasting, should be handled primarily by electrical engineers. In this context, it is useful to recall how load
forecasting is approached. A shortcut to grasp the essence of load forecasting is to study, in chronological order,
the reviews by Gross and Galiana [24], Hippert et al. [25], Hong and Fan [23], and Hong et al. [26]. Through these
reviews, one can easily notice that a core concept in load forecasting for horizons up to a few days is to include
forecasts of weather variables such as ambient temperature as exogenous variables, since they significantly affect the
load profile. Hence, various regression techniques have been used to capture the relationships between those weather
variables and electricity demand [27,28]. Naturally, when the same group of researchers are tasked to forecast the
electricity output from PV systems or concentrating solar power (CSP) systems, they have the tendency to regress that
output on a collection of weather variables, such as GHI8DNI,9air temperature, or wind speed. Several entries took
this regression-based approach in the solar forecasting track of Global Energy Forecasting Competition 2014 [21].
From a general standpoint, this approach is impacted by three main limitations.
The first limitation originates from the difference in forecast accuracies of those explanatory variables used for load
forecasting and those used for solar forecasting. The electric load (dependent variable) is known to be a highly
5In a recent review by Hong et al. [6], journals with the most publications in each energy forecasting domain are analyzed, e.g., load forecasting
is often published in IEEE journals or price forecasting is often published in economics journals, which give rise to this statement.
6Or even worse, one may not even be aware of the existence of certain areas that could be potentially complementary.
7A grid-tied PV system consists of solar panels, inverters, an (optional) power conditioning unit, grid connection equipment, and (optional)
energy storage. The power output of the system is directly injected into the grid. In contrast, stand-alone systems, also known as off-grid PV
systems, do not inject any energy to the grid.
8GHI is a term widely used in solar energy meteorology, whereas equivalent terms in atmospheric sciences are surface solar irradiance or
downwelling shortwave radiation at the surface.
9The beam component of GHI, typically measured by a pyrheliometer on a plane normal to the direction of propagation of sunbeams.
5
dependent on weather and calendar (independent variables). While the values of most calendar variables are known,
one has to forecast weather variables. Forecasts of temperature and humidity are far more accurate than those of solar
irradiance. The reason is that in numerical weather prediction (NWP) models, temperature and humidity forecasts are
subject to continuity equations thus having inertia whilst solar irradiance forecasts depend heavily on cloud modeling
which is stochastic in nature. Therefore, an ex post load forecast is expected to be close to the ex ante load forecast
generated by the same load forecasting model. In contrast, an ex post solar power forecast generated by a regression
model with GHI variables could be substantially different from its ex ante counterpart due to the large error in GHI
forecasts. Therefore, solar forecasting introduces a different class of problem than load forecasting. Solar irradiance
forecast errors are generally larger and more widely distributed, so that significant work remains to be done to improve
forecasts of the regressors.
The second limitation pertains to the question “is it appropriate to input NWP-based or remote-sensed weather vari-
ables directly into a PV power forecasting model?”, to which the short answer is “no.” The reason is that the vast
majority of NWP forecasts do not provide the variables needed to operate a complete PV power model. (The con-
cept of “model chain,” which is necessary to perform accurate PV energy simulations, is further explained below in
Section 4.3.) One important justification for this answer is linked to what drives a solar system—the incident solar
irradiance. To maximize their annual energy yield, it is customary to install solar collectors on a near-equator-facing
tilted surface with an angle comparable to the latitude of the site, or on motorized solar trackers. Because of these ge-
ometries, the radiation quantity relevant to non-concentrating solar power is the global tilted irradiance (GTI), which
is neither available from NWP nor remote sensing. In solar energy meteorology, it is customary to obtain GTI from
GHI using transposition models, such as the Perez model [29], which have attracted considerable interest [see 30,
for review]. The best such models account for the anisotropy of the sky diffuse radiance, which can be challenging
to do with just machine-learning methods, although some recent studies attempt to do just that with apparently good
success [e.g., 31]. In any case, it is argued here that, it is generally necessary to base PV power forecasts on GTI
rather than GHI.10 Hence, to optimally utilize weather variables as input to a PV forecasting model, one must possess
certain knowledge of atmospheric sciences and solar radiation modeling, which is generally lacking among electrical
engineers.
The third limitation of the engineering-style forecasting approaches is related to the restricted explanatory power
of local meteorological measurements. It is true that one can install a pyranometer on a fixed or variable tilt to
measure GTI locally, but multiple-step-ahead forecasting by extrapolation is deficient because the uncertainty of the
forecast increases drastically with the forecast horizon. In fact, forecasting using data from locally installed sensors
is currently unable to produce any decent result for horizons beyond a few steps [12]. Moreover, point-location
irradiance observations or predictions are unable to capture the spatio-temporal dynamics of clouds, which has a
decisive effect on the power output of a PV system. Since grid integration of solar energy always requires multiple-
step-ahead forecasting, may it be in an intra-hour, intra-day, or day-ahead scenario [32], purely regression-based
single-location data-driven models have limited applicability in practice [33]. Almost all weather processes are driven
by spatial gradients (e.g., wind is a function of the pressure gradient), weather processes are highly nonlinear, and
local measurements tend to decorrelate very quickly with time or distance. Therefore, physics-based approaches are
absolutely necessary to produce truly meaningful large-scale forecasts. Physics-based weather forecasting, consisting
of NWP and advection of cloud or irradiance fields derived from geostationary satellite or ground-based sky camera
imagery, has hitherto been a major focus of atmospheric scientists, who should consequently take a leading role in
solar forecasting.
3.3. Salient features of solar irradiance
Aside from the above reasons, more arguments can be advanced in terms of why solar forecasting, as a topic, should
rightfully include participation of atmospheric scientists. This is done by analyzing the salient features of solar
irradiance, which can be consolidated into three main aspects: (1) double-seasonal pattern, (2) spatio-temporal nature,
and (3) probabilistic representation.
10The argument here provokes a more general question: “whether or not variable transformation is useful during forecasting?” Clearly, the
answer to this question is problem-specific. But in the case of PV power forecasting, at least at this date, almost all solar engineers would agree
that converting GHI to GTI (or more specifically, effective irradiance) is needed for PV power estimation.
6
GOES
Meteosat
Himawari
500 1000 1500 2000 2500
Annual GHI [kWh/m2]
Figure 1: Several geostationary weather satellites jointly cover all locations of the Earth between ±60◦latitudes, providing decade-long gridded
satellite-derived irradiance data. Data source: National Solar Radiation Data Base, https://maps.nrel.gov/nsrdb- viewer/.
Solar irradiance—thus solar power—time series exhibit double-seasonal patterns (a yearly cycle and a diurnal cycle),
owing to the apparent movement of the Sun. It is a well-known forecasting principle that when seasonality is present,
one ought to remove it prior to forecasting, or to model it as a separate component during forecasting [see pp. 224, 249,
337 of 19]. On this point, the best way of removing the double-seasonal pattern is through a clear-sky radiation model
[22,34,35], which describes the amount of radiation reaching the Earth’s surface under a cloud-free atmosphere [36].
A physical clear-sky model often uses a reduced form of radiative transfer, such as via empirically fitted transmittances
or a look-up table, which can achieve an accuracy unparalleled by any data-driven alternative [37]. Semi-physical
clear-sky models, such as the REST2 model [38] or the McClear model [39], have thus found their place in augmenting
NWP models [e.g., 14,40] and deriving irradiance from satellite images [e.g., 41,42]. The ratio between the actual
irradiance and its clear-sky expectation is known as the clear-sky index,11 which is a deseasonalized quantity that is
advantageously used in the construction of solar forecasting models. Since radiative transfer is the principal aspect of
clear-sky radiation modeling, the latter belongs to atmospheric sciences, even though solar applications at the Earth’s
surface might not need all the complexity of physics-based atmospheric models, which need to provide spectral details
for many layers over the whole vertical column. That is why semi-physical solar radiation models are normally
preferred in solar resource modeling or forecasting because they are substantially faster and easier to use.
The next important salient feature of solar irradiance is the spatio-temporal nature of the surface radiation process. As
argued in Section 3.2, the inability of detecting incoming clouds, which can drop the irradiance by hundreds of W/m2
(equivalent to tens of percent) in a few seconds, limits the forecast quality greatly. Therefore, one easy way to pick
11Clear-sky index differs from clearness index; the latter refers to the ratio between GHI and its extraterrestrial counterpart. In solar forecasting,
the clear-sky index is almost always preferred.
7
out advanced solar forecasting models is to check whether spatio-temporal information is integrated into forecasting
[22]. There are three main methods to attain spatio-temporal information in solar forecasting: ground-based all-sky
imagers or cameras, instruments onboard satellites, and NWP [34,43]. In fact, the utilization of remote-sensing
data is one of those aspects of solar forecasting that is more advanced than in wind forecasting (pers. comm. Pierre
Pinson). Geostationary satellites jointly provide a complete coverage of all locations at latitudes between ±60◦, and
satellite-derived irradiance products are available at hourly or sub-hourly resolution and moderate (typically 3–4 km)
spatial resolution, see Fig. 1. Besides sky camera, satellite and NWP, sensor networks can also be used, but are limited
by construct, in terms of capturing the cloud dynamics beyond the spatial scale of the network. To that end, remote-
sensing of the atmosphere and weather forecasting are again topics of atmospheric sciences. It is, however, worth
mentioning that whenever spatio-temporal correlation is investigated during solar forecasting, it ought to be applied
to the clear-sky index. This is because correlation coefficient calculated based on GHI would be inflated due to the
aforementioned double-seasonal pattern, which is unable to quantify the actual correlation caused by moving clouds.
On the other hand, the clear-sky index is largely related to the behavior of clouds, such as their spatial distribution,
cloud cover fraction, strength of convection, or velocity of advection, which constitutes the main source of irradiance
variability [44].
Last but not least, it should be noted that the best climate or weather forecast is necessarily probabilistic [45,46].
Solar irradiance, like many other atmospheric variables, is five-dimensional by nature, spanning three-dimensional
space, time, and probability. There is thus a strong desire for probabilistic solar forecasting. The weather community
has much to say on probabilistic forecasting because ensemble forecasting, as a strong defining feature, has become
so well established in the weather domain, as well as in any domain where uncertainty quantification is of interest
[47,48]. There is thus little need to elaborate this point any further.
4. Five technical aspects of solar forecasting: An overview
So far, most reviews of solar forecasting have focused on individual forecasting techniques, which are referred to
here as base methods. Despite the importance of base methods, there are other considerations that should not be
overlooked. In this section, a bird’s eye view of the field of solar forecasting is presented, which covers most, if not
all, of its multifaceted aspects. For an overview, the five technical aspects of solar forecasting are summarized in
Fig. 2.
Five aspects of solar forecasting
Base methods Post-processing Irradiance-
to-power
conversion
Forecast
verification Grid integration
Camera
Satellite
NWP
Statistics
Machine learning
Deterministic
to deterministic
Deterministic
to probabilistic
Probabilistic to
deterministic
Probabilistic
to probabilistic
Solar power
curve
Radiation
modeling
Solar energy
system modeling
Reference
methods
Skill score
Materialization
of value
Regulation and
load following
System sta-
bility control
Transmission
and distribution
planning
Figure 2: Five technical aspects of solar forecasting and the main concepts associated with each aspect.
8
4.1. Three classes of base methods
Base solar forecasting methods have been categorized, on the basis of forecast horizon and the most relevant exoge-
nous data, into three types. There are those methods based on sky cameras12 for sub-15-min forecasting [e.g., 49–51],
those based on satellite data for intra-day forecasting [e.g., 52–54], and those based on NWP for day-ahead forecasting
[e.g., 55–57]. Aside from the three main types, microscale sensor networks are also used for sub-minute forecasting
[e.g., 58–60]. Last but not least, myriads of statistical and machine-learning methods, which are applicable to all
regression (or classification) problems, have also been widely applied to solar forecasting. Whereas their success is
limited whenever physics is completely disregarded [22], they are frequently combined with NWP outputs to improve
results through post-processing or blending [61,62].
100101102103104105106
10−4
10−3
10−2
10−1
100
Statistical learning
extrapolation
1 s–1 mon
1 m–2 km
Sky camera
3–30 min
1 m–2 km
Local sensor
network
20 s–3 min
1 m–1 km
Satellite
30 min–6 h
1–10 km NWP
4–36 h
5–20 km
Forecast horizon [s]
Spatial resolution [m−1]
100101102103104105106
10−4
10−3
10−2
10−1
100
Statistical learning
as augmentation
and post-processing
tools only
Very-high-resolution
NWP models
Forecast horizon [s]
Spatial resolution [m−1]
Figure 3: (top) A legacy version of the perceived association among time horizon, spatial resolution, and solar forecasting approaches, as proposed
by Inman et al. [63]. Currently, with the accumulation of new knowledge and facts, this stratification of solar forecasting approaches is becoming
increasingly outdated. (bottom) The current perception of the spatio-temporal scale of different forecasting methods. Dotted ellipse indicates that
statistical learning now only acts as augmentation and post-processing tools, rather than a stand-alone category. Very-high-resolution NWPs, such
as the Firecaster model from the Università di Corsica, using Météo-France’s AROME initial and boundary conditions, are able to reach a spatial
resolution of 200 m on 60 levels, and a temporal granularity of 2 min (pers. comm. with Jean-Baptiste Filippi).
The classification just described (and illustrated in Fig. 3) can be traced back to Inman et al. [63], who presented
the first major review on solar forecasting; it is still taken as the standard by most solar forecasters, as evidenced
12Sky cameras, also known as all-sky imagers, consist of a sky-viewing fish-eye lens camera. Recently, the more affordable security cameras
have been popularized, with the trade-offof needing more sophisticated calibration for image distortion.
9
by a recent article that included a dataset for solar forecast benchmarking [43]. Nevertheless, as the spatio-temporal
resolution of satellite- and NWP-based data becomes increasingly higher, such mapping between forecast horizon and
the type of exogenous data that is deemed most useful is subject to questioning. Indeed, there seems to be very little
reason for not leveraging hourly updated NWP models [64,65] or microscale satellite data [66,67] for intra-day or
intra-hour solar forecasting. As the scale of data decreases further, solar forecasters would no longer be restricted by a
single form of exogenous data. In principle, all possible information able to explain the future state of the atmosphere
should be considered collectively. Particularly true for satellite-derived radiation data is that there are always multiple
products available for the same location [68–70]. Hence, using these autonomous data sources increases accuracy,
while also enabling uncertainty quantification by means of ensemble modeling.
4.2. Post-processing
Owing to measurement uncertainties, as well as to the insufficient granularity in modeling, the output of base fore-
casting methods, particularly those from NWP models, are often found to be biased [71,72] or incorrectly dispersed
[73,74]. Therefore, post-processing is viewed as a vital step during solar forecasting. At a minimum, one should
verify whether or not post-processed forecasts can outperform the raw ones. The most valuable weather forecasts are
necessarily probabilistic [46]. However, some methods, such as camera-based forecasting, usually produce only de-
terministic forecasts. Whereas probabilistic forecasts are often not yet required in forecast submissions to power grid
operators, a majority of them are already, or in the process of, exploring probabilistic operations. Solar forecasters
should, therefore, develop the capability to easily convert from deterministic to probabilistic forecasts.
On this point, Yang and van der Meer [61] advocated to divide post-processing into four categories in accordance
with the direction of forecast conversion: (1) deterministic to deterministic (D2D), (2) deterministic to probabilistic
(D2P), (3) probabilistic to deterministic (P2D), and (4) probabilistic to probabilistic (P2P) post-processing. They also
argued that forecasters should not be distracted by the myriads of fancy post-processing techniques now available.
Instead, it is the underlying styles or mechanisms of post-processing that matter—Yang and van der Meer [61] termed
these styles or mechanisms as “thinking tools.” The reader is referred to that review, which is possibly the most
complete document to date on post-processing of solar forecasts, for more details. Another notable compendium of
post-processing techniques is the recent book by Vannitsem et al. [75], although it is not specific to solar applications.
4.3. Irradiance-to-power conversion
The relationship between wind speed and wind power is known as the wind power curve [13]. Analogously, the
relationship between GHI and PV power ought to be referred to as the solar power curve, although such a term
has yet to receive any major acceptance. As shown in Fig. 4, the mapping between the main atmospheric variables
and the power are not monotonic, as a single wind speed (or GHI) value corresponds to a range of power values.
Furthermore, even if additional variables, such as temperature or humidity, are involved, they do not necessarily
improve the explanation of the variation in power, see Fig. 5. This demonstrates why wind- and irradiance-to-power
conversion is particularly challenging. In a statistical sense, knowing the cause of deviations from the conditional
mean values (i.e., conditional on wind speed or GHI) is what solar forecasters desire; the reader is referred to Lee
et al. [76] for more discussions on the statistical aspects of power curve estimation.
The literature on irradiance-to-power conversion is already vast and scattered, but it can be broadly categorized into
two schools. There are those methods that take a direct (i.e., one-step data-driven) approach, as opposed to the
indirect (i.e., multi-step physical and semi-empirical) ones [77]. Direct approaches regress the previous PV power
based on weather variable inputs, and once a statistical relationship is established, future weather inputs can use it to
estimate future PV power. In contrast, the indirect approaches sequentially leverage several models, each addressing
a particular step of conversion, including but not limited to separation model,13 transposition model, temperature
13Separation models empirically split the beam and diffuse components from GHI, which typically leads to significant errors in the split com-
ponents. The best approach is to use a physical model, similar to that used for GHI, to derive diffuse and beam components from remote-sensing
data, but this avenue is rarely followed because of its complexity.
10
0.00
0.25
0.50
0.75
1.00
0 5 10 15
Wind speed [m/s]
Normalized wind power
0.00
0.25
0.50
0.75
1.00
0 300 600 900 1200
Global horizontal irradiance [W/m2]
Normalized solar power
Figure 4: (top) A typical wind power curve, which describes the relationship between wind speed and wind power. (bottom) A typical solar power
curve, which describes the relationship between global horizontal irradiance and PV power. The color in this figure denotes the density of points.
Data source: Hong et al. [21].
0 200 400 600 800 1000 1200
0.0 0.2 0.4 0.6 0.8 1.0
−5
0 510
15
20
25
30
35
Global horizontal irradiance [W/m2]
Temperature @ 2 m [°C]
Normalized solar power
Figure 5: Same as Fig. 4(bottom), but with 2-m temperature as an additional explanatory variable. Data source: Hong et al. [21].
model,14 DC model,15 inverter model,16 and loss model.17 These models jointly form a model chain, as shown in
14A temperature model estimates the PV cell temperature from air temperature, wind speed, and GTI.
15A DC model estimates the DC power from GTI and PV cell temperature. This is done by augmenting the rated power under standard test
condition, namely, 1000 W/m2and 25◦C, with the actual operating condition, i.e., actual GTI and PV cell temperature.
16An inverter model converts DC power and voltage to AC power using the real characteristics and efficiency of inverters. Naturally, inverter
models depend on the brand and model of the inverter.
17A loss model accounts for the loss in power due to various non-idealities, such as soiling, shading, snow, mismatch, or wiring losses.
11
Fig. 6. As argued in Section 3.2, the direct approaches are suboptimal as compared to the model-chain approach, due
to the fact that they completely ignore physical principles that govern energy conversion.
Input:
Model chain:
Output:
Date and location GHI Temperature Wind speed
Solar positioning
Separation
Cell temperature
Transposition
Reflection
PV module
Shading losses
Soiling losses
DC cable losses
Inverter losses
AC cable losses
Transformer
PV power output
Figure 6: Schematic of irradiance-to-power conversion via model chain. A model chain takes global horizontal irradiance as the main input and
outputs PV power. Arrow(s) going into a block indicates the required input, whereas arrow(s) leaving the block indicates the output.
The difficulty of leveraging a model chain during irradiance-to-power conversion is that it requires significant do-
main knowledge, which can be overwhelming for anyone who is not already familiar with solar energy conversion.
Additionally, for each stage within a model chain, there are many options available. Take GHI-to-GTI transposition
modeling for instance: There are more than 25 commonly used empirical models. With regard to clear-sky radiation
models and separation models, options are even far more numerous. Fortunately, several comprehensive, worldwide
comparisons of transposition, separation, and clear-sky radiation models exist [30,78–82], which, to a large extent,
help to narrow down the choices of best models. Nonetheless, since an error in one stage of the model chain prop-
agates to the next and interacts with the latter’s error (resulting in either compensation or amplification of errors), it
is unclear whether simple concatenation of the best model at each stage would lead to an eventual best performance.
For instance, using high-quality irradiance observations from the very few sites where separation and transposition
models can be tested, it has been shown that the combination of the best separation model and the best transposition
model is not necessarily optimal [83–85]. More generally, testing a large collection of complete model chains seems
unavoidable and critical, but has not attracted much interest so far, with a few exceptions [e.g., 83,86–88].
Fortunately, the pvlib package in Python18 has good prospect of becoming a universal tool for model-chain applica-
tions. The pvlib package offers several basic models for each stage of the model chain, particularly classical models.
For anyone who wishes to move from irradiance forecast to PV power forecast, mastering this package is highly
beneficial. Nonetheless, this software ought to be used with appropriate caveats. In particular, not all the best known
clear-sky radiation models, separation models, or transposition models are coded yet. Moreover, the latest version of
the widely used PVWatts DC model that appears in pvlib was validated using data collected from only small-scale
solar farms (<1 MW). This kind of details can only be known with experience, which takes time to accumulate.
4.4. Verification methodologies
Forecast verification is a topic with rich history in the field of atmospheric sciences. Particularly important are those
pioneering contributions by Murphy [e.g., 89–91], which have led to an ongoing enthusiasm among atmospheric
18https://pvlib- python.readthedocs.io/en/stable/modelchain.html
12
scientists for developing forecast verification methodologies. This effort is still unparalleled by any other domain of
forecasting. A great portion of these developments has been consolidated in the book by Jolliffe and Stephenson [92].
Verifying solar irradiance forecasts, whilst resembling the verification of other meteorological variables, has key dis-
tinctions that must be highlighted. First and foremost is the selection of the most appropriate reference method. For
deterministic solar forecasting, climatology, persistence, and their optimal convex combination are the most popular
choices [93], as is the case in general weather forecasting [94]. Nonetheless, these reference methods need to be first
applied to the clear-sky index and then converted to irradiance or PV power for verification, for the reasons men-
tioned in Section 3.3. In the case of probabilistic forecasting, climatology—that is, the unconditional distribution of
observations—could be used, which is calibrated by construct but lacks sharpness.19 On this account, several alter-
native standards of reference have been proposed in the literature [96–98]. A benchmarking method for multivariate
probabilistic solar irradiance forecasts has also been developed very recently [99], which is relevant when forecasting
the multivariate probability distribution across space and time.
Another point that calls for attention is that not all accuracy measures are adequate when evaluating the quality
of solar forecasts. Given the fact that there may be many near-zero solar irradiance values among the forecast–
verification pairs, relative errors expressed in percent may be extremely high, especially during early mornings and
late afternoons. In this regard, when percentage error is of interest, solar forecasters either divide the root mean square
error by the mean irradiance/PV power, or use the skill score. In addition, it is vital to use multiple measures to verify
various aspects of solar forecasts. For example, it is well known that simple averaging (e.g. in space, time, or from
multiple models) tends to reduce mean errors at the price of sacrificing variability (which may lead to under-dispersed
forecasts), which can be illustrated by a Taylor diagram [e.g. 72].
Lastly, based on the critical statement by Murphy [91] that “forecasts possess no intrinsic value, they acquire value
through their ability to influence decisions made by users of the forecasts,” it can be postulated that the optimal
verification should follow the directive under which solar forecasts are solicited. Most solar forecast users are grid
operators, system owners, and energy traders. In that respect, forecasts should be issued, and the optimality of the
forecasts should be judged, according to the requirements and/or success metrics of the user, such as the forecast sub-
mission standards and grid code put forth by grid operators [see, e.g., 32,100, for reviews on China’s and California’s
grid codes for solar integration, respectively]. In summary, the reader is referred to the reviews by Yang et al. [101]
and Lauret et al. [102] for verification of deterministic and probabilistic solar forecasts, respectively.
4.5. Implications on grid integration
Since the value of solar forecasts cannot be realized unless they are being used, it is worthwhile to examine how exactly
solar forecasting could affect the power grid. At low penetration levels, e.g., a decade ago in the most popular solar
areas, the grid is able to absorb the power, voltage, and frequency disturbances caused by variable solar irradiance.
Nonetheless, to achieve a much higher penetration, the conventional, deterministic and passive ways of operating the
power system must be revised.
The challenges to the power grid brought by high solar penetration may be categorized based on time scale. On a time
scale of up to an hour, the cloud-induced ramps in system-level solar power generation constitute the main challenge
[103–105]. On a daily time scale, the problem is also exemplified by the renowned “duck curve” (see Fig. 7), which
describes the situations in which solar generation peaks at noon but the load peaks in the late afternoon, which on
occasions may create a steep supply–demand peak mismatch [106,107];20 On a longer time scale ranging from weeks
to a year, the weekly and seasonal changes in the amount of solar resource call for major attention, since the current
seasonal energy storage technologies, including solar thermal energy storage [108,109], are mostly experimental and
yet to be cost-effective [110,111]. Among many aspects that could benefit from accurate solar forecasts, this section
focuses on three of them: (1) power system regulation and load following;21 (2) power system stability control (e.g.,
via probabilistic load flow); and (3) transmission and distribution (T&D) planning.
19Probabilistic forecasts are evaluated based on, according to Gneiting et al. [95], “the paradigm of maximizing the sharpness of the predictive
distributions subject to calibration.”
20Note that the duck curve is not always true of all climates and geographies, e.g., regions with large cooling loads tend to have much higher
13
02:00 06:00 10:00 14:00 18:00 22:00
Time of day
Net load [MW]
Year 2010
2011
2012
2013
2014
2015
2016
2017
2018
2019
2020
2021
2022
2023
2024
2025
Figure 7: An illustration of a typical “duck curve.” As solar installations increase over the years, the daytime power from solar gradually causes
larger difference in peak and mid-day net load, which needs to be met by wind or conventional generation. Data source: https://vuo.org/
node/1927.
4.5.1. Regulation and load following
Traditional ancillary services22 are to formulate the power generation plan of each generator according to the results
of load forecasting and market bidding,23 as to maintain the balance between demand and supply. Load forecasting,
scheduling, and dispatch are to be conducted on different time horizons. In a two-settlement market structure, a day-
ahead market allows bidding and scheduling of generators, while a real-time market seeks to clear the discrepancies
between the day-ahead plan and the most-updated load conditions via unit commitment and economic dispatch [9].
In the early days when renewable penetration was low, electric loads on different buses and nodes of a power system
exhibited strong regularity and therefore were quite predictable—transmission-level load forecasting could achieve
a mean absolute percentage error (MAPE) of ≈3% [23]. Under such conditions, the amount of regulation and
load following can be obtained via deterministic models. With the increasing penetration level of renewable energy,
deterministic optimization is no longer adequate due to the high uncertainty in renewable generation [113]. Although
new theoretical models, such as stochastic optimal dispatch and robust optimal dispatch, have emerged, the real-
world adoption of these models is rather slow [114]. For the day-ahead market with high renewable penetration,
the system operators have to come up with a plan based not only on day-ahead load forecasts but also on day-ahead
renewable forecasts [115], with the latter being far more difficult to forecast than the former—one can expect a
MAPE of about 15% for wind [23] and a much higher value for solar [116]. Because of large errors in day-ahead
renewable forecasts, which then lead to an inaccurate day-ahead schedule of conventional generators, the burden of
supply-demand balancing falls onto the real-time market, in which regulation and load following take place.
Fig. 8illustrates operations of a real-time market, such as that managed by the California Independent System Operator
(CAISO). Firstly, intra-day scheduling compensates deviations in the day-ahead schedule on a rolling basis. Like in
the day-ahead market, the selection of generators is based on market participants’ bids for the real-time market. Real-
time unit commitment then decides when and which generating units provide power at each system node. CAISO
load synergy with solar power.
21Both regulation and load following seek to control the power generation so as to match the temporal load variations. The key distinction
between regulation and load following is the time period over which these load fluctuations occur [112]. Regulation responds to rapid load
fluctuations (on the order of 1 min), whereas load following responds to slower changes (on the order of 5–30 min). Naturally, the power system
operations used for regulation and load following are also different.
22Ancillary services are services which support the continuous flow of electricity from generators to consumers and the stability of the power
system (as opposed to trading of energy between consumers and generators). Beside generation and transmission, ancillary services are also tasked
to perform active power control (or frequency control) and reactive power control (or voltage control), on various timescales.
23In competitive electricity markets, sellers participate by submitting bids to sell energy, and market operators set the price at which supply
equals demand.
14
MW
t
Operating hour
Day-ahead schedule Intra-day schedule
After economic dispatch Actual generation
Intra-day scheduling revise
Regulation
Load following
Figure 8: California Independent System Operator (CAISO) real-time regulation and load following. The dashed black line indicates the result
of day-ahead schedule, whereas the orange line corresponds to the revised intra-day schedule. Based on the orange line, unit commitment and
economic dispatch are conducted, arriving at the blue load-following curve. Finally, any remaining discrepancy between the blue curve and the
green curve (i.e., actual generation/load) is the amount of regulation needed, which is provided by flexible resources, such as spinning reserve or
batteries. Inspired by Makarov et al. [9].
performs real-time unit commitment every 15 min [9]. Within each cycle of real-time unit commitment, multiple
rounds of real-time economic dispatch are required. The power output from most conventional generators is not fixed,
but can vary around a set power. Such adjustment is usually performed via the load-following automated dispatch
system (ADS). Hence, economic dispatch allows generators to move towards new set points according to the current
demand.24 This process takes place every 5 min in CAISO [9]. In short, load following is an instructed deviation
from scheduled output by the real-time economic dispatch [9]. Finally, regulation refers to minute or sub-minute
deviations between the generation and load, which need to be fulfilled through flexible resource in the power system.
Traditionally, flexible resources mostly refer to spinning reserves25 and hydro power, whereas more recently, batteries
and other forms of energy storage also contribute towards flexible resources, but their capacity has been limited
due to their high cost. There are emerging flexibility markets actively deployed by some TSOs or DSOs whereby
contracted services offer load turn down or up (for example, industries that can scale up or down operations), as well
as generation curtailment (for those who accepted a flexible connection rather than a firm connection), all for better
load management across various assets on the grid. These flexibility markets are initially planned for areas of the grid
where asset reinforcement can be meaningfully deferred, hence flexible services are scheduled months in advance.
These services are reevaluated up until a day ahead when the bidding settlement period closes; after that, the operators
may request the services if needed. With greater integration of forecasting, these innovative smart-grid solutions can
be more widely used and become business-as-usual operations.
The details above indicate that the accuracy of solar forecasts has a profound impact on the amount of flexible re-
sources needed to maintain balance between generation and load. Under high PV penetration in markets where
renewable generators receive preferential treatment (if they are exempt from bidding or if they are price-takers), an-
cillary services are provided in response to fluctuations in the net load of the system. The latter is the load minus
the renewable generation, that is, the amount of load that needs to be met via conventional generators and flexi-
24Conventional generators require a substantial spin-up time before they can inject power to the grid. Even if at the “On” state, some lead time
is needed to substantially adjust the power output of the generator.
25Spinning reserve refers to generators that do not continually output power, but are on “stand-by” to respond within a certain time horizon to
grid operator requests.
15
ble resources [117]. Large forecast error of solar power generation often implies large forecast errors of net load,
which subsequently result in sub-optimal unit commitment and economic dispatch. This superimposed uncertainty of
variable solar and load requires more regulation, i.e., more spinning reserves or energy storage, which translate into
higher costs. In fact, keeping a large amount of such flexible resources not only affects the cost-effectiveness of power
generation, but also reduces equipment utilization, which has hitherto been viewed as a major adverse component of
energy economics. Besides the economic impact, there is also a technical difficulty imposed on reserve allocation.
Traditional spinning reserves are determined based on the experience of the operators to quantify demand uncertainty.
With high penetration of solar generation, the reserve allocation is affected by the solar forecast error, though that
effect remains opaque due to the confounding sources of uncertainty. Lack of operator familiarity with solar forecast
error characteristics often causes reliance on anecdotal evidence and overly cautious reserve allocation. One thing is
however certain: there is a correspondence between the quality of forecast (e.g., accuracy or skill) and the value of
forecast (e.g., savings on reserves). For a large power system, a small improvement in forecast quality usually leads
to a substantial amount of economical benefit [118–120].
4.5.2. Stability and control
Load flow26 is usually performed in a deterministic fashion [121], which describes the operating state of the power
system, and indicates the voltage, current, and power delivery from source to load in the power system, under a certain
wiring and operating mode. The wiring mode of the complete power system (e.g., at regional scale) refers to the
connection between each electric component over wide areas. This connection is generally fixed and would not have
a major impact on load flow. Contrary to the wiring mode, the operating mode of the whole power system changes
with the power injection to the nodes. In traditional power systems, operators can decide whether the system is in the
desired steady-state secure region, based on the load of a few typical “load” scenarios.27 The load flow calculation
in various operating modes can grasp the operating status of each component (e.g., generator or transformer) in the
power system comprehensively and accurately, and thus determine a power supply plan reasonably and efficiently
[122]. Through load flow, one can find the weak links in the system and check whether the power system components
are overloaded. It ensures that all nodes of the power system can be at a normal voltage level.
With the high uncertainty introduced by solar and wind generators, deterministic approaches can no longer support the
analysis of modern grid systems, because the definition of their operating modes is now more complicated. As such,
the results from load flow may become unreliable and may eventually lead to biased assessments. Considering that
power system operators are supposed to be conservative rather than taking risks, remedies for regaining that high-level
visibility on operating status must be sought. To anticipate the possible changes in system operating status brought by
renewable fluctuations, and to obtain the risk probability of the power system, probabilistic solar forecasting is needed
to support load flow calculations [123]. Stated differently, probabilistic load flow (PLF) is becoming increasingly
essential. Under PLF, the load flow result not only describes the expectation, but also allows a probabilistic judgement
on the bus voltage and line power.
To quantify the uncertainty in solar power injection when conducting PLF, different numerical or analytical approaches
can be used. Numerical approaches sample the possible states of the stochastic variables and parameters of the
system model, via Monte Carlo and Latin hypercube sampling. Given the spatial and temporal correlation that affects
the stability and control of the power system, multivariate probabilistic solar forecasting becomes relevant in which
equiprobable irradiance or PV power samples describe the multivariate distribution over space and time [12,60].
On the other hand, analytical PLF requires as input the probabilistic distributions of solar and load forecasting error
[124]. At present, such probability distributions are determined based on the operators’ experience. For instance, load
forecasting errors are often described as a Gaussian distribution [124], whereas solar radiation is assumed to follow a
26Load flow computes the magnitude and phase angle of the voltage at each bus (i.e., a node in a power system), as well as the real and reactive
power in each transmission line, such that the operation of an existing system can be optimized. Load flow is also required for many other power
system operations such as transient stability analysis, unit commitment, or economic dispatch.
27Typical “load” scenarios refer to the most representative daily grid load for different seasons. In mid-latitude countries or temperate climates,
these scenarios are often divided into four categories, namely, maximum daily load in winter, minimum daily load in winter, maximum daily load
in summer, and minimum daily load in summer.
16
Beta distribution [123],28 and wind speed to follow a Weibull distribution [122]. This experience-based assignment of
the probability density function may deviate from reality. Accurate probability information of solar can be obtained
through advanced probabilistic solar forecasting methods, which in turn help make the calculation of probabilistic
power flow resemble the actual system operating conditions, and thus increase the credibility of PLF.
4.5.3. T&D planning
Unlike traditional power generation, PV and other renewable energy technologies are not just sensitive to weather
changes, but also distributed unevenly across geographical locations, time periods, and climatic zones. In China for
instance, there is a strong mismatch between its solar-resource-rich regions and load centers. As shown in Fig. 9, solar
energy resources are abundant in northwestern China, whereas the majority of the population is concentrated in the
coastal regions in the east. This brings significant challenges to the grid operators. Theoretically, ultra-high-voltage
large-capacity long-distance transmission is able to deliver the solar-generated power from the west to the load centers
in the east [125]. These large-scale interconnection and cross-regional power distribution capabilities of the ultra-high
voltage transmission act as a “highway” for solar energy transmission and consumption. Owing to the self-rotation of
the Earth, the peak of solar power from the west aligns well with the late-afternoon load peak in the east. Nonetheless,
exploiting this beneficial temporal alignment will not be possible without appropriate long-lead transmission system
planning.
1200 1600 2000
Annual GHI [kWh/m2]
Figure 9: Annual global horizontal irradiance (GHI), in kWh/m2, over Asia. The artifact line at 88◦longitude is due to the use of two different
satellite-derived GHI products, each with incomplete coverage over China. Data source: National Solar Radiation Data Base, https://maps.
nrel.gov/nsrdb-viewer/.
Most importantly, the primary purpose of transmission system planning is to set a timeline for expansion of the current
system such as to meet the future energy demand. According to Willis and Northcote-Green [126], system planning
can be divided into short-term planning and long-term planning. For short-term planning, the goal is to devise a
strategy that best ensures a timely order and delivery of power system equipment to suit the needs for next-batch
commitment, construction, and upgrade of the power system. On the other hand, long-term planning is essential to
analyze the economic viability of the short-term commitments; it relates to the utilization efficiency of the equipment
throughout their lifetime. In the past, when load was stable, the need for inter-zonal29 transmission was moderate.
28Solar forecasters might object to the choice of the Beta distribution because it appears suspicious. It might have been useful to find the best-fit
parametric distributions based on certain forecast errors, but one should not regard it as valid if another forecasting model is applied to another
dataset.
29At the regional or country scale considered here, a whole power system constitutes an interconnection, itself composed of different zones, each
spanning wide geographical areas, such as one or several territories, provinces, or states.
17
But under high solar penetration, inter-zone transmission would become far more frequent than before, which would
then provide justification for building new cross-regional transmission lines.
Accurate long-lead solar resource assessment is beneficial to evaluating system reserves over large geographical region
and to T&D planning [127]. Underestimating the areal solar resource may cause serious transmission congestion30 in
a system with a high proportion of PV power. An inadequately planned transmission system sets barrier to transporting
excess PV power across regions, and in that case, one has no choice but to curtail the excess and waste the energy. On
the distribution side, underestimating or overestimating solar generation may lead to incorrect sizing of distribution
transformers and other equipment, and unfair allocation of cost to rate payers.
On the other hand, allocating sufficient flexible resources is another way to compensate daily, mesoscale, and seasonal
solar variability, though the cost may be difficult to justify. In the future, with the gradual phase-out of carbon-based
thermal generators and the increasing penetration of solar and wind, the demand for power system flexibility over long
time scales will continue to increase. Over prolonged periods of cloudy, rainy or stormy weather31, it is impossible
for PV alone to meet the demand, regardless of how much PV is installed in that region. To anticipate such prolonged
low-sun periods, it is still unclear what form of flexible resource should be allocated, and with how much flexibility.
In any case, issuing accurate week-ahead solar forecasts is a clear and critical objective. More generally, assessing the
inter-week and seasonal variations in solar resource would soon become essential in a solar-predominant grid. This is
closely related to the hot topic called sub-seasonal to seasonal (S2S) forecasting in meteorology.
5. Potential research topics for atmospheric scientists
In this section, several pressing issues in solar forecasting are explained from the perspective of solar engineering.
Whereas certain issues, such as spatial forecast verification, already have standard procedures available in the at-
mospheric science community, they lack transferability to solar forecasting; by contrast, other issues require further
investigation by atmospheric scientists. Figure 10 gives an overview of the topics on which the participation of atmo-
spheric scientists is thought highly beneficial. In any case, it should be noted that the list is non-exhaustive, so that
periodic revisits to these research topics are necessary.
Ten potential research topics
1. NWP as
a service
2. NWP
configuration and
augmentation
for solar
3. Forecast
downscaling
4. Large eddy
simulation
5. Dimming
and Brightening
6. Aerosols
7. Spatial fore-
cast verification
8. Multivariate
probabilistic fore-
cast verification
9. Predictability
10. Extreme
weather
Figure 10: Ten potential research topics for atmospheric scientists in order to improve solar forecasting.
5.1. Providing NWP datasets and tools for solar forecasting
As discussed in Section 4.1, exogenous data for solar forecasting are of three main types: (1) sky images captured by
all-sky imagers or cameras, (2) satellite-derived cloud, reflectance, or irradiance fields, and (3) NWP output. The first
two types of data can be obtained with relative ease [see e.g., 128–130, for instructions], but the collection of the third
type of data is often hindered by the operational nature of NWP, where historical forecasts are not publicly available,
usually.
Take for instance the various NWP models run operationally by the National Oceanic and Atmospheric Administration
(NOAA), such as the North American Mesoscale (NAM) forecast system or the Rapid Refresh (RAP) model. Their
30Transmission congestion occurs when there is not enough transmission capacity to support all requests for transmission services.
31For instance, when Typhoon In-Fa made landfall in eastern China on July 24, 2021, Shanghai did not see the sun until 15 days later.
18
forecasts are publicly available online for a few days only; then, they are moved to a data storage facility without
public access. Because it is essential to conduct verification over several years of data before one can conclude on the
performance of any forecasting technique, collecting these operational forecasts needs an automated client program
or a downloading tool; local disk space may be another concern. On the other hand, some other NWP forecasts, such
as the ones issued by the European Centre for Medium-Range Weather Forecasts’ (ECMWF’s) Integrated Forecasting
System (IFS) may come with a fee, which tends to limit their uptake. Therefore, it would be highly useful if an amount
of relevant NWP output variables that is sufficient to perform solar forecasting can be provided in an easily-accessible
and easy-to-use fashion; the reader is referred to Pedro et al. [43], Yang et al. [131] for sample datasets of that nature.
Another major data issue faced by solar forecasters is related to ensemble forecasts. Although ensemble forecasting
has been the default ever since the 1960s when Edward Lorenz first founded chaos theory, ensemble solar forecasting
has yet to become popular. This is again thought to be related to the limited accessibility to ensemble NWP forecasts.
Ensemble forecasts can be divided into several kinds [132]. The one that is most used by solar forecasters is called
poor man’s ensemble, which is nothing but a collection of outputs from several models. Following the principle
of forecasting [19], simple averaging or linear blending of these forecasts from different models is often beneficial,
which has been also demonstrated in the case of solar irradiance [71,133]. However, to truly explore the benefits of
ensemble solar forecasting, dynamical ensembles are needed. These are ensembles that consist of forecasts generated
by perturbing the initial conditions such as those produced by ECMWF’s Ensemble Prediction System [47,134,
135]. In other words, due to the nonlinear complexity of the system, purely statistical uncertainty quantification is
insufficient, and an ensemble with many complete, physical, nonlinear realizations of the system is essential [15].
Nevertheless, there are not many operational NWP models that provide outputs of GHI as an ensemble parameter, let
alone its beam and diffuse components. As mentioned above, probabilistic modeling of solar power and probabilistic
power system operations are expected to become the norm in the future, so that ensemble solar forecasts are in stark
demand.
Last but not least, the value of software packages and tools is obvious. In atmospheric science, many packages have
a long history, such as those containing radiative transfer codes. However, a long history comes with legacy issues,
within which the most apparent one is related to the language in which the libraries are written. C and FORTRAN
were popular and fast, but should no longer be viewed as the best option insofar as mainstream scientific research is
considered. That said, porting those gigantic C and FORTRAN codes into the more popular Python and R would be
a monumental task, and would come with a substantial loss of speed. Even though it can be argued that computing
power steadily increases, the priority of weather forecasters is to improve the spatio-temporal resolution of their
models and increase the quantity of observations to be ingested as frequently as possible. Hence, it is currently
unimaginable to reprogram and run NWP or radiative transfer codes directly in, e.g., Python. An interesting solution
to this dilemma is to use wrapper functions that provide an interface between programming languages. One example
of this is fv3gfs-wrapper in Python [136]. Similarly, a recent R framework called Climate4R is an excellent
example of how to best deliver atmospheric science tools to statisticians [137–139].
5.2. Improving the configuration and augmentation of NWP models for solar forecasting
NWP data is generated from physics-based models that attempt to simulate all processes in the atmosphere such as
clouds, rain, heat flows, or topography-induced convection. The equations of fluid mechanics and thermodynamics are
solved on a numerical grid that covers thousands of kilometers, or even the entire globe [140]. Due to the importance
of weather on the economy and national security, since the mid-20th century, weather forecasts are routinely generated
by government entities in all countries that have both the knowledge and capacity to run NWP models [141]. National
and international weather centers are responsible for maintaining and improving the NWP models and maintaining
the computational infrastructure to run NWP models fast enough for forecasts to be practically relevant.
Nonetheless, since NWP runs typically consume thousands of computing core-hours, generating such data opera-
tionally requires dedicated computing resources, typically orders of magnitude larger than the resources required for
satellite or statistical forecasting approaches. In fact, high-performance computers used for NWP are among the most
powerful computing systems in the world [142], and can execute computations at petaflop per second rates [15]; this is
clearly not the kind of resource that can be accessed by every individual who is interested in NWP. Consequently, the
vast majority of solar forecasters do not usually run any NWP themselves, but rely on the output of operational NWPs
19
and perform post-processing through statistical or machine-learning means [61,62,73,143]. A notable exception is
the Weather Research and Forecasting (WRF) mesoscale model,32 as discussed below.
WRF is in the public domain and is used by many scientists and individual or institutional forecasters to prepare
forecasts for a wide variety of applications, including wind and solar energy. Like any other NWP model, it provides
a wealth of output data. By default, WRF generates time series of over 100 variables at 106or more grid points, and at
output intervals from minutes to an hour. Clearly, there are too many degrees of freedom for inputting all its outputs
to machine-learning models. However, the down-selection of variables is challenging, as the relevance of type and
location of variables depends on weather conditions. Here, expertise in atmospheric sciences can help identify the
proper variables to extract from NWP outputs as most relevant to solar forecasting. Works of this kind are exceedingly
rare, but Yang et al. [144] presented a variable selection method through a tangent linear sensitivity analysis.
NWP models typically contain hundreds of thousand of lines of computer code. Since the spatial resolution is usually
poor, meteorological processes cannot be simulated and described using first principles, but need to be parameterized.
The meteorological processes are split into separate physics “packages” or “schemes” for disparate physical processes
such as radiation (solar and terrestrial), land surface processes such as water and heat movement in the soil, boundary-
layer (near-surface) turbulent mixing, and cloud and rain formation (microphysics). Within each physics package
there are several models that are often named after the scientists who developed them, and each model has several
parameters. These parameters are typically designed to be tuned only by experts, while different physics packages
can easily be selected by anyone running the models.
For example, for microphysics (cloud and rain) models, the current version of WRF provides over 20 packages.33 The
schemes differ by how many “classes” of microphysics components are simulated, e.g., liquid cloud water, rain, ice
clouds, ice precipitation (snow, hail, graupel), by solving either one prediction equation for mass per class (single-
moment) or two equations (double-moment) with an additional prediction of number concentration per class, by how
the descent of precipitation in the atmosphere (i.e., rainfall) is modeled, and how the vapor-liquid-rain thermodynamic
processes and their interaction with cloud condensation nuclei (CCNs) are parameterized.
There are probably not many experts in the world who understand all physics packages well enough to make the
best choice for each type of possible application or climate-environment situation. Most WRF users are faced with
the dilemma of having to pick physics packages out of more than 106possible combinations. Typically researchers
follow the choices of domain experts, but that limits the value of customization as the choices by the domain experts
are often exactly the ones implemented in the NWP instances by the national weather centers.
For example, one of the authors’ work using the Mellor-Yamada Nakanishi and Nino (MYNN) planetary boundary
layer (PBL) scheme [145] showed that solar forecasts under stratocumulus clouds in Southern California are very sen-
sitive to the microphysics (MP) options and that there are significant inconsistencies in cloud modeling in WRF [146].
Most modelers use the Thompson-with-aerosol MP package [147], as in the NOAA’s High Resolution Rapid Re-
fresh (HRRR) operational forecasts, and the following discussion adopts this specific scheme. Counter-intuitively, the
Thompson MP and MYNN PBL have separate algorithms for calculating condensation (i.e., cloud water content).34
While the (parent) WRF model simulates potential temperature, the PBL scheme simulates liquid water potential
temperature. To convert between the two, it is necessary to know the cloud liquid water content (LWC). The PBL
scheme often has a different condensation assumption than MP; as a result, the cloud LWC can be very different.
In other words, sometimes separate physics packages (e.g., MP) are embedded into another physics package (e.g.,
PBL) because of the inter-dependency between clouds, radiation, and boundary layer turbulence, for instance. This
creates substantial confusion for the user/developer and compatibility issues between different schemes within WRF,
resulting in inconsistencies and inaccuracies (pers. comm. with Elynn Wu).
All of the above issues have convinced these authors that rigorous solar forecasting research needs dedicated re-
searchers in the atmospheric sciences community who can maintain a holistic view of physics packages associated
with any NWP model that is intended for solar forecasts. In particular, the inter-dependencies of physics packages in
32https://doi.org/10.5065/D68S4MVH.
33https://www2.mmm.ucar.edu/wrf/users/physics/phys_references.html
34In some cases, different combinations of parameter choices for WRF can even result in different condensation algorithms within MYNN.
20
WRF make it hard to isolate the impact of specific model improvements, how well-reasoned they might be. Moreover,
with the increase in spatio-temporal resolution of NWP models, their parameterizations would be gradually replaced
by explicit considerations of various physical processes impacting solar radiation and clouds. In parallel, the rapid
development of high-resolution atmospheric observations would provide valuable data to fulfill this goal.
5.3. Forecast downscaling
Although NWP is normally able to generate forecasts at arbitrary temporal resolutions,35 most operational NWP mod-
els, such as ECMWF’s IFS or NOAA’s RAP and HRRR, issue hourly forecasts by default, largely because of stringent
data storage concerns. In parallel, power system operations are often conducted at a higher temporal granularity such
as 5- or 15-min intervals. CAISO requires its intra-day forecasts to be submitted at 15-min resolution out to 5 h [100],
whereas the Chinese grids require 15-min forecasts out to 7 days [32]. In any case, the need for forecast downscal-
ing arises. Forecast downscaling refers to the procedure of converting low-resolution time series to one at a higher
resolution. Mathematically, downscaling is an inverse problem; its difficulty resides in information reconstruction.
By analogy, this means guessing an object from its shadow. More specifically, reconstructing the high-frequency
fluctuations in the high-resolution solar time series from a smooth low-resolution time series is thought to be a major
challenge.
There are currently two methods for downscaling forecasts temporally, one trivial and the other slightly more tech-
nical. The trivial one is interpolation; exogenous information, such as the sun’s zenith angle or clear-sky irradiance,
which can both be calculated at high temporal resolution, can be used to guide the interpolation. However, interpolat-
ing time series usually has no effect on its variability, unless high-resolution cloud information is available, which is
rarely the case. Nevertheless, having a smoothed forecast transient is not necessarily harmful to accuracy. The more
technical approach is known as analogue ensemble, which is a familiar term to weather forecasters. Analogues refer
to similar patterns in weather. By identifying past (low-resolution) forecasts that resemble the current one, the past
(high-resolution) observations can be treated as a potential scenario, according to which the future high-resolution
event materializes. When multiple candidates are gathered, corresponding to the top analogues, a high-resolution
ensemble forecast is formed [see 148]. Since analogue ensemble as a technique contains countless variations, future
works to explore its full potential are welcome. One pitfall of the analogue-based temporal downscaling is that it
requires a large historical database to operate optimally, because the quality of analogues depends largely on the simi-
larity between the query and its top matches. This poses a major limitation to the method, since long-term high-quality
forecast–observation pairs are hard to find.
Besides temporal downscaling, spatial downscaling is also required. Conversely to the under-development of temporal
downscaling, spatial downscaling of solar radiation data has gained some attention [e.g., 149,150]. These existing
methods mostly leverage topographic features, such as elevation, which are often available at high spatial resolution, to
correct for the sky view factor and horizon blocking. Certainly, these methods would lead to a greater difference before
and after downscaling in areas in which topography is complex. Moving forward, it would be useful to explore the
effects on spatial downscaling of other anchor variables that are available at high spatial resolution, such as cloudiness
climatology, aerosol climatology, or the Köppen–Geiger climate class.
5.4. Large eddy simulation
Due to ever-increasing computational resources, operational global NWP models have undergone an exponential
increase in horizontal resolution from about 300 km in the 1970s to <10 km now. However, the inherent weakness
of current NWP models—independent of their resolution—is the inability to simulate turbulence in the atmosphere.
Especially for the modeling of cumulus clouds, turbulence modeling is essential in several regards. The entrainment
of dry and warm air from the free troposphere decreases cloud lifetime, yet state-of-the-art PBL schemes in WRF
dramatically underestimate entrainment, leading to a tripling of cloud liquid water [151]. In addition, clouds generate
35The latest operational NWPs have a horizontal resolution of 3 km with hourly outputs. If specifically requested, i.e., by modifying the output
code, 15-min output can also be easily produced. Additionally, the forecast products can be ready within 45 min, so it seems possible to produce
“true” intra-hour solar radiation forecasts from most NWP systems. At the moment, however, these high-resolution NWPs have not yet gained
widespread uptake by solar forecasters.
21
their own turbulence, which is critical for prolonging the lifetime of stratocumulus clouds, for example. Nudging
model parameters and coefficients has provided some success in solar forecasting with WRF, but a broad applicability
of these models is still elusive.
On the other hand, large eddy simulation (LES) methods can simulate turbulence in the atmosphere in great detail.
For example, the Parallelized Large-Eddy Simulation Model, which has been developed at the Leibniz University
of Hannover over more than 20 years [152], has been applied to various research topics concerning the atmospheric
boundary layer. The FORTRAN95/2003 code is highly-optimized, displays nearly-perfect scalability for more than
25,000 processor cores, and is prepared for the use of new accelerator hardware [152]. Whereas traditionally LES
domains have been limited to a few kilometers, simulations with domains over 400 km are currently possible [153].
The time seems now ripe to implement LES into operational solar forecasting at city scale, starting with areas ex-
periencing high PV penetration. For instance, Whiffle36 leverages graphical processing units (GPUs) to accelerate
intensive computations and offers operational LES forecasts on a commercial basis.
5.5. Dimming and brightening
As discussed in Section 4.5.3, power system transmission planning requires long-term forecasting, or rather, projec-
tion, of solar resource availability. The uncertainty in the future solar resource is a critical aspect of the financing
and profitability of any solar project, which has prompted the solar community to revisit the conventional resource
assessments (based exclusively on historical data) to also include future projections [154]. In this regard, a related
topic in the climate field is that referred to as “global dimming and brightening” (GBD), which refers to the downward
or upward decadal variations in surface solar radiation. For instance, many areas have experienced dimming until the
late 1980s and brightening thereafter [155–157].
From the existing literature, it is apparent that one main cause for GBD effects comes from long-term fluctuations in
anthropogenic aerosol emissions at continental scale, themselves directly related to regional economic and industrial
development, as well as air quality regulations [158–160]. Other apparent causes are variations in atmospheric water
vapor and cloud regime [161,162]. These three underlying causes tend to impact DNI much more than GHI. Based on
current studies, GBD-induced decadal trends can reach a few percent of the long-term mean annual GHI, and possibly
more than 10% for DNI. Such effects are significant and need to be understood by the solar industry.
Owing to the logarithmic sensitivity of clouds to CCNs [163], small changes in CCNs potentially bring about larger
modifications to cloud properties over regions without pollution than with pollution [164]. Overall, it can be said
that human activity (comprising economic and industrial development, as well as carbon and air-quality policies)
contributes to changes in aerosol emissions, and thus directly or indirectly alter surface radiation.
Since solar radiation has a profound impact on surface climate, hydrological cycle, glaciers, and ecosystems, the GBD
phenomenon plays a key part in global climate models. Those are likely to be the only tools that are powerful enough
to investigate the (future) impacts of anthropogenic activities on climate, as already suggested by a few studies [165–
167]. Although the solar industry would like to rely on projections of the solar resource for the next 20–30 years, it
is obvious that those entail substantial uncertainty because of the complex nature and interactions in the underlying
phenomena. Moreover, a huge unknown is whether some international effort (e.g., under the auspices of IPCC) will
attempt to reduce the incident solar radiation over the whole planet by injecting layers of reflective particles or gases
in the high atmosphere. Such a drastic measure to reduce global climate change effects was suggested and studied
by IPCC and others [168–170]. Such geoengineering (also called “solar radiation management”) techniques would
decrease the solar resource considerably more than any of the global dimming effects discussed above, at the risk of
devastating the solar industry, all depending on the intensity of the high-altitude filter being deployed [171].
Even though the literature on GBD is rich, some findings are derived from satellite-derived irradiance data and ground-
based radiometry with suboptimal quality. This is particularly true for pyranometer or sunshine data that extend
far back into the past. The quality of such measurements is hard to guarantee and may induce an incorrect trend
quantification, at least in some cases [172]. Similarly, most of the current-generation satellite-derived irradiance
36https://www.weatherfinecasting.com
22
products might exhibit heterogeneous bias or discontinuities across space and time, particularly at the seams between
bordering satellite domains [173]. Despite all these artifacts, the overwhelming evidence is that the observed trends
are real and consistent. In any case, relying on a single product seems to place too much confidence on the data, and
ensemble modeling may be deemed useful in this respect.
5.6. Not just clouds: The role of aerosols
Cloudiness changes fast over space and time, and thus logically constitutes the main focus in solar forecasting. More-
over, solar technologies have developed most rapidly in countries having a temperate climate, where cloudiness is the
main driver of solar radiation variability. That situation is rapidly evolving because solar applications now spread over
the whole world. In particular, arid regions with little cloudiness are acknowledged for their high solar resource and
now receive critical attention from solar developers.
In addition to their low cloudiness, an important characteristic of arid areas is their abundance of sand and dust, which
can be easily uplifted by wind. Many such areas thus behave as large sources of mineral aerosols at regional scale.
Moreover, dust storms or sand storms can occur frequently over large deserts. Considerable quantities of mineral
aerosols can be transported over thousands of kilometers [174–176]. The same phenomenon of transcontinental or
intercontinental long-range transport of large aerosol clouds is also true for smoke or pollution events [177–180]. All
this is important because large aerosol loads in the atmosphere (appropriately characterized by the quantity called
aerosol optical depth, AOD) result in haze, whose direct impact is a concomitant decrease in the local solar resource
[181]. In turn, aerosol-induced changes in the incident irradiance can seriously impact the energy production of solar
installations [182–184]. Furthermore, through the process of dust deposition in particular, the presence of large loads
of atmospheric aerosols can result in soiling of solar collectors or modules, thus decreasing their power output and
yield [185,186]. Keeping solar modules or mirrors clean is costly and typically requires water, which is scarce in
arid areas [184]. Hence, the scheduling of cleaning periods and the overall operation of solar power plants could be
optimized if dust deposition could be predicted with enough accuracy—a few days ahead at least.
In parallel, aerosols also act as CCNs and ice nuclei, which are key to the formation of water or ice clouds. Through
that indirect process, the prediction of cloud formation is linked to that of aerosols. Hence, proper solar forecasts de-
pend (both directly and indirectly) on the proper forecast of aerosol properties. Currently, accurate aerosol forecasts of
a few days ahead are still computationally intensive and extremely difficult—thus uncertain. A few chemical transport
models, coupled with global or mesoscale NWP models, exist to forecast aerosols, but their predictions substantially
disagree in terms of spatial spread and intensity of AOD, in particular. To visually appreciate such differences over
northern Africa, the Middle East, and Europe, the reader is referred to online comparisons between the forecasts of
dust AOD and dust surface concentration obtained with 12 different models (plus their median combination), as pro-
vided by the World Meteorological Organization’s Sand and Dust Storm Warning Advisory and Assessment System
(SDS-WAS) regional center.37 (Note that those forecasts are only for dust, so that the total AOD is actually larger.)
Similar regional centers exist for Asia38 and the Americas,39 but they use fewer forecast models.
Among the global models that can be used to forecast aerosol loads up to a few days ahead, two of them—namely,
Copernicus Atmosphere Monitoring Service (CAMS) and Goddard Earth Observing System, Version 5 (GEOS-5)—
are of particular interest in the present context because their forecasts of total AOD and other aerosol-related quantities
are publicly available and accessible relatively easily. CAMS is a chemical transport model developed by the ECMWF
that is able to forecast aerosol quantities at a resolution of 0.4◦×0.4◦up to 120 h ahead. Public access to the data is
possible through ECMWF’s Atmosphere Data Store. The GEOS-5 forecast model is developed by the National Aero-
nautics and Space Administration (NASA) to provide high-resolution forecasts of the global atmosphere, including
aerosols and surface solar irradiance. Those hourly forecasts are at 0.3◦×0.25◦spatial resolution up to 120 h ahead
and are available from NASA’s Center for Climate Simulation website.40 Interestingly, the GEOS-5 aerosol forecasts
can also be used to feed the solar-augmented version of the WRF, namely, the WRF-Solar©mesoscale model [14].
37https://sds- was.aemet.es/forecast-products/dust- forecasts/forecast-comparison)
38http://www.asdf- bj.net/
39http://sds- was.cimh.edu.bb/
40https://fluid.nccs.nasa.gov/weather/
23
As mentioned above, that model is used by many forecasters worldwide to predict the solar irradiance components at
high spatio-temporal resolution over a specific region [57,187,188]. An application of using GEOS-5 aerosol data as
inputs to WRF-Solar to forecast the beam and global irradiance components has been demonstrated over the Middle
East [189], but revealed some limitations caused in large part by the uncertainty in the aerosol forecasts.
Although intensive research is devoted to the improvement of global models that are able to forecast the trajectory and
intensity of various aerosol species, the prediction of AOD is still not accurate enough for the needs of CSP applica-
tions, because they require a precise determination of DNI, which is substantially more dependent on AOD than GHI
[181]. Similarly, the prediction of the exact time of passage and strength of future dust, smoke, or pollution episodes
is still relatively imprecise. It is anticipated that ensemble modeling, bias correction of remote-sensed observations
used for initialization, and appropriate post-processing, can gradually improve such forecasts.
5.7. Spatial solar forecast verification
The reason for performing spatial solar forecast verification is two-fold. From a meteorological perspective, spatial
verification has several benefits over single-location verification, such as its ability to detect and quantify shift, rota-
tion, and sheer in the forecast field, so that two-dimensional error features can be corrected during post-processing
[190]. From an engineering viewpoint, spatial solar forecast verification is needed to quantify the errors in regional
solar power output, which is of core interest to power system operators when computing nodal-level solar generation
and net load [191]. Unfortunately, as suggested by the current literature, spatial solar forecast verification has yet to
receive any attention from solar forecasters.
One of the possible explanations for the under-development of spatial verification in solar forecasting is the historical
lack of interest in areal PV forecasts. Previous solar power forecasting studies were mostly focused on single-location
solar (PV or CSP) systems, apparently because most market mechanisms promote the decentralized installation of
such systems. Naturally, when forecasting is needed for grid-integration purposes, solar system owners are only
tasked to generate forecasts for their own systems, and to submit those forecasts to independent system operators
(ISOs); then the ISOs aggregate the received forecasts to form an estimate of the total solar generation within an
area [192,193]. In recent years, this bottom-up strategy of obtaining areal power forecasts has been criticized,
due to its inability of leveraging the zonal- and nodal-level patterns, which may contain useful information that can
benefit forecasting. In fact, hierarchical forecasting, which views the solar power generation in a power system as
a hierarchy,41 has been shown to be able to improve substantially the bottom-up strategy in terms of both accuracy
and uncertainty [116,194,195]. Therefore, as the hierarchical forecasting concept is progressively accepted by more
stakeholders, the need for spatial forecast verification naturally arises.
Spatial forecast verification is no doubt a familiar topic in atmospheric sciences. Over the past decades, the general
scientific progress in spatial forecast verification has been generally satisfactory, insofar as the applicability of those
verification methods is concerned. This is owing to the fact that the spatial features in weather events such as cyclone
[196] or rainfall [197] have long been regarded as important in order to understand and model their dynamics. Whereas
the reader is referred to the reviews by Gilleland et al. [190], Ebert [198], Marzban et al. [199] for the general strategies
of verifying forecast spatially, some solar-specific considerations are briefly discussed next.
It is the usual practice in spatial forecast verification to leverage moderate-accuracy remote-sensed data to gauge low-
accuracy NWP forecasts (just as high-accuracy ground observations are used to validate moderate-accuracy remote-
sensed data). In terms of solar radiation, geosynchronous satellite-derived irradiance products cover the whole of low-
to mid-latitude region of the Earth [128,200], while retrievals from polar orbiters can be used globally, albeit with
a significantly lower spatio-temporal resolution [201]. More importantly, these satellite-derived irradiance products
have been empirically shown to be able to replace, with appropriate caveats, ground-based measurements, during
41The bottom-level time series are simply those power output time series from each individual solar system: The middle-level time series is
typically composed of nodal and zonal aggregates, whereas the top-level time series is that of the total solar generation within an interconnection.
Hierarchically, forecasting considers each of these time series, produces forecasts, and reconciles the forecasts such that they are consistent when
aggregated.
24
single-location forecast verification [202] and post-processing [203]. Notwithstanding, the accuracy of satellite-
derived irradiance is spatially inhomogeneous [3,204], which can subsequently limit one’s confidence in the veri-
fication results: it is unclear whether an observed error is due to forecast or due to satellite-derived irradiance, so long
as the ground-truth is unavailable. Another important consideration for verifying solar forecasts spatially is that the
spatial representativeness, i.e., scales, of satellite-derived or ground-observed irradiance and concomitant NWP pre-
dictions are different. This is known as the spatial mismatch problem, which does not have any mature solution in the
field of solar energy [205]. Whereas regridding [206] and resampling [207] are apparent solutions, how to interpolate
and sample optimally, and what defines the optimality, remain largely unknown.
5.8. Multivariate probabilistic solar forecast verification
Probabilistic solar forecasts represent a marginal probability distribution expected at a certain location or area and
at a certain forecast horizon. One challenge is that such forecasts neglect spatial and temporal dependencies, which
must be considered when performing unit-commitment and economic dispatch with slow ramping thermal generators.
In such a setting, multivariate probabilistic solar forecasting becomes relevant because the output is a multivariate
probability distribution that, for instance, spans the entire day-ahead bidding period for a single or multiple PV plants
[12]. A vital aspect of such multivariate probabilistic forecasts, which are often issued as trajectories, is that the
correlations need to be modeled correctly to facilitate optimal scheduling of conventional generators. As such, the
calibration of multivariate probabilistic forecasts not only pertains to the statistical consistency between the forecasts
and the observations, evaluated by a histogram of the probability integral transform variables, but also whether the
correlation of the forecasts resembles that seen in the observations [208].
In 2008, only a small part of the literature focused on multivariate continuous data [209]. Since then, many impor-
tant articles have been published on multivariate forecast verification [see e.g., 208,210–212]. However, one of the
remaining challenges is the effect of correlation and limited sample size on the uncertainty of multivariate verification
tools. In the case of univariate probabilistic forecasting, Bröcker and Smith [213] introduced consistency bands to
account for randomness induced by a testing set of limited size. Subsequently, Pinson et al. [214] proposed an ap-
proach to account for serial correlation between forecast–observation pairs when evaluating the calibration of marginal
predictive distributions. In essence, correlation reduces the effective sample size, which is exacerbated by the high
dimensionality of multivariate forecasting tasks [215]. As of yet, there is no method that accounts for the sample size
and correlation between forecast–observation pairs in multivariate probabilistic forecasts. Given the importance of
multivariate probabilistic solar forecasts in power system operation, proper verification is crucial to enable high PV
power penetration into the power system, and is thought to be a topic atmospheric scientists can contribute to.
5.9. Estimating predictability
The notion of predictability is widely acknowledged by forecasters, however, its quantification has hitherto been
controversial. Most generally, predictability should describe the difficulty of a forecasting situation. As such, it
is known a priori to vary in the high-dimensional space on which the forecast quantity is defined. In the case of
solar irradiance, predictability is at least a function of forecast horizon, geographic location, time period, climate and
weather conditions, sky conditions, and the distribution of atmospheric particulates.
From the layman’s standpoint, situations with high predictability should correspond to small errors, whereas situations
with low predictability should correspond to large errors. Nonetheless, this view is inappropriate because the forecast
error depends highly on the forecaster’s skill. There might be cases where a good forecaster is able to produce small
errors in situations with low predictability, and similarly, cases where a novice is unable to produce any meaningful
forecast even if the underlying forecasting situation is very predictable. For example, if the novice does not account
for the double-seasonal pattern of the irradiance time series via a clear-sky radiation model, and chooses a machine-
learning method instead, it would be quite difficult for that machine to learn the precise sunrise and sunset times, as
well as the peak irradiance at solar noon throughout a year, especially when the training data is restricted to a few
years or less. In conclusion, to quantify predictability, some rules about which forecasting method to use and which
error metric to be leveraged, must be agreed upon first.
In the field of time series forecasting, quantification of predictability is commonly started from a so-called “data-
generating process,” based on which the quantity of interest is assumed to evolve, with some random perturbations
25
known as noise [216,217]. For instance, statisticians might argue that a particular macroeconomic time series is
known to follow an autoregressive moving average process of a certain order. Subsequently, the predictability is
quantified by examining the statistical properties of errors of a set of optimal forecasts, that is, the conditional mean
of that data-generating process. One such predictability quantification procedure is exemplified by Granger and New-
bold [218], who argued that the predictability of a stationary time series is just one minus the ratio between the
variance of perfect-forecast error and variance of the original time series. Nevertheless, this approach of predictability
quantification is perpetually challenged because the data-generating process is never known in reality.
In the field of weather forecasting, the situation is similar. It is customary to attribute the errors of weather forecasts
to two sources, one being diagnosis and the other being prognosis, which constitute two iterative stages of weather
forecasting [141,219,220]. Diagnosis is the analysis procedure for estimating the initial state of a system from
diversity of observations, whereas prognosis refers to the projected evolution of the system state by the NWP [221].
Owing to limited observations and incomplete knowledge about the system, weather forecasts are never perfect.
However, if one wishes to quantify predictability, either diagnosis or prognosis has to be assumed to be perfect,
such that the forecast errors can be regarded as solely due to the other. Anthes and Baumhefner [222], for instance,
assumed the model that prepares the analysis is perfect, such that the remaining errors is due to the uncertainty in
analysis; when that uncertainty propagates as the forecast horizon increases, the forecast errors grow larger, i.e., the
predictability drops, until a certain horizon beyond which there is no predictability. This school of thought appears
too idealistic, since there is no perfect model nor perfect analysis.
Following this type of arguments, many of the existing approaches for quantifying predictability can be dismissed, or
at least can be deemed to lack practicality. In fact, there are only a few things that are known with absolute certainty
when it comes to quantifying the predictability of solar radiation. For instance, the latter’s most important aspect—the
double-seasonal variation in surface solar radiation—can be described to a large extent via a decent clear-sky radiation
model; such deterministic events therefore need to be excluded from predictability quantification. In other words,
predictability is to be quantified based on the clear-sky index. Secondly, predictability should generally decrease
as the forecast horizon increases. Hence, it is perfectly logical to view climatology as the “worst-case scenario.”
Stated differently, when the forecast error, regardless of which model is used, is approaching that of climatology,
predictability is lowest. Currently, there are only a handful of documents that explicitly discuss the predictability of
solar irradiance [223,224]. Hence, it would be beneficial for atmospheric scientists, who know about both clear-sky
radiation and climatology, to revisit and revise these initial ideas.
5.10. Extreme weather and peak load events
Weather conditions affect many aspects of power systems, such as electricity demand and price, renewable generation,
and transmission and distribution outages. For instance, a few severe weather-induced disruptions could drive up the
cost of firm generation significantly; this concept of firm generation is discussed in detail in Section 6. Moreover, in
contrast to regular aging of power system equipment, which is related to the system’s reliability evaluation, extreme
weather events constitute low-probability high-threat disruptions, which pertain to the evaluation of resilience; such
joint evaluation has hitherto been under-developed [see 225, for a recent advance on this subject]. Since power
engineers have to deal with the effects of weather holistically, it is desirable for atmospheric scientists and operational
meteorologists to collaborate with power engineers to investigate compound weather effects in high solar penetration
scenarios. For example, distributed generation and load information is required in net-load forecasting, which directly
assists grid operators with scheduling base load and/or energy storage facilities [e.g., 226,227]. Similar studies have
investigated the synergy or complimentary effects between solar and other renewable energy sources [e.g., 228–230],
which are useful for the long-term planning and siting of solar and wind farms to maximally reduce spatio-temporal
variability of the combined renewable resources. Due to the increased solar penetration, studies of solar eclipse
impacts are becoming more and more important to grid operations, but at least the impacted region and period is
precisely known beforehand [e.g., 231,232].
Weather is one of the major root causes to power distribution outages [e.g. 225,233,234]. Extreme weather events
such as cold spells, heat waves, wild fires, icy rain storms, or wind storms often pose great threats to the power
systems and could cause blackouts. For example, the Australian Energy Market Commission (AEMC) reports that
about 95.6% of blackouts are caused by sudden poles and wires breakdowns in the grid, which are typically caused
26
by disruptive and extreme weather events [235]. On the demand side, extreme temperature events drastically increase
peak demand due to the need for air conditioning or heating [e.g., 236]. On the generation side, heat wave events
generally reduce the production of thermal plants [e.g., 2] and solar PV and wind farms [4]. Unfortunately, extreme
weather events are hard to predict and they often occur together [e.g., heat wave and fire, or wind and rain storms,
237]. What further complicates the situation is that many extreme weather events occur when electricity is very much
needed, as what happened in the Texas freeze of February 2021 [238]. In [239], a specific method was laid out to
identify and exclude major event days when calculating reliability indices for power distribution systems.
As climate change continues to increase the frequency and intensity of extreme weather events, solar penetration
continues to rise in many countries and interconnected grids. It is thus hard to separate solar forecasting research from
the climate change context. Specific promising areas include, but are not limited to: (1) partnering with grid operators
to forecast the net load or other relevant quantities that are more relevant in grid systems with high solar penetration;
(2) investigating past power outages caused by extreme weather events towards a thorough understanding of the
underlying meteorological factors and solar forecasting improvements; and (3) tailoring solar forecasting techniques
for extreme weather conditions. Finally, because operational NWP models are normally employed to forecast extreme
weather events, the NWP improvements play a key role in improving the response from the concerned authorities to
the possible damages that extreme weather events might inflict to the power system.
6. Towards ultra-high PV penetration: Firm generation and forecasting
The possibility of having a grid with 100% renewables has been debated for a long time. Proposals, policies, and
legislation of various sorts have also been made worldwide. From a technical perspective, many obvious (but often
just conceptual ideas) for enabling ultra-high PV penetration have been well known by most, if not all, people who
work in the domain of power system engineering. For instance, allocating storage is the most apparent solution to the
problem of intermittency and variability of solar power. Different energy storage devices and technologies, such as
pumped hydro energy storage, lithium-ion battery, or supercapacitor, have different response rate and capacity, which
may help dampen the solar variabilities on different time scales, ranging from a few days to a few seconds [240].
Another popular solution is demand-side management, which is a proactive form of demand response encouraging
energy usage at off-peak hours. In a more general sense, demand-side management is a way of performing load
shaping, which refers to actions for fitting the consumption patterns to power generation [241]. Of course, there is
also forecasting: from minutes-ahead to days ahead, solar forecasts have become integral to utility operations as solar
power generation penetrates power grids. The models underlying these forecasts are becoming more refined over
the years. In particular, probabilistic forecasts that complement deterministic forecasts with condition-specific ranges
have the potential to integrate effectively with current grid management practices.
Obviously, none of the above solutions is problem-free, and it is neither ecologically nor economically favorable to
rely on a single solution [242]. Battery energy storage is costly, whereas pumped hydro is limited by environmental
and geographical constraints [243]. On the other hand, load shaping alters people’s life styles and business processes,
and not everyone is willing to respond to such strategies. Furthermore, lack of metering and communication infras-
tructure also constitutes another major barrier to load shaping [244]. Last but not least, forecasting, as has been
discussed intensively in this article, is not good enough yet. In some countries, such as China, penalties for both
under- and over-forecast can easily overturn all monetary gain from selling electricity to the grid, making investment
on PV far less attractive [32]. Overlooking these limitations may present a scenario that is too optimistic, which is
perhaps why the real-life uptake of these solutions has been slow.
6.1. Firm solar power generation
The current operational context is a “marginal PV” context whereby PV operates at the margin of a conven-
tional/dispatchable generation core—the management and economics of which necessitate operational solar forecast-
ing. Importantly, however, this marginal context is bound to evolve. As societal mandates, enabled by ever-decreasing
solar costs, push for ultra-high renewable penetration and the eventual elimination of the underlying dispatchable
conventional core, PV would soon evolve from a marginal to a grid-dominant resource. The principal solar resource
management concern must also evolve from load imbalance and markets management to meeting demand with cer-
tainty at all times—including times of low or no resource (e.g., winters and nights).
27
Meeting demand 24/365 requires enabling technologies and strategies to transform PV generation from inherently
intermittent/variable to firm. These enablers include the aforementioned energy storage, demand and supply-side
flexibility, as well as geographic dispersion, which refers to the smoothing of variability due to the cancellation of
ramp ups and downs in PV installations spread over a wide area, and optimal matching with other renewables (in
particular wind), which refers to the complementarity of different forms of renewable generation over different time
periods. Nonetheless, as the cost for solar continues to decrease, another genuine idea for solving the intermittency
and variability of solar is now possible, that is, resource overbuilding and its proactive curtailment [245,246].
The central idea in grid management is that all the power generated must be either consumed or stored. Hence, the
notion of overbuilding & curtailment has hitherto been viewed as wasteful and inefficient. In other words, overbuilding
& curtailment, in the current operational context, is something to be avoided. However, in a series of works [245,247,
248], overbuilding & curtailment has been shown to be the most effective design measure for achieving acceptably
low firm solar power generation costs. To elaborate the concept, a toy example is shown in Fig. 11. Using one year
(2020) of satellite-derived irradiance data from Harbin (45.24◦N, 126.69◦E), China, a daily PV energy generation
scenario is simulated and plotted, alongside a hypothetical constant load. Two competing strategies are shown: on
the left, a strategy using storage alone, and on the right, a strategy applying storage, PV overbuilding (by a factor of
2), and curtailment. In both figures, the blue shaded areas indicate the amount of energy storage needed. Clearly,
overbuilding by a factor of 2 doubles the cost of PV; nevertheless, the cost reduction on the required amount of energy
storage by a factor of 5.6 may easily present an overall lower cost for firm generation.
Unconstrained
Oversized by a factor of 2
Jan Feb Mar Apr May Jun Jul Aug Sep Oct Nov Dec Jan Jan Feb Mar Apr May Jun Jul Aug Sep Oct Nov Dec Jan
10
20
30
Date
Nominal daily energy production [kWh]
Figure 11: Comparing unconstrained and oversized (by a factor of 2) daily PV output relative to a hypothetical constant load (orange line), for a
period of one year. Area shaded with blue is to be fulfilled by energy storage. Overbuilding doubles the cost of PV, but reduces the cost of energy
storage by a factor of 5.6.
In energy economics, it is customary to employ the levelized cost of electricity (LCOE) as a measure of the average
net present cost of electricity generation. However, with the fundamental challenges introduced by variable and non-
dispatchable generation, one must now factor into consideration the additional cost of firming-up that generation—this
is referred to as “firm generation LCOE.” Figure 12 shows how firm-generation LCOE varies according to the fraction
of PV overbuilding. Firm-generation LCOE includes contributions from both storage and PV, and an optimal mix is
able to bring the total cost below grid parity. In this regard, one can think of the overbuilt and curtailable part of PV
as “implicit storage,” because this part enables real storage to perform its central function, i.e., to meet demand when
the solar supply is insufficient, but at a considerably reduced cost.
6.2. Firm solar power forecasting
Firm forecasting—predicting solar production ahead with zero uncertainty—constitutes an effective entry point to
eventually achieving firm power generation. The operational strategy for firm forecasting is the same as for firm
power generation, but applied on a much lighter scale—transforming PV from uncertain to fully predictable at any
28
0% 80%40% 120% 160%
PV overbuilding
Firm generation LCOE (US ¢/kWh)
0
5
10
15
20
25
30
AC
D
BFirm LCOE
Storage contribution
PV contribution
Grid parity
Figure 12: Correspondence between firm-generation LCOE and fraction of PV overbuilding. The green and orange lines represent the contribution
of PV and energy storage to the total LCOE (black line), respectively. While unconstrained variable PV (point A) is inexpensive (apparently below
grid parity), firming PV to meet demand 24/365 with storage alone (B) is unrealistically expensive. Overbuilding (and proactive curtailment) of
PV fleets reduces storage requirements to point C, where firm PV power generation can achieve true grid parity (D). Source: Perez et al. [245]
given forecast horizon [249,250]. While the underlying forecasting models are not error-free, the production of a
PV plant or a fleet of plants can be guaranteed operationally by applying the same enablers to these plants as for firm
power generation: first and foremost real storage to make up for over-forecasts, and implicit storage (overbuilding) to
safely curtail output without reducing planned production in cases of under-forecasts.
The firm forecasting strategy has been shown to be economical in several markets as current forecast uncertainties
and resulting operational costs are eliminated and replaced by the cost of additional system hardware, i.e., real and
implicit storage. Most importantly, however, the strategy is an effective practice for grid operators who become
able to transform PV generation management from marginal to central by applying the same real/implicit storage
management operations on an increasingly large scale. As PV penetration increases, the strategy can seamlessly
progress from firmly meeting forecasts to firmly meeting demand without the need for any other non-solar generators
for power system operation.
The cost of performing firm forecasting naturally acts as a valuable error metric for forecasting models; this metric is
labeled “perfect forecast metric” (PFM) [249]. Indeed, the cost of firm forecasting is a direct reflection of a forecasting
model’s operational performance: a perfect model requires no storage (i.e., zero cost), whereas a poor model requires
extensive real and implicit storage to make up for errors. This metric is effective because, unlike standard metrics such
as mean absolute error (MAE) or root mean square error (RMSE), it is directly linked to the cost of doing business.
The metric also removes the probabilistic aspect of forecasts (firm forecast are 100% accurate by definition), as well
as the need to define still elusive probabilistic assessment metrics.
It has also been shown that PFM is quite effective at revealing the key strengths and weaknesses of models that
otherwise are less apparent if using standard metrics. In particular, the smart persistence forecast is a considerably
better model, in economic operational terms, than would be apparent through standard metric assessment [249], as
illustrated in Fig. 13. Metrics, such as MAE or RMSE, quantify short-term forecast errors, whereas PFM exposes
hours- or days-persistent biases that are operationally costly. Persistence does exhibit large short-term errors, but
these errors are well balanced and do not produce persistent biases. Still, the longest and largest events with persistent
biases are those that dictate the required storage size and therefore the cost of PFM.
6.3. Solar forecasts in a future grid-dominant PV context
Firm, flexible PV power generators need to be designed to meet demand during worst-case conditions: typically,
prolonged cloudy periods in wintertime [247,248] or extreme weather events. This implies ample reserves of real
29
0
50
100
150
SUNY Persistence GFS NDFD ECMWF HRRR
Relative error [%]
MAE
PMF
Figure 13: Comparing MAEs and PFMs of 6 forecasting models. The forecasts are made for multiple locations and horizons, see [249]. A value of
100% amounts to the mean value of the considered metric for all considered models/locations/horizons. Source: Perez et al. [249].
and implicit storage. These reserves would be considerably larger than the reserves needed to handle any short-term
intra-day or multi-day over-/under-supply situation.
The accuracy and sophistication of short-term forecasts are to be guided by different criteria in this future context
because the super-flexibility of a system designed to meet worst-case supply conditions is also able to absorb any
short-term fluctuations (e.g., forecast errors) without loss of its primary function. The management of marginal
solar generation operating in a system designed for conventional generators requires the active management of all
fluctuations because supply must always meet demand, and the underlying conventional generation core needs to be
actively managed. In a future grid-dominant PV context, demand will always be met by design. This context will
have different operational forecast requirements, particularly on a macro scale. Of course, short-term forecasts will
still be used, but the current valuators (markets, penalties, imbalances, etc.) will largely disappear to be replaced by
new valuators such as battery charge/discharge optimization to reduce wear and tear.
7. Outlook
Considering the vast goal of achieving planetary carbon neutrality, the need for inter-disciplinary research has often
been mentioned, but true collaborations cannot be possible without a clear description of the technical difficulties
faced by each party. Currently, generating electricity accounts for about 27% of global carbon emission [251], which
is significant. The transition to a power sector that is dominated by renewable energy generators is well underway.
In comparison with the traditional dispatchable power generation, the intermittency and variability of solar or wind
generators cause imbalance between generation and load. This impediment was initially thought to be a major barrier
that would prevent or significantly slow down the transition. The present review shows that considerable efforts have
been deployed to alleviate those hurdles during the last few years. Overall, solar forecasting, just like the more mature
wind forecasting, belongs to the subject area of energy meteorology; it calls for a closer look at how atmospheric
scientists and power system engineers can best work together. Hence, various challenges faced by power system
engineers in regard to solar forecasting have been discussed here.
The road to good solar forecasting requires two parts. First, addressing the fundamental atmospheric physics problems
is key to improving the accuracy of base irradiance forecasts; this can only be achieved if atmospheric scientists play an
active role. Second, post-processing the base irradiance forecasts into solar power forecasts, thereby filling the various
needs for ultra-high grid penetration of solar energy, and materializing the forecasts’ value. One must therefore attach
importance to the perspective of forecast users, namely, the power system operators. In view of these, the following
outlook is provided.
(1) Satellite-derived irradiance has high potential for fulfilling the intra-day (up to 4 h) solar forecasting needs,
which is required by the real-time electricity market of most power systems. Current satellite-derived irradi-
ance databases are still mostly at the mesoscale, with hourly or sub-hourly resolution and 1- to 10-km spatial
30
resolution [see 128, for review]. Since moving towards the microscale (1 km or better and 5 min or better) will
improve sub-hourly forecast skill, issues, such as microscale aerosol assimilation, accurate and fast radiative
transfer calculation, parallax effects, or spatially-resolved transmittance of the clear-sky irradiance, still demand
improvements in technology.
(2) The revolution of areal solar forecasting, as opposed to the current practice of single-location forecasting, would
start as soon as wide-area solar energy system installations ramp up. Research topics such as spatial forecast
verification, geographical smoothing effects of irradiance [252,253], or forecast downscaling are to be tackled.
(3) Long-term (multi-day) forecast errors incur solar integration costs at ultra-high solar penetration. Multi-day-
ahead forecast heavily rely on NWP. Physical process representation, ensemble forecasting, and model initial-
ization are what have been driving the forecast skill improvements. Moving forward, however, they are also the
areas that present the more challenging questions. Based on a general review on the relatively recent progress in
NWP [15], it is evident that, at least for radiation and cloud processes and land surface models, a transition from
the current methodology of using parameterization to fully explicit models can be expected and is expected to
lead to substantial forecast skill improvements.
(4) As climate change results in an increasing occurrence of extreme weather events [1], mitigating their threats to
our daily electricity consumption via flexible reserves and firm generation is undoubtedly of value. The questions
such as “how much reserve is needed,” “what forms should the reserve be,” and “how much and where is PV
needed,” are to be answered in the near future.
(5) Ready-to-use data and software packages facilitate worldwide uptake of the accumulated scientific knowledge
in regard to atmospheric physics, which is otherwise difficult to be grasped or traced by non-physicists. As data
science is taking an increasingly vital role in all scientific domains, while programming frameworks become
more accessible, the solar forecasting community should periodically assess how to best interact with data and
tools, and make adjustments when necessary.
(6) Last but not least, regardless of what advances atmospheric scientists make concerning solar forecasting, power
system engineers need to be kept informed as rapidly and efficiently as possible. An example partnership de-
signed to involve atmospheric scientists, solar engineers, power system engineers, and various stakeholders
from the electricity industry has recently provided fruitful results in terms of efficient and directly applicable
solar forecasting [62]. Based on that successful experience, it is expected that similar efforts will be made in
other countries.
References
[1] IPCC, Summary for policymakers, in: V. Masson-Delmotte, P. Zhai, A. Pirani, S. L. Connors, C. Péan, S. Berger, N. Caud, Y. Chen,
L. Goldfarb, M. I. Gomis, M. Huang, K. Leitzell, E. Lonnoy, J. B. R. Matthews, T. K. Maycock, T. Waterfield, O. Yelekçi, R. Yu, B. Zhou
(Eds.), Climate Change 2021: The Physical Science Basis. Contribution of Working Group I to the Sixth Assessment Report of the Inter-
governmental Panel on Climate Change, Cambridge University Press, Cambridge, UK, 2021, pp. SPM–1–SPM–41.
[2] J. Añel, M. Fernández-Gonzáález, X. Labandeira, X. López-Otero, L. De la Torre, Impact of cold waves and heat waves on the energy
production sector, Atmosphere 8 (2017) 209. doi:10.3390/atmos8110209.
[3] D. Palmer, E. Koubli, I. Cole, T. Betts, R. Gottschalg, Satellite or ground-based measurements for production of site specific hourly
irradiance data: Which is most accurate and where?, Solar Energy 165 (2018) 240–255. URL: https://www.sciencedirect.com/
science/article/pii/S0038092X18302408. doi:10.1016/j.solener.2018.03.029.
[4] J. Huang, B. Jones, M. Thatcher, J. Landsberg, Temperature impacts on utility-scale solar photovoltaic and wind power generation output
over Australia under RCP 8.5, Journal of Renewable and Sustainable Energy 12 (2020) 046501. doi:10.1063/5.0012711.
[5] V. Eyring, S. Bony, G. A. Meehl, C. A. Senior, B. Stevens, R. J. Stouffer, K. E. Taylor, Overview of the Coupled Model Intercomparison
Project Phase 6 (CMIP6) experimental design and organization, Geoscientific Model Development 9 (2016) 1937–1958. URL: https:
//gmd.copernicus.org/articles/9/1937/2016/. doi:10.5194/gmd-9- 1937-2016.
[6] T. Hong, P. Pinson, Y. Wang, R. Weron, D. Yang, H. Zareipour, Energy forecasting: A review and outlook, IEEE Open Access Journal of
Power and Energy 7 (2020) 376–388. doi:10.1109/OAJPE.2020.3029979.
[7] M. Melikoglu, Vision 2023: Forecasting Turkey’s natural gas demand between 2013 and 2030, Renewable and Sustainable Energy Re-
views 22 (2013) 393–400. URL: https://www.sciencedirect.com/science/article/pii/S1364032113000798. doi:10.1016/
j.rser.2013.01.048.
[8] S. Babaei, C. Zhao, L. Fan, A data-driven model of virtual power plants in day-ahead unit commitment, IEEE Transactions on Power
Systems 34 (2019) 5125–5135. doi:10.1109/TPWRS.2018.2890714.
[9] Y. V. Makarov, C. Loutan, J. Ma, P. de Mello, Operational impacts of wind generation on California power systems, IEEE Transactions on
Power Systems 24 (2009) 1039–1050. doi:10.1109/TPWRS.2009.2016364.
[10] D. Ganger, J. Zhang, V. Vittal, Forecast-based anticipatory frequency control in power systems, IEEE Transactions on Power Systems 33
(2018) 1004–1012. doi:10.1109/TPWRS.2017.2705761.
31
[11] D. Yang, G. M. Yagli, D. Srinivasan, Sub-minute probabilistic solar forecasting for real-time stochastic simulations, Renewable and Sustain-
able Energy Reviews 153 (2022) 111736. URL: https://www.sciencedirect.com/science/article/pii/S1364032121010078.
doi:10.1016/j.rser.2021.111736.
[12] C. Sweeney, R. J. Bessa, J. Browell, P. Pinson, The future of forecasting for renewable energy, WIREs Energy and Environment 9 (2020)
e365. doi:10.1002/wene.365.
[13] Y. Wang, Q. Hu, L. Li, A. M. Foley, D. Srinivasan, Approaches to wind power curve modeling: A review and discussion, Re-
newable and Sustainable Energy Reviews 116 (2019) 109422. URL: https://www.sciencedirect.com/science/article/pii/
S1364032119306306. doi:10.1016/j.rser.2019.109422.
[14] P. A. Jimenez, J. P. Hacker, J. Dudhia, S. E. Haupt, J. A. Ruiz-Arias, C. A. Gueymard, G. Thompson, T. Eidhammer, A. Deng, WRF-Solar:
Description and clear-sky assessment of an augmented NWP model for solar power prediction, Bulletin of the American Meteorological
Society 97 (2016) 1249–1264. doi:10.1175/BAMS-D- 14-00279.1.
[15] P. Bauer, A. Thorpe, G. Brunet, The quiet revolution of numerical weather prediction, Nature 525 (2015) 47–55. doi:10.1038/
nature14956.
[16] Y. Chu, M. Li, C. F. Coimbra, D. Feng, H. Wang, Intra-hour irradiance forecasting techniques for solar power integration: A review, iScience
24 (2021) 103136. URL: https://www.sciencedirect.com/science/article/pii/S2589004221011044. doi:10.1016/j.isci.
2021.103136.
[17] A. Nguyen, M. Velay, J. Schoene, V. Zheglov, B. Kurtz, K. Murray, B. Torre, J. Kleissl, High PV penetration impacts on five local
distribution networks using high resolution solar resource assessment with sky imager and quasi-steady state distribution system simulations,
Solar Energy 132 (2016) 221–235. URL: https://www.sciencedirect.com/science/article/pii/S0038092X16002061. doi:10.
1016/j.solener.2016.03.019.
[18] T. E. Hoff, R. Perez, Modeling PV fleet output variability, Solar Energy 86 (2012) 2177–2189. URL: https://www.sciencedirect.
com/science/article/pii/S0038092X11004154. doi:10.1016/j.solener.2011.11.005.
[19] J. S. Armstrong, Principles of Forecasting: A Handbook for Researchers and Practitioners, Springer, 2001.
[20] D. Yang, J. Kleissl, C. A. Gueymard, H. T. C. Pedro, C. F. M. Coimbra, History and trends in solar irradiance and PV power forecasting:
A preliminary assessment and review using text mining, Solar Energy 168 (2018) 60–101. URL: https://www.sciencedirect.com/
science/article/pii/S0038092X17310022. doi:10.1016/j.solener.2017.11.023, Advances in Solar Resource Assessment and
Forecasting.
[21] T. Hong, P. Pinson, S. Fan, H. Zareipour, A. Troccoli, R. J. Hyndman, Probabilistic energy forecasting: Global Energy Forecasting Com-
petition 2014 and beyond, International Journal of Forecasting 32 (2016) 896–913. URL: http://www.sciencedirect.com/science/
article/pii/S0169207016000133. doi:10.1016/j.ijforecast.2016.02.001.
[22] D. Yang, A guideline to solar forecasting research practice: Reproducible, operational, probabilistic or physically-based, ensemble, and
skill (ROPES), Journal of Renewable and Sustainable Energy 11 (2019) 022701. doi:10.1063/1.5087462.
[23] T. Hong, S. Fan, Probabilistic electric load forecasting: A tutorial review, International Journal of Forecasting 32 (2016) 914–938. URL:
https://www.sciencedirect.com/science/article/pii/S0169207015001508. doi:10.1016/j.ijforecast.2015.11.011.
[24] G. Gross, F. Galiana, Short-term load forecasting, Proceedings of the IEEE 75 (1987) 1558–1573. doi:10.1109/PROC.1987.13927.
[25] H. Hippert, C. Pedreira, R. Souza, Neural networks for short-term load forecasting: a review and evaluation, IEEE Transactions on Power
Systems 16 (2001) 44–55. doi:10.1109/59.910780.
[26] T. Hong, J. Xie, J. Black, Global energy forecasting competition 2017: Hierarchical probabilistic load forecasting, International Journal of
Forecasting 35 (2019) 1389–1399. URL: https://www.sciencedirect.com/science/article/pii/S016920701930024X. doi:10.
1016/j.ijforecast.2019.02.006.
[27] A. D. Papalexopoulos, T. C. Hesterberg, A regression-based approach to short-term system load forecasting, IEEE Transactions on Power
Systems 5 (1990) 1535–1547. doi:10.1109/59.99410.
[28] P. Wang, B. Liu, T. Hong, Electric load forecasting with recency effect: A big data approach, International Journal of Forecasting 32 (2016)
585–597. URL: https://www.sciencedirect.com/science/article/pii/S0169207015001557. doi:10.1016/j.ijforecast.
2015.09.006.
[29] R. Perez, P. Ineichen, R. Seals, J. Michalsky, R. Stewart, Modeling daylight availability and irradiance components from direct and global
irradiance, Solar Energy 44 (1990) 271–289. URL: http://www.sciencedirect.com/science/article/pii/0038092X9090055H.
doi:10.1016/0038-092X(90)90055- H.
[30] D. Yang, Solar radiation on inclined surfaces: Corrections and benchmarks, Solar Energy 136 (2016) 288–302. URL: https://www.
sciencedirect.com/science/article/pii/S0038092X16302432. doi:10.1016/j.solener.2016.06.062.
[31] G. Notton, C. Voyant, A. Fouilloy, J. L. Duchaud, M. L. Nivet, Some applications of ANN to solar radiation estimation and forecasting for
energy applications, Applied Sciences 9 (2019) 209. URL: https://www.mdpi.com/2076- 3417/9/1/209. doi:10.3390/app9010209.
[32] D. Yang, W. Li, G. M. Yagli, D. Srinivasan, Operational solar forecasting for grid integration: Standards, challenges, and out-
look, Solar Energy 224 (2021) 930–937. URL: https://www.sciencedirect.com/science/article/pii/S0038092X21002826.
doi:10.1016/j.solener.2021.04.002.
[33] D. Yang, W. Wang, X. Xia, A concise overview on solar resource assessment and forecasting, Advances in Atmospheric Sciences In Press
(2022). doi:10.1007/s00376-021- 1372-8.
[34] J. Kleissl, Solar Energy Forecasting and Resource Assessment, Academic Press, 2013.
[35] D. Yang, Choice of clear-sky model in solar forecasting, Journal of Renewable and Sustainable Energy 12 (2020) 026101. doi:10.1063/
5.0003495.
[36] C. A. Gueymard, Clear-sky radiation models and aerosol effects, in: J. Polo, L. Martín-Pomares, A. Sanfilippo (Eds.), Solar Resources Map-
ping: Fundamentals and Applications, Springer International Publishing, Cham, 2019, pp. 137–182. doi:10.1007/978- 3-319- 97484-2_
5.
[37] C. A. Gueymard, Clear-sky irradiance predictions for solar resource mapping and large-scale applications: Improved validation method-
ology and detailed performance analysis of 18 broadband radiative models, Solar Energy 86 (2012) 2145–2169. URL: https://www.
32
sciencedirect.com/science/article/pii/S0038092X11004221. doi:10.1016/j.solener.2011.11.011, Progress in Solar En-
ergy 3.
[38] C. A. Gueymard, REST2: High-performance solar radiation model for cloudless-sky irradiance, illuminance, and photosynthetically active
radiation – Validation with a benchmark dataset, Solar Energy 82 (2008) 272–285. URL: http://www.sciencedirect.com/science/
article/pii/S0038092X07000990. doi:10.1016/j.solener.2007.04.008.
[39] B. Gschwind, L. Wald, P. Blanc, M. Lefèvre, M. Schroedter-Homscheidt, A. Arola, Improving the McClear model estimating the
downwelling solar radiation at ground level in cloud-free conditions – McClear-v3, Meteorologische Zeitschrift 28 (2019) 147–163.
doi:10.1127/metz/2019/0946.
[40] E. Gleeson, V. Toll, K. P. Nielsen, L. Rontu, J. Mašek, Effects of aerosols on clear-sky solar radiation in the ALADIN-HIRLAM NWP
system, Atmospheric Chemistry and Physics 16 (2016) 5933–5948. URL: https://acp.copernicus.org/articles/16/5933/2016/.
doi:10.5194/acp-16- 5933-2016.
[41] Y. Xie, M. Sengupta, J. Dudhia, A Fast All-sky Radiation Model for Solar applications (FARMS): Algorithm and performance evaluation,
Solar Energy 135 (2016) 435–445. URL: https://www.sciencedirect.com/science/article/pii/S0038092X16301827. doi:10.
1016/j.solener.2016.06.003.
[42] Z. Qu, A. Oumbe, P. Blanc, B. Espinar, G. Gesell, B. Gschwind, L. Klüser, M. Lefèvre, L. Saboret, M. Schroedter-Homscheidt, L. Wald,
Fast radiative transfer parameterisation for assessing the surface solar irradiance: The Heliosat-4 method, Meteorologische Zeitschrift 26
(2017) 33–57. URL: https://hal.archives-ouvertes.fr/hal- 01512589. doi:10.1127/metz/2016/0781.
[43] H. T. C. Pedro, D. P. Larson, C. F. M. Coimbra, A comprehensive dataset for the accelerated development and benchmarking of solar
forecasting methods, Journal of Renewable and Sustainable Energy 11 (2019) 036102. doi:10.1063/1.5094494.
[44] E. Arias-Castro, J. Kleissl, M. Lave, A Poisson model for anisotropic solar ramp rate correlations, Solar Energy 101 (2014) 192–202. URL:
https://www.sciencedirect.com/science/article/pii/S0038092X13005549. doi:10.1016/j.solener.2013.12.028.
[45] M. R. Allen, D. A. Stainforth, Towards objective probabilistic climate forecasting, Nature 419 (2002) 228.
[46] T. Gneiting, M. Katzfuss, Probabilistic forecasting, Annual Review of Statistics and Its Application 1 (2014) 125–151. doi:10.1146/
annurev-statistics- 062713-085831.
[47] T. Palmer, The ECMWF ensemble prediction system: Looking back (more than) 25 years and projecting forward 25 years, Quarterly Journal
of the Royal Meteorological Society 145 (2019) 12–24. URL: https://rmets.onlinelibrary.wiley.com/doi/abs/10.1002/qj.
3383. doi:10.1002/qj.3383.
[48] T. Gneiting, A. E. Raftery, Weather forecasting with ensemble methods, Science 310 (2005) 248–249. URL: https://www.science.
org/doi/abs/10.1126/science.1115255. doi:10.1126/science.1115255.
[49] C. W. Chow, B. Urquhart, M. Lave, A. Dominguez, J. Kleissl, J. Shields, B. Washom, Intra-hour forecasting with a total sky imager at the
UC San Diego solar energy testbed, Solar Energy 85 (2011) 2881–2893. URL: https://www.sciencedirect.com/science/article/
pii/S0038092X11002982. doi:10.1016/j.solener.2011.08.025.
[50] Z. Peng, D. Yu, D. Huang, J. Heiser, S. Yoo, P. Kalb, 3D cloud detection and tracking system for solar forecast using multiple sky
imagers, Solar Energy 118 (2015) 496–519. URL: http://www.sciencedirect.com/science/article/pii/S0038092X15002972.
doi:10.1016/j.solener.2015.05.037.
[51] P. Kuhn, B. Nouri, S. Wilbert, C. Prahl, N. Kozonek, T. Schmidt, Z. Yasser, L. Ramirez, L. Zarzalejo, A. Meyer, L. Vuilleumier, D. Heine-
mann, P. Blanc, R. Pitz-Paal, Validation of an all-sky imager–based nowcasting system for industrial PV plants, Progress in Photo-
voltaics: Research and Applications 26 (2018) 608–621. URL: https://onlinelibrary.wiley.com/doi/abs/10.1002/pip.2968.
doi:10.1002/pip.2968.
[52] P. Blanc, J. Remund, L. Vallance, Short-term solar power forecasting based on satellite images, in: G. Kariniotakis (Ed.), Renewable Energy
Forecasting, Woodhead Publishing Series in Energy, Woodhead Publishing, 2017, pp. 179–198. URL: http://www.sciencedirect.
com/science/article/pii/B9780081005040000068. doi:10.1016/B978-0- 08-100504- 0.00006-8.
[53] S. D. Miller, M. A. Rogers, J. M. Haynes, M. Sengupta, A. K. Heidinger, Short-term solar irradiance forecasting via satellite/model
coupling, Solar Energy 168 (2018) 102–117. URL: http://www.sciencedirect.com/science/article/pii/S0038092X17310435.
doi:10.1016/j.solener.2017.11.049, Advances in Solar Resource Assessment and Forecasting.
[54] V. Kallio-Myers, A. Riihelä, P. Lahtinen, A. Lindfors, Global horizontal irradiance forecast for Finland based on geostationary weather satel-
lite data, Solar Energy 198 (2020) 68–80. URL: https://www.sciencedirect.com/science/article/pii/S0038092X20300141.
doi:10.1016/j.solener.2020.01.008.
[55] E. Wu, R. E. S. Clemesha, J. Kleissl, Coastal stratocumulus cloud edge forecasts, Solar Energy 164 (2018) 355–369. URL: http:
//www.sciencedirect.com/science/article/pii/S0038092X18302056. doi:10.1016/j.solener.2018.02.072.
[56] D. K. Sahu, H. Yang, J. Kleissl, Assimilating observations to simulate marine layer stratocumulus for solar forecasting, Solar En-
ergy 162 (2018) 454–471. URL: http://www.sciencedirect.com/science/article/pii/S0038092X18300185. doi:10.1016/j.
solener.2018.01.006.
[57] A. Razagui, K. Abdeladim, K. Bouchouicha, N. Bachari, S. Semaoui, A. Hadj Arab, A new approach to forecast solar irradiances using
WRF and libRadtran models, validated with MERRA-2 reanalysis data and pyranometer measures, Solar Energy 221 (2021) 148–161. URL:
https://www.sciencedirect.com/science/article/pii/S0038092X2100311X. doi:10.1016/j.solener.2021.04.024.
[58] J. Bosch, Y. Zheng, J. Kleissl, Deriving cloud velocity from an array of solar radiation measurements, Solar Energy 87 (2013) 196–203.
URL: https://www.sciencedirect.com/science/article/pii/S0038092X12003854. doi:10.1016/j.solener.2012.10.020.
[59] D. Yang, Ultra-fast preselection in lasso-type spatio-temporal solar forecasting problems, Solar Energy 176 (2018) 788–796. URL: https:
//www.sciencedirect.com/science/article/pii/S0038092X18308120. doi:10.1016/j.solener.2018.08.041.
[60] D. van der Meer, D. Yang, J. Widén, J. Munkhammar, Clear-sky index space-time trajectories from probabilistic solar forecasts: Comparing
promising copulas, Journal of Renewable and Sustainable Energy 12 (2020) 026102. doi:10.1063/1.5140604.
[61] D. Yang, D. van der Meer, Post-processing in solar forecasting: Ten overarching thinking tools, Renewable and Sustainable Energy Re-
views 140 (2021) 110735. URL: https://www.sciencedirect.com/science/article/pii/S1364032121000307. doi:10.1016/
j.rser.2021.110735.
33
[62] S. E. Haupt, B. Kosovi´
c, T. Jensen, J. K. Lazo, J. A. Lee, P. A. Jiménez, J. Cowie, G. Wiener, T. C. McCandless, M. Rogers, S. Miller,
M. Sengupta, Y. Xie, L. Hinkelman, P. Kalb, J. Heiser, Building the Sun4Cast system: Improvements in solar power forecasting, Bulletin
of the American Meteorological Society 99 (2018) 121–136. URL: https://journals.ametsoc.org/view/journals/bams/99/1/
bams-d- 16-0221.1.xml. doi:10.1175/BAMS- D-16- 0221.1.
[63] R. H. Inman, H. T. C. Pedro, C. F. M. Coimbra, Solar forecasting methods for renewable energy integration, Progress in Energy and
Combustion Science 39 (2013) 535–576. URL: https://www.sciencedirect.com/science/article/pii/S0360128513000294.
doi:10.1016/j.pecs.2013.06.002.
[64] S. G. Benjamin, S. S. Weygandt, J. M. Brown, M. Hu, C. R. Alexander, T. G. Smirnova, J. B. Olson, E. P. James, D. C. Dowell, G. A.
Grell, H. Lin, S. E. Peckham, T. L. Smith, W. R. Moninger, J. S. Kenyon, G. S. Manikin, A North American hourly assimilation and model
forecast cycle: The Rapid Refresh, Monthly Weather Review 144 (2016) 1669–1694. doi:10.1175/MWR-D-15- 0242.1.
[65] G. Zhang, D. Yang, G. Galanis, E. Androulakis, Solar forecasting with hourly updated numerical weather prediction, Renewable and Sustain-
able Energy Reviews 154 (2022) 111768. URL: https://www.sciencedirect.com/science/article/pii/S1364032121010364.
doi:10.1016/j.rser.2021.111768.
[66] G. Buster, M. Bannister, A. Habte, D. Hettinger, G. Maclaurin, M. Rossol, M. Sengupta, Y. Xie, Physics-guided machine learning for
improved accuracy of the National Solar Radiation Database, Solar Energy 232 (2022) 483–492. URL: https://www.sciencedirect.
com/science/article/pii/S0038092X22000044. doi:10.1016/j.solener.2022.01.004.
[67] D. Yang, Validation of the 5-min irradiance from the National Solar Radiation Database (NSRDB), Journal of Renewable and Sustainable
Energy 13 (2021) 016101. doi:10.1063/5.0030992.
[68] D. Yang, C. A. Gueymard, Probabilistic post-processing of gridded atmospheric variables and its application to site adaptation of
shortwave solar radiation, Solar Energy 225 (2021) 427–443. URL: https://www.sciencedirect.com/science/article/pii/
S0038092X21004205. doi:10.1016/j.solener.2021.05.050.
[69] D. Yang, Ensemble model output statistics as a probabilistic site-adaptation tool for satellite-derived and reanalysis solar irradiance, Journal
of Renewable and Sustainable Energy 12 (2020) 016102. doi:10.1063/1.5134731.
[70] D. Yang, Ensemble model output statistics as a probabilistic site-adaptation tool for solar irradiance: A revisit, Journal of Renewable and
Sustainable Energy 12 (2020) 036101. doi:10.1063/5.0010003.
[71] J. Huang, L. J. Rikus, Y. Qin, J. Katzfey, Assessing model performance of daily solar irradiance forecasts over Australia, Solar En-
ergy 176 (2018) 615–626. URL: https://www.sciencedirect.com/science/article/pii/S0038092X18310594. doi:10.1016/
j.solener.2018.10.080.
[72] J. Huang, M. Thatcher, Assessing the value of simulated regional weather variability in solar forecasting using numerical weather prediction,
Solar Energy 144 (2017) 529–539. URL: https://www.sciencedirect.com/science/article/pii/S0038092X17300774. doi:10.
1016/j.solener.2017.01.058.
[73] B. Schulz, M. El Ayari, S. Lerch, S. Baran, Post-processing numerical weather prediction ensembles for probabilistic solar ir-
radiance forecasting, Solar Energy 220 (2021) 1016–1031. URL: https://www.sciencedirect.com/science/article/pii/
S0038092X21002097. doi:10.1016/j.solener.2021.03.023.
[74] S. Sperati, S. Alessandrini, L. Delle Monache, An application of the ECMWF Ensemble Prediction System for short-term solar power fore-
casting, Solar Energy 133 (2016) 437–450. URL: https://www.sciencedirect.com/science/article/pii/S0038092X1630041X.
doi:10.1016/j.solener.2016.04.016.
[75] S. Vannitsem, D. S. Wilks, J. Messner, Statistical Postprocessing of Ensemble Forecasts, Elsevier, 2018.
[76] G. Lee, Y. Ding, M. G. Genton, L. Xie, Power curve estimation with multivariate environmental factors for inland and offshore wind farms,
Journal of the American Statistical Association 110 (2015) 56–67. doi:10.1080/01621459.2014.977385.
[77] P. Gupta, R. Singh, PV power forecasting based on data-driven models: a review, International Journal of Sustainable Engineering 14 (2021)
1733–1755. doi:10.1080/19397038.2021.1986590.
[78] D. Yang, Estimating 1-min beam and diffuse irradiance from the global irradiance: A review and an extensive worldwide comparison of
latest separation models at 126 stations, Renewable and Sustainable Energy Reviews In Press (2022).
[79] D. Yang, C. A. Gueymard, Ensemble model output statistics for the separation of direct and diffuse components from 1-min global irradiance,
Solar Energy 208 (2020) 591–603. URL: https://www.sciencedirect.com/science/article/pii/S0038092X2030582X. doi:10.
1016/j.solener.2020.05.082.
[80] C. A. Gueymard, J. A. Ruiz-Arias, Extensive worldwide validation and climate sensitivity analysis of direct irradiance predictions
from 1-min global irradiance, Solar Energy 128 (2016) 1–30. URL: https://www.sciencedirect.com/science/article/pii/
S0038092X15005435. doi:10.1016/j.solener.2015.10.010, Special issue: Progress in Solar Energy.
[81] X. Sun, J. M. Bright, C. A. Gueymard, B. Acord, P. Wang, N. A. Engerer, Worldwide performance assessment of 75 global clear-sky
irradiance models using principal component analysis, Renewable and Sustainable Energy Reviews 111 (2019) 550–570. URL: https:
//www.sciencedirect.com/science/article/pii/S1364032119302187. doi:10.1016/j.rser.2019.04.006.
[82] X. Sun, J. M. Bright, C. A. Gueymard, X. Bai, B. Acord, P. Wang, Worldwide performance assessment of 95 direct and diffuse clear-sky
irradiance models using principal component analysis, Renewable and Sustainable Energy Reviews 135 (2021) 110087. URL: https:
//www.sciencedirect.com/science/article/pii/S1364032120303786. doi:10.1016/j.rser.2020.110087.
[83] J. J. Roberts, A. A. Mendiburu Zevallos, A. M. Cassula, Assessment of photovoltaic performance models for system simulation, Re-
newable and Sustainable Energy Reviews 72 (2017) 1104–1123. URL: https://www.sciencedirect.com/science/article/pii/
S1364032116306712. doi:10.1016/j.rser.2016.10.022.
[84] C. A. Gueymard, Direct and indirect uncertainties in the prediction of tilted irradiance for solar engineering applications, Solar En-
ergy 83 (2009) 432–444. URL: https://www.sciencedirect.com/science/article/pii/S0038092X08002983. doi:10.1016/j.
solener.2008.11.004.
[85] M. Lave, W. Hayes, A. Pohl, C. W. Hansen, Evaluation of global horizontal irradiance to plane-of-array irradiance models at locations
across the united states, IEEE Journal of Photovoltaics 5 (2015) 597–606. doi:10.1109/JPHOTOV.2015.2392938.
[86] M. J. Mayer, G. Gróf, Extensive comparison of physical models for photovoltaic power forecasting, Applied Energy
34
283 (2021) 116239. URL: https://www.sciencedirect.com/science/article/pii/S0306261920316330. doi:10.1016/j.
apenergy.2020.116239.
[87] M. J. Mayer, Effects of the meteorological data resolution and aggregation on the optimal design of photovoltaic power
plants, Energy Conversion and Management 241 (2021) 114313. URL: https://www.sciencedirect.com/science/article/pii/
S0196890421004891. doi:10.1016/j.enconman.2021.114313.
[88] M. J. Mayer, Influence of design data availability on the accuracy of physical photovoltaic power forecasts, Solar Energy
227 (2021) 532–540. URL: https://www.sciencedirect.com/science/article/pii/S0038092X2100801X. doi:10.1016/j.
solener.2021.09.044.
[89] A. H. Murphy, R. L. Winkler, A general framework for forecast verification, Monthly Weather Review 115 (1987) 1330–1338. doi:10.
1175/1520-0493(1987)115< 1330:AGFFFV>2.0.CO;2.
[90] A. H. Murphy, Forecast verification: Its complexity and dimensionality, Monthly Weather Review 119 (1991) 1590–1601. doi:10.1175/
1520-0493(1991)119< 1590:FVICAD>2.0.CO;2.
[91] A. H. Murphy, What is a good forecast? An essay on the nature of goodness in weather forecasting, Weather and Forecasting 8 (1993)
281–293. doi:10.1175/1520-0434(1993)008< 0281:WIAGFA>2.0.CO;2.
[92] I. T. Jolliffe, D. B. Stephenson, Forecast Verification: A Practitioner’s Guide in Atmospheric Science, John Wiley & Sons, 2012.
[93] D. Yang, Making reference solar forecasts with climatology, persistence, and their optimal convex combination, Solar Energy 193 (2019)
981–985. URL: http://www.sciencedirect.com/science/article/pii/S0038092X19309880. doi:10.1016/j.solener.2019.
10.006.
[94] A. H. Murphy, Climatology, persistence, and their linear combination as standards of reference in skill scores, Weather and Forecasting 7
(1992) 692–698. doi:10.1175/1520-0434(1992)007< 0692:CPATLC>2.0.CO;2.
[95] T. Gneiting, F. Balabdaoui, A. E. Raftery, Probabilistic forecasts, calibration and sharpness, Journal of the Royal Statistical Society: Series B
(Statistical Methodology) 69 (2007) 243–268. URL: https://rss.onlinelibrary.wiley.com/doi/abs/10.1111/j.1467-9868.
2007.00587.x. doi:10.1111/j.1467-9868.2007.00587.x.
[96] D. Yang, A universal benchmarking method for probabilistic solar irradiance forecasting, Solar Energy 184 (2019) 410–416. URL: http:
//www.sciencedirect.com/science/article/pii/S0038092X19303457. doi:10.1016/j.solener.2019.04.018.
[97] K. Doubleday, V. Van Scyoc Hernandez, B.-M. Hodge, Benchmark probabilistic solar forecasts: Characteristics and recommendations, Solar
Energy 206 (2020) 52–67. URL: http://www.sciencedirect.com/science/article/pii/S0038092X20305429. doi:10.1016/j.
solener.2020.05.051.
[98] J. Le Gal La Salle, M. David, P. Lauret, A new climatology reference model to benchmark probabilistic solar forecasts, Solar En-
ergy 223 (2021) 398–414. URL: https://www.sciencedirect.com/science/article/pii/S0038092X21004072. doi:10.1016/
j.solener.2021.05.037.
[99] D. van der Meer, A benchmark for multivariate probabilistic solar irradiance forecasts, Solar Energy 225 (2021) 286–296. URL: https:
//www.sciencedirect.com/science/article/pii/S0038092X2100579X. doi:10.1016/j.solener.2021.07.010.
[100] Y. V. Makarov, P. V. Etingov, J. Ma, Z. Huang, K. Subbarao, Incorporating uncertainty of wind power generation forecast into power system
operation, dispatch, and unit commitment procedures, IEEE Transactions on Sustainable Energy 2 (2011) 433–442. doi:10.1109/TSTE.
2011.2159254.
[101] D. Yang, S. Alessandrini, J. Antonanzas, F. Antonanzas-Torres, V. Badescu, H. G. Beyer, R. Blaga, J. Boland, J. M. Bright, C. F. M. Coimbra,
M. David, Â. Frimane, C. A. Gueymard, T. Hong, M. J. Kay, S. Killinger, J. Kleissl, P. Lauret, E. Lorenz, D. van der Meer, M. Paulescu,
R. Perez, O. Perpiñán-Lamigueiro, I. M. Peters, G. Reikard, D. Renné, Y.-M. Saint-Drenan, Y. Shuai, R. Urraca, H. Verbois, F. Vignola,
C. Voyant, J. Zhang, Verification of deterministic solar forecasts, Solar Energy 210 (2020) 20–37. URL: http://www.sciencedirect.
com/science/article/pii/S0038092X20303947. doi:10.1016/j.solener.2020.04.019, Special Issue on Grid Integration.
[102] P. Lauret, M. David, P. Pinson, Verification of solar irradiance probabilistic forecasts, Solar Energy 194 (2019) 254–271. URL: http:
//www.sciencedirect.com/science/article/pii/S0038092X19310382. doi:10.1016/j.solener.2019.10.041.
[103] X. Chen, Y. Du, E. Lim, H. Wen, K. Yan, J. Kirtley, Power ramp-rates of utility-scale PV systems under passing clouds: Module-level
emulation with cloud shadow modeling, Applied Energy 268 (2020) 114980. URL: https://www.sciencedirect.com/science/
article/pii/S030626192030492X. doi:10.1016/j.apenergy.2020.114980.
[104] O. Gandhi, D. S. Kumar, C. D. Rodríguez-Gallegos, D. Srinivasan, Review of power system impacts at high PV penetration Part I:
Factors limiting PV penetration, Solar Energy 210 (2020) 181–201. URL: https://www.sciencedirect.com/science/article/
pii/S0038092X20307118. doi:10.1016/j.solener.2020.06.097, Special Issue on Grid Integration.
[105] D. S. Kumar, O. Gandhi, C. D. Rodríguez-Gallegos, D. Srinivasan, Review of power system impacts at high PV penetration Part II: Potential
solutions and the way forward, Solar Energy 210 (2020) 202–221. URL: https://www.sciencedirect.com/science/article/pii/
S0038092X20308847. doi:10.1016/j.solener.2020.08.047, Special Issue on Grid Integration.
[106] P. Denholm, M. O’Connell, G. Brinkman, J. Jorgenson, Overgeneration from solar energy in California: A field guide to the duck chart,
Technical Report NREL/TP-6A20-65023, National Renewable Energy Lab.(NREL), Golden, CO (United States), 2015.
[107] Q. Hou, N. Zhang, E. Du, M. Miao, F. Peng, C. Kang, Probabilistic duck curve in high PV penetration power system: Concept, modeling,
and empirical analysis in China, Applied Energy 242 (2019) 205–215. URL: https://www.sciencedirect.com/science/article/
pii/S0306261919304763. doi:10.1016/j.apenergy.2019.03.067.
[108] S. K. Shah, L. Aye, B. Rismanchi, Simulated performance of a borehole-coupled heat pump seasonal solar thermal storage system for
space heating in cold climate, Solar Energy 202 (2020) 365–385. URL: https://www.sciencedirect.com/science/article/pii/
S0038092X20303571. doi:10.1016/j.solener.2020.03.111.
[109] Z. Ma, H. Bao, A. P. Roskilly, Feasibility study of seasonal solar thermal energy storage in domestic dwellings in the UK, Solar En-
ergy 162 (2018) 489–499. URL: https://www.sciencedirect.com/science/article/pii/S0038092X18300227. doi:10.1016/
j.solener.2018.01.013.
[110] O. J. Guerra, J. Zhang, J. Eichman, P. Denholm, J. Kurtz, B.-M. Hodge, The value of seasonal energy storage technologies for the integration
of wind and solar power, Energy & Environmental Science 13 (2020) 1909–1922. doi:10.1039/D0EE00771D.
35
[111] M. Y. Haller, D. Carbonell, M. Dudita, D. Zenhäusern, A. Häberle, Seasonal energy storage in aluminium for 100 percent solar heat
and electricity supply, Energy Conversion and Management: X 5 (2020) 100017. URL: https://www.sciencedirect.com/science/
article/pii/S2590174519300157. doi:10.1016/j.ecmx.2019.100017.
[112] E. Hirst, B. Kirby, Separating and measuring the regulation and load-following ancillary services, Utilities Policy 8 (1999) 75–81. URL:
https://www.sciencedirect.com/science/article/pii/S0957178799000119. doi:10.1016/S0957-1787(99)00011- 9.
[113] Q. Tan, Y. Ding, J. Zheng, M. Dai, Y. Zhang, The effects of carbon emissions trading and renewable portfolio standards on the integrated
wind–photovoltaic–thermal power-dispatching system: Real case studies in China, Energy 222 (2021) 119927. doi:10.1016/j.energy.
2021.119927.
[114] B. Li, J. Zhang, A review on the integration of probabilistic solar forecasting in power systems, Solar Energy 210 (2020) 68–86. URL:
https://www.sciencedirect.com/science/article/pii/S0038092X20307982. doi:10.1016/j.solener.2020.07.066, Spe-
cial Issue on Grid Integration.
[115] S. Pineda, R. Fernández-Blanco, J. M. Morales, Time-adaptive unit commitment, IEEE Transactions on Power Systems 34 (2019) 3869–
3878. doi:10.1109/TPWRS.2019.2903486.
[116] D. Yang, H. Quan, V. R. Disfani, L. Liu, Reconciling solar forecasts: Geographical hierarchy, Solar Energy 146 (2017) 276–286. URL:
https://www.sciencedirect.com/science/article/pii/S0038092X17301020. doi:10.1016/j.solener.2017.02.010.
[117] M. Emmanuel, K. Doubleday, B. Cakir, M. Markovi´
c, B.-M. Hodge, A review of power system planning and operational models for
flexibility assessment in high solar energy penetration scenarios, Solar Energy 210 (2020) 169–180. URL: https://www.sciencedirect.
com/science/article/pii/S0038092X20307489. doi:10.1016/j.solener.2020.07.017, Special Issue on Grid Integration.
[118] M. David, J. Boland, L. Cirocco, P. Lauret, C. Voyant, Value of deterministic day-ahead forecasts of PV generation in PV +storage operation
for the Australian electricity market, Solar Energy 224 (2021) 672–684. URL: https://www.sciencedirect.com/science/article/
pii/S0038092X21004862. doi:10.1016/j.solener.2021.06.011.
[119] G. Notton, M.-L. Nivet, C. Voyant, C. Paoli, C. Darras, F. Motte, A. Fouilloy, Intermittent and stochastic character of renewable energy
sources: Consequences, cost of intermittence and benefit of forecasting, Renewable and Sustainable Energy Reviews 87 (2018) 96–105.
URL: https://www.sciencedirect.com/science/article/pii/S1364032118300327. doi:10.1016/j.rser.2018.02.007.
[120] A. Kaur, L. Nonnenmacher, H. T. C. Pedro, C. F. M. Coimbra, Benefits of solar forecasting for energy imbalance markets, Renewable
Energy 86 (2016) 819–830. URL: https://www.sciencedirect.com/science/article/pii/S0960148115302901. doi:10.1016/
j.renene.2015.09.011.
[121] B. Stott, Review of load-flow calculation methods, Proceedings of the IEEE 62 (1974) 916–929. doi:10.1109/PROC.1974.9544.
[122] D. Villanueva, J. L. Pazos, A. Feijoo, Probabilistic load flow including wind power generation, IEEE Transactions on Power Systems 26
(2011) 1659–1667. doi:10.1109/TPWRS.2010.2096436.
[123] B. R. Prusty, D. Jena, A critical review on probabilistic load flow studies in uncertainty constrained power systems with photovoltaic
generation and a new approach, Renewable and Sustainable Energy Reviews 69 (2017) 1286–1302. URL: https://www.sciencedirect.
com/science/article/pii/S1364032116311066. doi:10.1016/j.rser.2016.12.044.
[124] Y. Wang, N. Zhang, Q. Chen, J. Yang, C. Kang, J. Huang, Dependent discrete convolution based probabilistic load flow for the active
distribution system, IEEE Transactions on Sustainable Energy 8 (2017) 1000–1009. doi:10.1109/TSTE.2016.2640340.
[125] M. Zubair, A. B. Awan, R. P. Praveen, M. Abdulbaseer, Solar energy export prospects of the Kingdom of Saudi Arabia, Journal of Renewable
and Sustainable Energy 11 (2019) 045902. doi:10.1063/1.5098016.
[126] H. Willis, J. Northcote-Green, Spatial electric load forecasting: A tutorial review, Proceedings of the IEEE 71 (1983) 232–253. doi:10.
1109/PROC.1983.12562.
[127] S. Nam, J. Hur, A hybrid spatio-temporal forecasting of solar generating resources for grid integration, Energy 177 (2019) 503–510. URL:
https://www.sciencedirect.com/science/article/pii/S0360544219307601. doi:10.1016/j.energy.2019.04.127.
[128] D. Yang, J. M. Bright, Worldwide validation of 8 satellite-derived and reanalysis solar radiation products: A preliminary evaluation and
overall metrics for hourly data over 27 years, Solar Energy 210 (2020) 3–19. URL: https://www.sciencedirect.com/science/
article/pii/S0038092X20303893. doi:10.1016/j.solener.2020.04.016, Special Issue on Grid Integration.
[129] C. Feng, D. Yang, B.-M. Hodge, J. Zhang, OpenSolar: Promoting the openness and accessibility of diverse public solar datasets, So-
lar Energy 188 (2019) 1369–1379. URL: https://www.sciencedirect.com/science/article/pii/S0038092X19306693. doi:10.
1016/j.solener.2019.07.016.
[130] D. Yang, SolarData: An R package for easy access of publicly available solar datasets, Solar Energy 171 (2018) A3–A12. URL: https:
//www.sciencedirect.com/science/article/pii/S0038092X18306583. doi:10.1016/j.solener.2018.06.107.
[131] D. Yang, W. Wang, T. Hong, A historical weather forecast dataset from the European Centre for Medium-Range Weather Forecasts
(ECMWF) for energy forecasting, Solar Energy 232 (2022) 263–274. URL: https://www.sciencedirect.com/science/article/
pii/S0038092X21010604. doi:10.1016/j.solener.2021.12.011.
[132] M. Roulston, L. Smith, Combining dynamical and statistical ensembles, Tellus A: Dynamic Meteorology and Oceanography 55 (2003)
16–30. doi:10.3402/tellusa.v55i1.12082.
[133] R. Perez, E. Lorenz, S. Pelland, M. Beauharnois, G. Van Knowe, K. Hemker, D. Heinemann, J. Remund, S. C. Müller, W. Traunmüller,
G. Steinmauer, D. Pozo, J. A. Ruiz-Arias, V. Lara-Fanego, L. Ramirez-Santigosa, M. Gaston-Romero, L. M. Pomares, Comparison of
numerical weather prediction solar irradiance forecasts in the US, Canada and Europe, Solar Energy 94 (2013) 305–326. URL: https:
//www.sciencedirect.com/science/article/pii/S0038092X13001886. doi:10.1016/j.solener.2013.05.005.
[134] R. Buizza, M. Leutbecher, L. Isaksen, Potential use of an ensemble of analyses in the ECMWF Ensemble Prediction System, Quarterly
Journal of the Royal Meteorological Society 134 (2008) 2051–2066. URL: https://rmets.onlinelibrary.wiley.com/doi/abs/
10.1002/qj.346. doi:10.1002/qj.346.
[135] F. Molteni, R. Buizza, T. N. Palmer, T. Petroliagis, The ECMWF Ensemble Prediction System: Methodology and validation, Quarterly
Journal of the Royal Meteorological Society 122 (1996) 73–119. URL: https://rmets.onlinelibrary.wiley.com/doi/abs/10.
1002/qj.49712252905. doi:10.1002/qj.49712252905.
[136] J. McGibbon, N. D. Brenowitz, M. Cheeseman, S. K. Clark, J. P. S. Dahm, E. C. Davis, O. D. Elbert, R. C. George, L. M. Harris, B. Henn,
36
A. Kwa, W. A. Perkins, O. Watt-Meyer, T. F. Wicky, C. S. Bretherton, O. Fuhrer, fv3gfs-wrapper: a Python wrapper of the FV3GFS
atmospheric model, Geoscientific Model Development 14 (2021) 4401–4409. URL: https://gmd.copernicus.org/articles/14/
4401/2021/. doi:10.5194/gmd-14- 4401-2021.
[137] M. Iturbide, J. Bedia, S. Herrera, J. Baño-Medina, J. Fernández, M. D. Frías, R. Manzanas, D. San-Martín, E. Cimadevilla, A. Cofiño,
J. Gutiérrez, The R-based climate4R open framework for reproducible climate data access and post-processing, Environmental Modelling &
Software 111 (2019) 42–54. URL: https://www.sciencedirect.com/science/article/pii/S1364815218303049. doi:10.1016/
j.envsoft.2018.09.009.
[138] J. Bedia, J. Baño-Medina, M. N. Legasa, M. Iturbide, R. Manzanas, S. Herrera, A. Casanueva, D. San-Martín, A. S. Cofiño, J. M. Gutiér-
rez, Statistical downscaling with the downscaleR package (v3.1.0): contribution to the VALUE intercomparison experiment, Geosci-
entific Model Development 13 (2020) 1711–1735. URL: https://gmd.copernicus.org/articles/13/1711/2020/. doi:10.5194/
gmd-13- 1711-2020.
[139] M. D. Frías, M. Iturbide, R. Manzanas, J. Bedia, J. Fernández, S. Herrera, A. S. Cofiño, J. M. Gutiérrez, An R package to visualize
and communicate uncertainty in seasonal climate prediction, Environmental Modelling & Software 99 (2018) 101–110. URL: https:
//www.sciencedirect.com/science/article/pii/S1364815217305157. doi:10.1016/j.envsoft.2017.09.008.
[140] V. E. Larson, Forecasting solar irradiance with numerical weather prediction models, in: J. Kleissl (Ed.), Solar Energy Forecasting and
Resource Assessment, Academic Press Boston, 2013, pp. 299–318.
[141] V. Spiridonov, M. ´
Curi´
c, Weather forecast and NWP, in: Fundamentals of Meteorology, Springer International Publishing, Cham, 2021, pp.
349–376. doi:10.1007/978-3- 030-52655- 9_23.
[142] T. Miyoshi, K. Kondo, K. Terasaki, Big ensemble data assimilation in numerical weather prediction, Computer 48 (2015) 15–21. doi:10.
1109/MC.2015.332.
[143] K. Bakker, K. Whan, W. Knap, M. Schmeits, Comparison of statistical post-processing methods for probabilistic NWP forecasts of solar ra-
diation, Solar Energy 191 (2019) 138–150. URL: https://www.sciencedirect.com/science/article/pii/S0038092X19308242.
doi:10.1016/j.solener.2019.08.044.
[144] J. Yang, J.-H. Kim, P. A. Jiménez, M. Sengupta, J. Dudhia, Y. Xie, A. Golnas, R. Giering, An efficient method to identify uncertainties
of WRF-Solar variables in forecasting solar irradiance using a tangent linear sensitivity analysis, Solar Energy 220 (2021) 509–522. URL:
https://www.sciencedirect.com/science/article/pii/S0038092X21002322. doi:10.1016/j.solener.2021.03.044.
[145] M. Nakanishi, H. Niino, Development of an improved turbulence closure model for the atmospheric boundary layer, Journal of the
Meteorological Society of Japan. Ser. II 87 (2009) 895–912. doi:10.2151/jmsj.87.895.
[146] E. Wu, H. Yang, J. Kleissl, K. Suselj, M. J. Kurowski, J. Teixeira, On the parameterization of convective downdrafts for marine stratocumulus
clouds, Monthly Weather Review 148 (2020) 1931–1950. URL: https://journals.ametsoc.org/view/journals/mwre/148/5/
mwr-d- 19-0292.1.xml. doi:10.1175/MWR- D-19- 0292.1.
[147] G. Thompson, T. Eidhammer, A study of aerosol impacts on clouds and precipitation development in a large winter cyclone, Jour-
nal of the Atmospheric Sciences 71 (2014) 3636–3658. URL: https://journals.ametsoc.org/view/journals/atsc/71/10/
jas-d- 13-0305.1.xml. doi:10.1175/JAS- D-13- 0305.1.
[148] D. Yang, E. Wu, J. Kleissl, Operational solar forecasting for the real-time market, International Journal of Forecasting 35 (2019) 1499–1519.
URL: https://www.sciencedirect.com/science/article/pii/S0169207019300755. doi:10.1016/j.ijforecast.2019.03.
009.
[149] M. Bessafi, V. Oree, A. Khoodaruth, G. Jumaux, F. Bonnardot, P. Jeanty, M. Delsaut, J.-P. Chabriat, M. Z. Dauhoo, Downscaling solar
irradiance using DEM-based model in young volcanic islands with rugged topography, Renewable Energy 126 (2018) 584–593. URL:
https://www.sciencedirect.com/science/article/pii/S0960148118303823. doi:10.1016/j.renene.2018.03.071.
[150] F. Antonanzas-Torres, F. J. Martínez-de Pisón, J. Antonanzas, O. Perpinan, Downscaling of global solar irradiation in complex areas in r,
Journal of Renewable and Sustainable Energy 6 (2014) 063105. doi:10.1063/1.4901539.
[151] M. S. Ghonima, H. Yang, C. K. Kim, T. Heus, J. Kleissl, Evaluation of WRF SCM simulations of stratocumulus-topped marine
and coastal boundary layers and improvements to turbulence and entrainment parameterizations, Journal of Advances in Model-
ing Earth Systems 9 (2017) 2635–2653. URL: https://agupubs.onlinelibrary.wiley.com/doi/abs/10.1002/2017MS001092.
doi:10.1002/2017MS001092.
[152] B. Maronga, M. Gryschka, R. Heinze, F. Hoffmann, F. Kanani-Sühring, M. Keck, K. Ketelsen, M. O. Letzel, M. Sühring, S. Raasch, The
Parallelized Large-Eddy Simulation Model (PALM) version 4.0 for atmospheric and oceanic flows: model formulation, recent developments,
and future perspectives, Geoscientific Model Development 8 (2015) 2515–2551. URL: https://gmd.copernicus.org/articles/8/
2515/2015/. doi:10.5194/gmd-8- 2515-2015.
[153] J. Schalkwijk, H. J. J. Jonker, A. P. Siebesma, E. V. Meijgaard, Weather forecasting using GPU-Based large-eddy simulations, Bulletin
of the American Meteorological Society 96 (2015) 715–723. URL: https://journals.ametsoc.org/view/journals/bams/96/5/
bams-d- 14-00114.1.xml. doi:10.1175/BAMS- D-14- 00114.1.
[154] B. Müller, M. Wild, A. Driesse, K. Behrens, Rethinking solar resource assessments in the context of global dimming and brightening, Solar
Energy 99 (2014) 272–282. URL: https://www.sciencedirect.com/science/article/pii/S0038092X13004933. doi:10.1016/
j.solener.2013.11.013.
[155] M. Wild, H. Gilgen, A. Roesch, A. Ohmura, C. N. Long, E. G. Dutton, B. Forgan, A. Kallis, V. Russak, A. Tsvetkov, From dimming to
brightening: Decadal changes in solar radiation at Earth’s surface, Science 308 (2005) 847–850. URL: https://science.sciencemag.
org/content/308/5723/847. doi:10.1126/science.1103215.
[156] M. Wild, Global dimming and brightening: A review, Journal of Geophysical Research: Atmospheres 114 (2009) D10. URL: https:
//agupubs.onlinelibrary.wiley.com/doi/abs/10.1029/2008JD011470. doi:10.1029/2008JD011470.
[157] M. Wild, Enlightening global dimming and brightening, Bulletin of the American Meteorological Society 93 (2012) 27–37. URL: https:
//journals.ametsoc.org/view/journals/bams/93/1/bams-d- 11-00074_1.xml. doi:10.1175/BAMS- D-11- 00074.1.
[158] J. Yoon, J. P. Burrows, M. Vountas, W. von Hoyningen-Huene, D. Y. Chang, A. Richter, A. Hilboll, Changes in atmospheric aerosol
loading retrieved from space-based measurements during the past decade, Atmospheric Chemistry and Physics 14 (2014) 6881–6902. URL:
37
https://acp.copernicus.org/articles/14/6881/2014/. doi:10.5194/acp-14- 6881-2014.
[159] X. Yang, W. Qin, L. Wang, M. Zhang, Z. Niu, Long-term variations of surface solar radiation in China from routine meteorolog-
ical observations, Atmospheric Research 260 (2021) 105715. URL: https://www.sciencedirect.com/science/article/pii/
S0169809521002714. doi:10.1016/j.atmosres.2021.105715.
[160] B. Zhao, J. H. Jiang, Y. Gu, D. Diner, J. Worden, K.-N. Liou, H. Su, J. Xing, M. Garay, L. Huang, Decadal-scale trends in regional aerosol
particle properties and their linkage to emission changes, Environmental Research Letters 12 (2017) 054021. doi:10.1088/1748-9326/
aa6cb2.
[161] J. M. Haywood, N. Bellouin, A. Jones, O. Boucher, M. Wild, K. P. Shine, The roles of aerosol, water vapor and cloud in future global
dimming/brightening, Journal of Geophysical Research: Atmospheres 116 (2011) D20. URL: https://agupubs.onlinelibrary.
wiley.com/doi/abs/10.1029/2011JD016000. doi:10.1029/2011JD016000.
[162] C. Wang, Z. Zhang, W. Tian, Factors affecting the surface radiation trends over China between 1960 and 2000, Atmospheric Environ-
ment 45 (2011) 2379–2385. URL: https://www.sciencedirect.com/science/article/pii/S1352231011001622. doi:10.1016/
j.atmosenv.2011.02.028.
[163] Y. J. Kaufman, I. Koren, L. A. Remer, D. Rosenfeld, Y. Rudich, The effect of smoke, dust, and pollution aerosol on shallow cloud
development over the atlantic ocean, Proceedings of the National Academy of Sciences 102 (2005) 11207–11212. URL: https://www.
pnas.org/content/102/32/11207. doi:10.1073/pnas.0505191102.
[164] D. Rosenfeld, U. Lohmann, G. B. Raga, C. D. O’Dowd, M. Kulmala, S. Fuzzi, A. Reissell, M. O. Andreae, Flood or drought: How do
aerosols affect precipitation?, Science 321 (2008) 1309–1313. URL: https://science.sciencemag.org/content/321/5894/1309.
doi:10.1126/science.1160606.
[165] M. Wild, D. Folini, F. Henschel, N. Fischer, B. Müller, Projections of long-term changes in solar radiation based on CMIP5 climate models
and their influence on energy yields of photovoltaic systems, Solar Energy 116 (2015) 12–24. URL: https://www.sciencedirect.com/
science/article/pii/S0038092X15001668. doi:10.1016/j.solener.2015.03.039.
[166] M. Gaetani, T. Huld, E. Vignati, F. Monforti-Ferrario, A. Dosio, F. Raes, The near future availability of photovoltaic energy in Europe
and Africa in climate-aerosol modeling experiments, Renewable and Sustainable Energy Reviews 38 (2014) 706–716. URL: https:
//www.sciencedirect.com/science/article/pii/S1364032114004936. doi:10.1016/j.rser.2014.07.041.
[167] L. Yang, J. Jiang, T. Liu, Y. Li, Y. Zhou, X. Gao, Projections of future changes in solar radiation in China based on CMIP5 cli-
mate models, Global Energy Interconnection 1 (2018) 452–459. URL: https://www.sciencedirect.com/science/article/pii/
S2096511718300653. doi:10.14171/j.2096-5117.gei.2018.04.005.
[168] A. Modak, G. Bala, Sensitivity of simulated climate to latitudinal distribution of solar insolation reduction in solar radiation management,
Atmospheric Chemistry and Physics 14 (2014) 7769–7779. URL: https://acp.copernicus.org/articles/14/7769/2014/. doi:10.
5194/acp-14- 7769-2014.
[169] National Research Council, Climate Intervention: Reflecting Sunlight to Cool Earth, The National Academies Press, Washington, DC, 2015.
doi:10.17226/18988.
[170] B. Kravitz, A. Robock, S. Tilmes, O. Boucher, J. M. English, P. J. Irvine, A. Jones, M. G. Lawrence, M. MacCracken, H. Muri, J. C.
Moore, U. Niemeier, S. J. Phipps, J. Sillmann, T. Storelvmo, H. Wang, S. Watanabe, The Geoengineering Model Intercomparison Project
Phase 6 (GeoMIP6): simulation design and preliminary results, Geoscientific Model Development 8 (2015) 3379–3392. URL: https:
//gmd.copernicus.org/articles/8/3379/2015/. doi:10.5194/gmd-8- 3379-2015.
[171] C. J. Smith, J. A. Crook, R. Crook, L. S. Jackson, S. M. Osprey, P. M. Forster, Impacts of stratospheric sulfate geoengineering on global
solar photovoltaic and concentrating solar power resource, Journal of Applied Meteorology and Climatology 56 (2017) 1483–1497. URL:
https://journals.ametsoc.org/view/journals/apme/56/5/jamc-d- 16-0298.1.xml. doi:10.1175/JAMC- D-16- 0298.1.
[172] K. Wang, Measurement biases explain discrepancies between the observed and simulated decadal variability of surface incident solar
radiation, Scientific reports 4 (2014) 6144.
[173] A. Habte, M. Sengupta, C. Gueymard, A. Golnas, Y. Xie, Long-term spatial and temporal solar resource variability over America using the
NSRDB version 3 (1998–2017), Renewable and Sustainable Energy Reviews 134 (2020) 110285. URL: https://www.sciencedirect.
com/science/article/pii/S1364032120305736. doi:10.1016/j.rser.2020.110285.
[174] A. P. Kuciauskas, P. Xian, E. J. Hyer, M. I. Oyola, J. R. Campbell, Supporting weather forecasters in predicting and monitoring saharan
air layer dust events as they impact the Greater Caribbean, Bulletin of the American Meteorological Society 99 (2018) 259–268. URL:
https://journals.ametsoc.org/view/journals/bams/99/2/bams-d- 16-0212.1.xml. doi:10.1175/BAMS- D-16- 0212.1.
[175] R. Wang, B. Liu, H. Li, X. Zou, J. Wang, W. Liu, H. Cheng, L. Kang, C. Zhang, Variation of strong dust storm events in Northern China
during 1978–2007, Atmospheric Research 183 (2017) 166–172. URL: https://www.sciencedirect.com/science/article/pii/
S0169809516303052. doi:10.1016/j.atmosres.2016.09.002.
[176] R. B. Husar, D. M. Tratt, B. A. Schichtel, S. R. Falke, F. Li, D. Jaffe, S. Gassó, T. Gill, N. S. Laulainen, F. Lu, M. C. Reheis, Y. Chun,
D. Westphal, B. N. Holben, C. Gueymard, I. McKendry, N. Kuring, G. C. Feldman, C. McClain, R. J. Frouin, J. Merrill, D. DuBois,
F. Vignola, T. Murayama, S. Nickovic, W. E. Wilson, K. Sassen, N. Sugimoto, W. C. Malm, Asian dust events of April 1998, Journal of
Geophysical Research: Atmospheres 106 (2001) 18317–18330. URL: https://agupubs.onlinelibrary.wiley.com/doi/abs/10.
1029/2000JD900788. doi:10.1029/2000JD900788.
[177] M. Chin, T. Diehl, P. Ginoux, W. Malm, Intercontinental transport of pollution and dust aerosols: implications for regional air quality,
Atmospheric Chemistry and Physics 7 (2007) 5501–5517. URL: https://acp.copernicus.org/articles/7/5501/2007/. doi:10.
5194/acp-7- 5501-2007.
[178] T. L. Zhao, S. L. Gong, P. Huang, D. Lavoué, Hemispheric transport and influence of meteorology on global aerosol climatol-
ogy, Atmospheric Chemistry and Physics 12 (2012) 7609–7624. URL: https://acp.copernicus.org/articles/12/7609/2012/.
doi:10.5194/acp-12- 7609-2012.
[179] K. H. Lee, J. E. Kim, Y. J. Kim, J. Kim, W. von Hoyningen-Huene, Impact of the smoke aerosol from Russian forest fires on the atmospheric
environment over Korea during May 2003, Atmospheric Environment 39 (2005) 85–99. URL: https://www.sciencedirect.com/
science/article/pii/S1352231004008982. doi:10.1016/j.atmosenv.2004.09.032.
38
[180] H. Yu, L. A. Remer, R. A. Kahn, M. Chin, Y. Zhang, Satellite perspective of aerosol intercontinental transport: From qualitative track-
ing to quantitative characterization, Atmospheric Research 124 (2013) 73–100. URL: https://www.sciencedirect.com/science/
article/pii/S0169809513000070. doi:10.1016/j.atmosres.2012.12.013.
[181] C. A. Gueymard, Temporal variability in direct and global irradiance at various time scales as affected by aerosols, Solar Energy
86 (2012) 3544–3553. URL: https://www.sciencedirect.com/science/article/pii/S0038092X12000291. doi:10.1016/j.
solener.2012.01.013, solar Resources.
[182] D. Rieger, A. Steiner, V. Bachmann, P. Gasch, J. Förstner, K. Deetz, B. Vogel, H. Vogel, Impact of the 4 April 2014 Saharan dust outbreak
on the photovoltaic power generation in Germany, Atmospheric Chemistry and Physics 17 (2017) 13391–13415. URL: https://acp.
copernicus.org/articles/17/13391/2017/. doi:10.5194/acp-17- 13391-2017.
[183] I. Neher, T. Buchmann, S. Crewell, B. Evers-Dietze, K. Pfeilsticker, B. Pospichal, C. Schirrmeister, S. Meilinger, Impact of atmospheric
aerosols on photovoltaic energy production scenario for the Sahel zone, Energy Procedia 125 (2017) 170–179. URL: https://www.
sciencedirect.com/science/article/pii/S1876610217336615. doi:10.1016/j.egypro.2017.08.168, european Geosciences
Union General Assembly 2017, EGU Division Energy, Resources & Environment (ERE).
[184] M. Al-Rasheedi, C. A. Gueymard, M. Al-Khayat, A. Ismail, J. A. Lee, H. Al-Duaj, Performance evaluation of a utility-scale dual-technology
photovoltaic power plant at the Shagaya renewable energy park in Kuwait, Renewable and Sustainable Energy Reviews 133 (2020) 110139.
URL: https://www.sciencedirect.com/science/article/pii/S1364032120304305. doi:10.1016/j.rser.2020.110139.
[185] M. Fathi, M. Abderrezek, P. Grana, Technical and economic assessment of cleaning protocol for photovoltaic power plants: Case
of Algerian Sahara sites, Solar Energy 147 (2017) 358–367. URL: https://www.sciencedirect.com/science/article/pii/
S0038092X17302256. doi:10.1016/j.solener.2017.03.053.
[186] A. Sayyah, M. N. Horenstein, M. K. Mazumder, Energy yield loss caused by dust deposition on photovoltaic panels, Solar En-
ergy 107 (2014) 576–604. URL: https://www.sciencedirect.com/science/article/pii/S0038092X1400259X. doi:10.1016/
j.solener.2014.05.030.
[187] R. Huva, G. Song, X. Zhong, Y. Zhao, Comprehensive physics testing and adaptive weather research and forecasting physics for day-ahead
solar forecasting, Meteorological Applications 28 (2021) e2017. URL: https://rmets.onlinelibrary.wiley.com/doi/abs/10.
1002/met.2017. doi:10.1002/met.2017.
[188] S. K. Mukkavilli, A. A. Prasad, R. A. Taylor, A. Troccoli, M. J. Kay, Mesoscale simulations of australian direct normal irradiance, featuring
an extreme dust event, Journal of Applied Meteorology and Climatology 57 (2018) 493–515. URL: https://journals.ametsoc.org/
view/journals/apme/57/3/jamc-d- 17-0091.1.xml. doi:10.1175/JAMC- D-17- 0091.1.
[189] C. A. Gueymard, P. A. Jimenez, Validation of real-time solar irradiance simulations over Kuwait using WRF-Solar, in: 12th Interna-
tional Conference on Solar Energy for Buildings and Industry, EuroSun2018, Rapperswil, Switzerland, 2018, p. 0130. doi:10.18086/
eurosun2018.09.14.
[190] E. Gilleland, D. A. Ahijevych, B. G. Brown, E. E. Ebert, Verifying forecasts spatially, Bulletin of the American Meteorologi-