ArticlePDF Available

Open-source multi-year power generation, consumption, and storage data in a microgrid

AIP Publishing
Journal of Renewable and Sustainable Energy
Authors:

Abstract and Figures

Open-source, high resolution power consumption data are scarce. We compiled, quality controlled, and released publicly a comprehensive power dataset of parts of the University of California, San Diego microgrid. The advanced microgrid contains several distributed energy resources (DERs), such as solar power plants, electric vehicles, buildings, a combined heat and power gas-fired power plant, and electric and thermal storage. Most datasets contain 15-min averages of real and reactive power from 1 January, 2015 until 29 February, 2020. We also include Python codes to fill missing data and flag and replace potentially erroneous data. The extensive dataset of conventional and new DERs is designed to accelerate research and development work in the area of sustainable microgrids.
Content may be subject to copyright.
J. Renewable Sustainable Energy 13, 025301 (2021); https://doi.org/10.1063/5.0038650 13, 025301
© 2021 Author(s).
Open-source multi-year power generation,
consumption, and storage data in a
microgrid
Cite as: J. Renewable Sustainable Energy 13, 025301 (2021); https://doi.org/10.1063/5.0038650
Submitted: 24 November 2020 • Accepted: 09 February 2021 • Published Online: 24 March 2021
Sushil Silwal, Colton Mullican, Yi-An Chen, et al.
ARTICLES YOU MAY BE INTERESTED IN
Impact of the COVID-19 pandemic on the reduction of electricity demand and the integration
of renewable energy into the power grid
Journal of Renewable and Sustainable Energy 13, 026304 (2021); https://
doi.org/10.1063/5.0045825
An analytical model of wind-farm blockage
Journal of Renewable and Sustainable Energy 13, 033307 (2021); https://
doi.org/10.1063/5.0046680
The impact of project financing in optimizing microgrid design
Journal of Renewable and Sustainable Energy 12, 065301 (2020); https://
doi.org/10.1063/5.0026187
Open-source multi-year power generation,
consumption, and storage data in a microgrid
Cite as: J. Renewable Sustainable Energy 13, 025301 (2021); doi: 10.1063/5.0038650
Submitted: 24 November 2020 .Accepted: 9 February 2021 .
Published Online: 24 March 2021
Sushil Silwal,
a)
Colton Mullican, Yi-An Chen, Avik Ghosh, John Dilliott, and Jan Kleissl
AFFILIATIONS
Center for Energy Research, University of California San Diego, La Jolla, California 92093, USA
a)
Author to whom correspondence should be addressed: ssilwal@ucsd.edu
ABSTRACT
Open-source, high resolution power consumption data are scarce. We compiled, quality controlled, and released publicly a comprehensive
power dataset of parts of the University of California, San Diego microgrid. The advanced microgrid contains several distributed energy
resources (DERs), such as solar power plants, electric vehicles, buildings, a combined heat and power gas-fired power plant, and electric and
thermal storage. Most datasets contain 15-min averages of real and reactive power from 1 January, 2015 until 29 February, 2020. We also
include Python codes to fill missing data and flag and replace potentially erroneous data. The extensive dataset of conventional and new
DERs is designed to accelerate research and development work in the area of sustainable microgrids.
V
C2021 Author(s). All article content, except where otherwise noted, is licensed under a Creative Commons Attribution (CC BY) license (http://
creativecommons.org/licenses/by/4.0/).https://doi.org/10.1063/5.0038650
I. INTRODUCTION
The modern electrical power grid is undergoing a massive restruc-
turing mostly due to integration of renewable and distributed energy
resources (DERs) to reduce our dependence on fossil fuels.
1
This transi-
tion of the power grid into a clean energy infrastructure requires exten-
sive research on emerging grid technologies, and it is partly limited by
the unavailability of practical energy generation, usage, and storage
data.
2,3
This lack of data in the research community is thought to be for
two reasons: (i) limited real-world adoption of DERs and (ii) power
utilities and customers are not willing to publish their data for public
consumption for reasons of privacy and competitiveness.
There are some publicly available DER datasets. Twenty four of
the available datasets are reviewed by Kapoor et al.
4
Most impactful
and notable among them is the Pecan Street data that contain energy
usage, EV charging, rooftop solar generation, and energy storage data
collected from more than 1000 submetered, mostly residential build-
ings located in Pecan Street in Texas, with time steps ranging from 1 s
to 15 min.
5
The Open Power System Data
6
and ENTOS-E
7
contain
aggregated power system data, mostly of European countries, but lack
building level and EV data. The work in Ref. 8presents five years of 1 s
power data of a small microgrid with a rooftop solar PV generator
(91kW),leadacidbatterystorage(326kWh,90kW),anemergency
back-up generator, and a single research building. The 1 min energy
consumption data of a seven-story office building in Bangkok are col-
lected in Ref. 9and cover 18 months. All these datasets lack one or
more of the following elements, all of which are essential components
of most commercially viable microgrids: multiple types of DERs and
multiple commercial buildings. We intend to fill this gap in data avail-
ability with data from 13 buildings and multiple DERs connected to a
microgrid at the University of California (UC), San Diego campus.
This will facilitate and accelerate research applying model predictive
control and optimization, machine learning, and other statistical meth-
ods to microgrid design (e.g., Ref. 10), simulated microgrid operation
(e.g., Ref. 11), and smart charging of electric vehicles (e.g., Ref. 12).
The remainder of this work is organized as follows. Section II
provides a brief introduction of UC San Diego’s microgrid for the con-
text of the collected data. Section III discusses the database. Section IV
provides guidelines for filling missing and replacing erroneous data
points. Section Vconcludes the paper with merits of published data.
II. MICROGRID OVERVIEW
The core content of this paper is the power generation, consump-
tion, and storage data from parts of the UC San Diego microgrid. The
microgrid serves the main campus at 9500 Gilman Drive, La Jolla,
California 92093, and includes the Scripps Institution of
Oceanography. UC San Diego has been at the forefront of clean energy
solutions.
13
As one of the most advanced microgrids in the world, the
UC San Diego hosts a central natural gas fired plant with two high effi-
ciency 13.5 MW combined cycle co-generation Solar Turbines Titan
130 turbines and a 3 MW Dresser-Rand steam turbine, 10 million gal-
lons of chilled thermal energy storage, 3 MW distributed solar PV gen-
erators, a 2.8MW fuel cell that is the largest on a US college campus,
J. Renewable Sustainable Energy 13, 025301 (2021); doi: 10.1063/5.0038650 13, 025301-1
V
CAuthor(s) 2021
Journal of Renewable
and Sustainable Energy ARTICLE scitation.org/journal/rse
2.5 MW battery energy storage systems, 125 electric vehicle charging
stations (many with dual ports), and energy efficient campus buildings
with controllable loads. Overall, UC San Diego self-generates about
85% of its electricity consumption and imports the remaining 15%
from the local utility, San Diego Gas & Electric (SDG&E).
UC San Diego also operates the 184,025 ft
2
logistics warehouse at
7835 Trade Street, San Diego, California 92121, which acts as the ship-
ping & receiving hub and storage location for the university. It is
equipped with a 233 kW solar PV system.
Selected data pertaining to different generators, loads, storage,
and grid imports are released in this paper. While there are more
metered data available than what is included in this data release—since
not all buildings are metered—we believe that a comprehensive dataset
would provide relatively little additional value. Most microgrids con-
sist only of a few buildings, and they can be represented with the data
released here.
An important caveat for these data is that cooling and heating
energy is not included in the building energy use, while the electric
power consumed by air handlers for ventilation is included. The cam-
pus operates a district heating and district cooling system. Waste heat
from the co-generation plant is captured and used to provide heat to
campus buildings. Waste heat is also converted to chilled water
through absorption chillers. The central chilling capacity is augmented
with electric chillers and differences in chilled water supply, and
demand can be managed with the thermal energy storage tank.
Chilled and hot water consumption to individual buildings is metered,
but it is not stored and therefore cannot be provided. As a proxy, we
provide volumetric flow rates and temperatures of hot and chilled
water supply from the central plant to the entire campus. A detailed
description of these data is included in Appendix B.
III. DATA RECORDS
A. Campus operations
The UC San Diego microgrid has utility-grade Schneider Electric
ION electricity meters on 70% of the campus’ generators, storage sys-
tems, grid import, and building loads. These meters monitor energy and
power quality and interface with an existing PowerLogic ION Enterprise
web-enabled software system. The data are live-streamed to the UC San
Diego intranet. The average of the preceding 15min is archived as a sin-
gle data point, i.e., a total of 96 data rows are saved per day for each
meter. All available data starting from 1 January, 2015 to 29 February,
2020 are downloaded as comma separated value (csv) files. Building
loads on a university campus are unique due to the academic year.
Depending on the building occupancy, loads can be substantially lower
during the summer break, the winter holidays, and spring break as most
students and teaching staff will be absent. However, research and depart-
ment operations continue during academic breaks, except at the end of
the year when UC San Diego institutes a complete campus closure for
about one week starting around December 24 and ending around
January 1. UC San Diego academic calendars are available in Ref. 14.
All data sources are presented in Table 1 and Appendix A,and
their physical locations are mapped in Figs. 2 and 3and Appendix B.
These data are broadly classified into seven groups.
B. Buildings
Real and reactive power consumption for the following campus
buildings is compiled:
1. Buildings without EV charging
Robinson Hall, Pepper Canyon Hall, Student Services Center,
Galbraith Hall, Geisel Library, Center Hall, Social Science Research,
Otterson Hall, East Campus Office Building, Economics, Music, and
Mandeville Center.
2. Building with EV charging
Hopkins Parking Structure and Police Department. The total
load data in the facility are separated into building power and EV
charging power consumption data.
A sample building load timeseries is illustrated in Fig. 1.The
upper subplot shows the entire real power consumption data for
Pepper Canyon Hall with missing data and a potential error marked
(see Sec. IV B). The lower subplot shows repetitive daily and weekly
power consumption trends.
C. Trade Street Warehouse microgrid
The real net load data for Trade Street Warehouse is from the
utility meter, which reports the building load minus the (behind the
meter) solar power generation. The Trade Street Warehouse microgrid
also includes a 200 kW/400 kWh BESS and a 10 kW V2G EV charging
station, both behind the meter. The solar power generated and BEES
datasets are also compiled and reported.
D. EV charging stations
As of August 1, 2020, there are 210 ChargePoint EV charging
ports of which four are DC fast chargers. These charging stations have
been installed over the years since 2017. The EV charging dataset con-
tains transactions from all ChargePoint
V
R
stations including charging
station details, charging duration with start and end time, and energy
consumed. The datasets of the Hopkins Parking Structure in Sec. III B
consist of separate aggregate timeseries of EV charging and associated
FIG. 1. Real power consumption time series data for the Pepper Canyon Hall cam-
pus building for (top) the complete dataset and (bottom) ten days during the aca-
demic year.
Journal of Renewable
and Sustainable Energy ARTICLE scitation.org/journal/rse
J. Renewable Sustainable Energy 13, 025301 (2021); doi: 10.1063/5.0038650 13, 025301-2
V
CAuthor(s) 2021
building power consumption at 15 min resolution, while this section
contains individual EV charging events at the Hopkins, Gilman, and
Osler Parking Structure. The EV state of charge (SOC) at the beginning
of charging is not available but can be backed out following Ref. 12.
E. Solar PV generators
Since all solar inverters operate at the unity power factor follow-
ing the 2003 IEEE 1547 standard, only real AC power generation data
for 26 on-campus PV plants are provided. Some PV systems such as
in Leichtag Biomedical Research experienced temporary decreases in
power due to unknown hardware issues.
F. Campus thermal load and storage
The microgrid operates a natural gas fired combined heat &
power plant that provides district heating and cooling to most build-
ings on the campus. The plant consists of two 13.5 MW natural gas
turbines, a steam generator, electric chillers, and a chilled water tank
for thermal energy storage. Since the building electric load data only
include fan ventilation power, but not cooling (or heating) load, we
include campus-wide chiller plant electricity consumption, chilled
water flow, cooling tons, and the chilled water tank capacity.
G. Demand charge related data
Demand charges are peak power charges ($/kW) for the highest
monthly consumption in any 15 min interval (non-coincident
charges) and the highest consumption during the peak period
(1600–2100 h every day). Since the campus has on-site generation,
the noncoincident demand charge calculation is complex. Therefore,
all data required to calculate demand charges are included in this sec-
tion. This includes the SDG&E real power import (which is the rele-
vant data for peak demand charges), on-campus generation (to
compute the actual campus demand), and adjusted demand (to com-
pute noncoincident demand charges).
H. Battery energy storage system (BESS)
UC San Diego owns a 2.5 MW, 5 MWh BESS, which has primar-
ily been used for demand charge management. Starting July 1, 2020,
the BESS also occasionally participates in the California Independent
System Operator (CAISO) demand response auction market. The load
power consumed while charging the battery is represented in positive
kW, and the generated power during discharging operation is repre-
sented in negative kW.
IV. MISSING DATA AND DATA QUALITY CONTROL
A. Missing data
Missing data fractions are shown in Table I and Appendix A.
Since redundant meters are not available, the actual values for the
missing data points are unknown and an imputation model is
required. Reasons for missing data are as follows: (i) meter outage, (ii)
data loss during transmission from meter to meter server, or (iii) meter
server error. Consistent with the commercial operation of the meters,
the reasons for missing data have not been logged and are unknown.
A simple imputation algorithm based on persistence will be discussed
in Subsection Vand is provided with the data.
Python scripts to find and fill the missing data and outliers as tab-
ulated in Table II and Appendix A are provided in the repository
alongside the raw datasets.
B. Data quality control
Data quality control was also conducted. Since building loads
result from the actions of hundreds of human occupants and automated
building controllers, data errors are difficult to ascertain. Generally,
errors in industrial grade electric meters are highly unlikely and small.
Nevertheless, the database is checked for incorrectly recorded data based
on unusual values (data range) and unusual changes in values (data
ramps). Usually—with the exception of solar power and BESS data
where cloud cover can induce large ramps over 15 min intervals—power
generation, load, and thermal storage data follow daily, weekly, or yearly
periodicity and experience only small ramps. Thus, sudden large ramps
for a brief period of time and large deviations from the periodic pattern
may be erroneous unless they can be explained by corresponding physi-
caleventssuchasafault.Erroneousdatapointsarelocatedasfollows:
Step I: Since all buildings have persistent loads such as lighting
and plug loads, electric power is never zero. Except for PV gener-
ators and BESS, zero readings or close to zero (<2 kW) are
marked as NaN data points. The reactive power consumption
data for Hopkins Parking Structure is exempted from this recom-
mendation as there are data points around zero.
Step II: Filter the data using a Gaussian filter with a suitable win-
dow size in the range of 10–25. The Gaussian filter is chosen for
smoothing the data such that an actual data point sufficiently far
from the expected value can be categorized as the outlier. The
window size (which is called the “standard deviation” in the
python code) and distance from the expected value are calibrated
iteratively such that outliers are detected with high confidence.
Step III: Locate outliers that are sufficiently far from the filtered
data: Let P(t) be raw data and ~
PðtÞbe filtered data at time t.If
jPðtÞ~
PðtÞj >Pt,thenP(t) is flagged. The flagged data points
are visually examined, and the permissible threshold P
t
is recali-
brated such that the remaining outliers can be categorized as errors
with high confidence. Figure 1 shows an example of an error for
the Pepper Canyon Hall real power consumption data. A total of
235 possible error data points are identified across all the datasets,
which can be replaced with suitable values as described in Sec. V.
V. FILLING MISSING DATA AND REPLACING OUTLIERS
The NaN, missing, or erroneous data points are replaced or filled
with the most recent valid data at the same time of day from the near-
est earlier day of the same type (business or non-business). For
instance, a data error at 1500 on Monday is replaced by data at 1500
on the preceding Friday (assuming that both days are business days).
If the missing data source day is unavailable, which was the case dur-
ing a few days at the beginning of the dataset in early January 2015,
the missing data and outliers are replaced with the most recent valid
data. Since both raw and quality-controlled data are provided, the user
can apply their own imputation methods.
The data quality control methods are basic but allow users to
quickly generate complete data for a period of around 5years, which—
given the relatively small amount of missing data and outliers—will
suffice for many applications such as microgrid sizing and dispatch. In
Journal of Renewable
and Sustainable Energy ARTICLE scitation.org/journal/rse
J. Renewable Sustainable Energy 13, 025301 (2021); doi: 10.1063/5.0038650 13, 025301-3
V
CAuthor(s) 2021
certain research areas such as forecasting, the data quality control
methodspresented here may not be acceptable and researchers should,
then, only use complete subsets of the data or write their own code
with sophisticated algorithms to populate the missing data points.
VI. CONCLUSIONS
We have compiled and released power system data of diverse
generation, consumption, and storage devices of the UC San Diego
microgrid. These includes datasets for buildings and building com-
plexes, EV charging stations, solar PV generators, and thermal energy
storage and load. Furthermore, the total power generation at UC San
Diego, imported power from local utility, adjusted UC San Diego
demand load, and UC San Diego peak demand are also included to
facilitate research in demand charge reduction. Recognizing the rarity
of large scale power system data in the scientific community, we intend
to serve the data needs of fellow researchers and accelerate research
and development work in the area of sustainable power grids.
Researchers in the area of emerging grid applications, especially opti-
mization or forecasting studies, will benefit from the published data.
SUPPLEMENTARY MATERIAL
See the supplementary material fordatasetsandscriptsas
described in Appendix A are the crux of the paper and are released
under a Creative Commons (CC) license at the open-access repository
at https://github.com/sushilsilwal3/UCSD-Microgrid-Database.
ACKNOWLEDGMENTS
The authors would like to thank the UC San Diego Vice
Chancellor of Resource Management and Planning Gary Matthews,
the Associate Vice Chancellor Steve Jackson, the Director of
Strategic Energy Initiatives Byron Washom, the Energy & Utilities
manager John Dilliott, and the facility management team including
Robert Austin and others for publishing the data to the campus
community and for allowing us to release the data.
APPENDIX A: DATA REPOSITORY AND CODE
All data and Python codes are tabulated in Table I. All numeri-
cal data files are in comma separated values (.csv) format. Missing
TABLE I. Metadata of the released data. Unk—unknown and NA—not applicable.
Facility/function PV (DC, AC kW) or Battery Rating Data file name Start date End date Missing days
EV charging stations NA Chargepointev 15 March, 2016 29 Feb, 2020 None
Thermal load/storage NA Thermalstorage.csv 12 May, 2016 29 Feb, 2020 None
Demand charges NA Demandcharge.csv 01 Jan, 2015 29 Feb, 2020 none
BYD battery energy storage 2.5 MW, 5.0 MWh Batterystorage.csv 30 Nov, 2015 29 Feb, 2020 8.1
Campus Buildings
Robinson Hall NA Robinsonhall.csv 01 Jan, 2015 29 Feb, 2020 412.8
Pepper Canyon Hall NA Peppercanyon.csv 01 Jan, 2015 29 Feb, 2020 36.5
Student Service Center NA Studentservices.csv 05 April, 2016 29 Feb, 2020 58.9
Social Sciences NA Socialscience.csv 01 Dec, 2016 29 Feb, 2020 0.5
Galbraith Hall NA Galbraithhall.csv 01 Jan, 2015 29 Feb, 2020 0.4
Geisel Library NA Geisellibrary.csv 29 Aug, 2017 29 Feb, 2020 0.2
Center Hall NA Centerhall.csv 07 Jan, 2016 29 Feb, 2020 0.25
East Campus Office NA Eastcampus.csv 08 July, 2017 29 Feb, 2020 0.2
Mandeville Center NA Mandeville.csv 01 Jan, 2015 29 Feb, 2020 1.2
Hopkins Parking Building NA Hopkinsbuilding.csv 01 April, 2019 01 Jan, 2020 None
Hopkins Parking EV NA Hopkinsev.csv 01 April, 2019 01 Jan, 2020 None
Police Department Building NA Policebuilding.csv 01 June, 2020 10 Oct, 2020 None
Police Department EV NA Policeev.csv 01 June, 2020 10 Oct, 2020 None
Otterson Hall NA Ottersonhall.csv 01 Jan, 2015 29 Feb, 2020 265.4
Music Building NA Musicbuilding.csv 28 Aug, 2015 29 Feb, 2020 0.2
Rady Hall NA Radyhall.csv 12 Feb, 2015 29 Feb, 2020 0.4
Trade Street Off-campus Microgrid
Trade Street Warehouse NA Tradestreettotal.csv 31 Aug, 2016 26 July, 2019 25.6
Trade Street PV 233, Unk Tradestreetpv.csv 08 Feb, 2016 29 Feb, 2020 38.6
Trade Street Battery 200 kW, 400 kWh Tradestreetbattery.csv 25 Oct, 2016 09 Sep, 2018 38.6
On-Campus Solar PV Generators
Biomedical Sciences Library Unk, 390 Bsb_librarypv.csv 21 Jan, 2015 29 Feb, 2020 0.5
Journal of Renewable
and Sustainable Energy ARTICLE scitation.org/journal/rse
J. Renewable Sustainable Energy 13, 025301 (2021); doi: 10.1063/5.0038650 13, 025301-4
V
CAuthor(s) 2021
data entries are left unfilled, and outliers as identified per Sec. IV B
are marked as NaN.
We have included Python codes as tabulated in Table II. The
codes read the data, find outliers, and replace outliers and missing
data points with a suitable value as described in Sec. V.
APPENDIX B: PHYSICAL LOCATION OF DATA
SOURCES
The building locations are shown in Fig. 2, and the solar PV
generator locations are shown in Fig. 3.
TABLE I. (Continued.)
Facility/function PV (DC, AC kW) or Battery Rating Data file name Start date End date Missing days
Biomedical Sciences Building 284, Unk bsb_buildingpv.csv 21 Jan, 2015 30 Aug, 2018 0.2
Bio-Engineering Hall Unk, 74 Bioengineeringpv.csv 21 Jan, 2015 29 Feb, 2020 26.7
Campus Service Complex Unk, 54 Csc_buildingpv.csv 15 Jan, 2016 29 Feb, 2020 49.4
Central Utility Plant Unk, 65 Cup_pv.csv 01 Jan, 2015 29 Feb, 2020 0.4
Engineering Building Unit II 43, 35 Ebu2_a_pv.csv 27 April, 2015 29 Feb, 2020 0.3
Engineering Building Unit II 37, 31 Ebu2_b_pv.csv 27 April, 2015 29 Feb, 2020 0.3
Electric Shop Unk, Unk Electricshoppv.csv 24 Oct, 2015 29 Feb, 2020 0.2
Fleet Services 29, 24 Garagefleetspv.csv 18 Mar, 2016 29 Feb, 2020 0.4
Gilman Parking 195, 200 Gilmanparkingpv.csv 09 May, 2015 29 Feb, 2020 812.1
Hopkins Parking 338, 350 Hopkinsparkingpv.csv 29 Aug, 2015 29 Feb, 2020 1.0
Keeling Apartments 41, Unk Keelinga_pv.csv 15 May, 2017 29 Feb, 2020 0.1
Keeling Apartments Unk, Unk Keelingb_pv.csv 15 May, 2017 29 Feb, 2020 0.1
Otterson Hall 18, Unk Kyoceraskylinepv.csv 14 Feb, 2016 29 Feb, 2020 200.7
Leichtag Biomedical Research Unk, 50 Leichtagpv.csv 01 Jan, 2015 29 Feb, 2020 0.3
MESOM Laboratory 61, Unk Mesom_pv.csv 31 Mar, 2016 29 Feb, 2020 14.3
Mayer Hall Unk, 120 Mayerhallpv.csv 01 Jan, 2015 29 Feb, 2020 0.4
Oslar Parking 268, Unk Oslerparkingpv.csv 17 Dec, 2018 29 Feb, 2020 185.4
Price Center 63, 75 Pricecentera_pv.csv 27 April, 2015 29 Feb, 2020 0.3
Price Center 66, 75 Pricecenterb_pv.csv 23 May, 2015 29 Feb, 2020 0.2
SD Supercomputing Center Unk, 65 Sdsc_pv.csv 01 Jan, 2015 29 Feb, 2020 0.3
Structural and Material Engineering Unk, 120 Sme_solarpv.csv 14 Oct, 2016 29 Feb, 2020 0.1
Powell Structural Research Lab 6.5, Unk Powellpv.csv 01 Jan, 2015 03 Mar, 2016 0.1
Birch aquarium 49, Unk Stephenbirchpv.csv 12 April, 2016 29 Feb, 2020 0.1
FIG. 2. Location of buildings and facilities: (1) Robinson Hall, (2) Pepper Canyon
Hall, (3) Student Services Center, (4) Social Sciences, (5) Galbraith Hall, (6) Geisel
Library, (7) Center Hall, (8) East Campus Office, (9) Mandeville Center, (10) Gilman
Parking, (11) Hopkins Parking, (12) Police Department, (13) Otterson Hall, (14) Music
Building, (15) Rady Hall, (16) Trade Street Warehouse, and (17) Central Utility Plant.
TABLE II. Python scripts to find and impute missing data and find potential outliers.
Since the thermal storage, EV charging, and demand charge data are already pro-
vided in a quality controlled form, no Python script is provided. In the EV charging
dataset, charging events with 0 kWh consumption were removed as they were likely
caused by operator error.
Script file name Corresponding data
PythonBuildingLoad.py Building load and trade street total
PythonPVGenerator.py Solar PV generator
PythonBatteryStorage.py Campus/trade street battery storage
Journal of Renewable
and Sustainable Energy ARTICLE scitation.org/journal/rse
J. Renewable Sustainable Energy 13, 025301 (2021); doi: 10.1063/5.0038650 13, 025301-5
V
CAuthor(s) 2021
APPENDIX C: DISTRICT COOLING ANALYSIS FOR
UC SAN DIEGO
UC San Diego uses district heating and cooling for building
thermal regulation. Due to mild temperatures and prevalence of
office buildings with large internal heat generation from equipment
and people, the district heating system uses significantly less energy
than the district cooling system. Therefore, this analysis focuses on
the district cooling system. The district cooling system consists of a
combined heat and power plant (CHP), a thermal energy storage
(TES) tank, and pipes that connect the tank to the campus build-
ings. Electric chillers are available to augment cooling capacity dur-
ing peak times. All variables are metered using a Schneider ION
metering system. The data streams are shown in Table III. The pur-
pose of this document is to describe this database, provide consis-
tency checks/validations, and present a modified data file that
contains the key variables.
All thermodynamic processes and variables are illustrated
in Figs. 4 and 5.Figure 5 shows a thermodynamic analysis of
the TES tank. The TES tank is cooled by the electric chiller and
by an absorption chiller attached to the cogeneration plant. It
loses heat to campus buildings through the chilled water loop
as manifested by the difference in supply and return flow
temperatures.
The TES tank is a closed system. Therefore, the mass flow rate
of water supplied from the TES is equal to the mass flow rate of
water returned to the TES, i.e., _
msupply ¼_
mreturn. The conservation
of energy dictates that
FIG. 3. Location of PV generators: (1) Biomedical Sciences Library, (2) Biomedical
Sciences Building, (3) Bio-engineering Hall, (4) Campus Services Complex, (5)
Central Utility Plant, (6) Engineering Building Unit II, (7) Electric Shop, (8) Fleet
Services, (9) Gilman Parking, (10) Hopkins Parking, (11) Keeling Apartments, (12)
Otterson Hall, (13) Leichtag Biomedical Research, (14) Mayer Hall, (15) Osler
Parking, (16) MESOM Laboratory, (17) Price Center, (18) SD Supercomputing
Center, (19) Structural and Material Engineering, (20) Birch Aquarium, (21) Power
Structural Research Lab, and (22) Trade Street Warehouse.
FIG. 4. Thermodynamic analysis of the chilled water loop with the control volume
being the thermal energy storage tank.
TABLE III. Variable names in the UC San Diego campus data files, derived variables, and input parameters related to thermal states and processes.
Variable name Units Definition Variable name in data file
qlb/gallon Density of water ¼8.343 lb/gallon N/A
hreturn;hsupply BTU/lb The enthalpy value as determined by the return/supply
temperature of the water at the TES. hreturn ¼10:72
BTU/lb and hsupply ¼21:07 BTU/lb
N/A
_
QTES BTU/h Total chilling power provided to campus buildings
from the thermal energy storage tank
CHW MBTU
_
Qchiller BTU/h Total chilling power provided to the TES tank from the
electric chiller
N/A
_
msupply;_
mreturn lb/h Mass flow rate of water supplied from the TES to the
campus buildings/returned to the TES tank
N/A
_
Vsupply Gallons/h Volumetric flow rate of water supplied from the TES to
the campus buildings
CHW supply flow
_
Vreturn Gallons/h Volumetric flow rate of water returned to the TES CHW return flow
Journal of Renewable
and Sustainable Energy ARTICLE scitation.org/journal/rse
J. Renewable Sustainable Energy 13, 025301 (2021); doi: 10.1063/5.0038650 13, 025301-6
V
CAuthor(s) 2021
dETES
dt ¼_
Qchiller þ_
mreturnhreturn _
msupplyhsupply ;(C1)
where ETES is in BTU/hour, _
mreturn=supply are in lb/hour, and
hreturn=supply is the enthalpy of water in BTU/lb.
Figure 5 shows the thermodynamic states and processes in the
chilled water loop. The chilled water loop splits into five different
pipes. One pipe serves to cool the air at the gas turbine inlet. The
remaining four pipes serve different neighborhoods of the campus
(Muir, School of Medicine, Revelle, and Galbraith). Within each
neighborhood, the cooling system of each building dumps heat into
the chilled water through a heat exchanger.
Given the closed system in Fig. 5,_
msupply ¼_
mreturn. The con-
servation of energy yields
dEneighborhood
dt ¼_
Qbuildings þR_
mihi;in R_
mihi;out;(C2)
where _
Qbuildings is the heat energy added to the control volumes by
all buildings in all neighborhoods in BTU/hour, _
miis the mass flow
rate through each neighborhood’s control volume iin lb/gallon, and
hi;in=i;outlet is the enthalpy of saturated water in BTU/lb as
determined from the given temperature. The sum over irepresents
the sum over the five control volumes for the neighborhoods.
Assuming the control volume to be in a steady state, we take
dEneighborhood
dt ¼0 and solve for _
Qbuildings as follows:
_
Qbuildings ¼R_
mihi;out R_
mihi;in:(C3)
DATA AVAILABILITY
This paper is based on the raw data collected at the UC San
Diego, and the data are released publicly as a supplementary material
to the paper.
REFERENCES
1
M. Moretti, S. N. Djomo, H. Azadi, K. May, K. De Vos, S. Van Passel, and N.
Witters, “A systematic review of environmental and economic impacts of smart
grids,” Renewable Sustainable Energy Rev. 68, 888–898 (2017).
2
V. Potdar, A. Chandan, S. Batool, and N. Patel, “Big energy data management
for smart grids-issues, challenges and recent developments,” in Smart Cities
(Springer, 2018), pp. 177–205.
TABLE III. (Continued.)
Variable name Units Definition Variable name in data file
_
Wchiller;in BTU/h Chiller electric power consumption (work) Total chiller plant Elec.
N/A Tons Cooling capacity of the TES return flow Tot CHW Tons (Ret Flo)
N/A Tons Cooling capacity of the TES supply flow Tot CHW Tons (Sup Flo)
_
Qbuildings BTU/h The respective heat energy contribution to the chilled
water flow from each main campus neighborhood
N/A
FIG. 5. Architecture and thermodynamic layout of the district cooling system. Control volumes cover the chilled water flow and heat exchange in the campus neighborhoods.
The flow through each neighborhood is treated as a closed, steady state system. Heat is produced in the building through solar heating through windows, conduction through
the envelope, human body heat, and dissipated heat from electric power consumption. In the steady state, the net heat from the building processes is added to the system
through a heat exchanger at each building.
Journal of Renewable
and Sustainable Energy ARTICLE scitation.org/journal/rse
J. Renewable Sustainable Energy 13, 025301 (2021); doi: 10.1063/5.0038650 13, 025301-7
V
CAuthor(s) 2021
3
K. Moharm, “State of the art in big data applications in microgrid: A review,”
Adv. Eng. Inf. 42, 100945 (2019).
4
S. Kapoor, B. Sturmberg, and M. Shaw, “A review of publicly available energy
data sets,” Wattwatchers’ My Energy Marketplace (MEM) (The Australian
National University, Canberra, Australia, 2020).
5
P. Street, Dataport: The World’s Largest Energy Data Resource (Pecan Street
Inc, 2015).
6
Open Power System Data, see https://open-power-system-data.org/data-sour-
ces for “Data Sources.”
7
ENTSO-E, see https://www.entsoe.eu/data/power-stats/ for “Power
Statistics.”
8
K. Vink, E. Ankyu, and M. Koyama, “Multiyear microgrid data from a research
building in Tsukuba, Japan,” Sci. Data 6, 190020 (2019).
9
M. Pipattanasomporn, G. Chitalia, J. Songsiri, C. Aswakul, W. Pora, S.
Suwankawin, K. Audomvongseree, and N. Hoonchareon, “Cu-bems, smart
building electricity consumption and indoor environmental sensor datasets,”
Sci. Data 7, 1–14 (2020).
10
K. Fahy, M. Stadler, Z. K. Pecenak, and J. Kleissl, “Input data reduction for
microgrid sizing and energy cost modeling: Representative days and demand
charges,” J. Renewable Sustainable Energy 11, 065301 (2019).
11
M. Ross, C. Abbey, F. Bouffard, and G. Jo
os, “Microgrid economic dispatch
with energy storage systems,” IEEE Trans. Smart Grid 9, 3039–3047 (2018).
12
G. C. Wang, E. Ratnam, H. V. Haghi, and J. Kleissl, “Corrective receding hori-
zon EV charge scheduling using short-term solar forecasting,” Renewable
energy 130, 1146–1158 (2019).
13
B. Washom, J. Dilliot, D. Weil, J. Kleissl, N. Balac, W. Torre, and C. Richter,
“Ivory tower of power: Microgrid implementation at the University of
California, San Diego,” IEEE Power Energy Mag. 11, 28–32 (2013).
14
UC San Diego, see https://blink.ucsd.edu/instructors/resources/academic/calen-
dars/2020.html for “Academic Calender.”
Journal of Renewable
and Sustainable Energy ARTICLE scitation.org/journal/rse
J. Renewable Sustainable Energy 13, 025301 (2021); doi: 10.1063/5.0038650 13, 025301-8
V
CAuthor(s) 2021
... Nevertheless, at the moment, there is a lack of high-resolution data that is publicly available to the research community, limiting the ability to complete such studies. 2 Some exceptions include Bright et al. 3 and Silwal et al. 2 Kapoor et al. 4 present a recent overview of additional available datasets. Yet, all these datasets typically feature a coarse temporal resolution (! 15 min) or present data for a short period (<1 month). ...
... Nevertheless, at the moment, there is a lack of high-resolution data that is publicly available to the research community, limiting the ability to complete such studies. 2 Some exceptions include Bright et al. 3 and Silwal et al. 2 Kapoor et al. 4 present a recent overview of additional available datasets. Yet, all these datasets typically feature a coarse temporal resolution (! 15 min) or present data for a short period (<1 month). ...
... Violin plot showing the distribution of the PV system DC capacity estimation errors that consider the difference between the simulated and measured power output for all clear sky days. The RMSE and bias are expressed relative to the DC capacity [and calculated by Eqs.(2) and(3)]. ...
... This data, collected via smart meters at 15-minute intervals, was used in the "FP7 SuSTAINABLE" project. 5) UCSD micro grid data sets: The UC San Diego (UCSD) microgrid [95] consists of 26 PV plants on campus, where solar inverters operate at unity power factor according to the 2003 IEEE 1547 standard. The data set includes real AC power generation data for the PV plants, which can be downloaded from the supplementary materials of their paper [95]. ...
... 5) UCSD micro grid data sets: The UC San Diego (UCSD) microgrid [95] consists of 26 PV plants on campus, where solar inverters operate at unity power factor according to the 2003 IEEE 1547 standard. The data set includes real AC power generation data for the PV plants, which can be downloaded from the supplementary materials of their paper [95]. ...
Preprint
Full-text available
Photovoltaic (PV) systems have become a cornerstone of renewable energy strategies, particularly due to the significant reduction in solar power costs over the past decade. However, the long-term reliability of PV installations presents a persistent challenge, requiring the development of advanced monitoring and predictive maintenance strategies. A wide range of data types are used to evaluate the health of PV systems, including environmental conditions, electrical performance, and inspection imagery. These data enable methodologies such as machine learning (ML) models for lifetime prediction and computer vision techniques for defect detection. However, the acquisition of high-quality and comprehensive data is difficult, particularly in terms of long-term consistency and data variety. Publicly available data sets serve as valuable resources for addressing these challenges, but they often suffer from fragmentation and are difficult to access. This paper presents a comprehensive review of existing open-source data sets related to PV degradation, analyzing their features, functionalities, and potential applications. We categorize these data sets based on the specific aspects of PV system information they cover, such as environmental conditions, operational monitoring, and material inspection, and propose relevant tools and ML models for processing them. In addition, we propose practices for future data collection and usage, while also discussing potential directions in data-driven research. Our aim is to enhance data utilization and publication among researchers and industry professionals, promoting a deeper understanding of the role of data in enhancing the performance and durability of PV systems.
... The data set was sourced from the repository titled Open-source multi-year power generation, consumption, and storage data in a microgrid [19]. It encompasses a detailed power dataset from a segment of the University of California, San Diego's microgrid, featuring various distributed energy resources (DERs) including solar power plants, electric vehicles, and electric and thermal storage systems. ...
Article
Full-text available
Microgrids are localized energy systems that operate independently or in conjunction with the main power grid, integrating various energy sources for a reliable and resilient power supply. Microgrid energy demand refers to the electrical power required to meet consumer needs, influenced by factors such as time, weather, and habits of consumers. Accurate forecasting of energy demand is crucial for efficient microgrid operation, resource management, and optimization. Machine learning offers advantages over traditional methods in forecasting, such as the ability to handle complex relationships, adaptability to changes, and scalability. This paper proposes a Convolutional Neural Network (CNN) based approach to forecast microgrid energy demand. The study considers a dataset of multi-year power generation, consumption, and storage data in a microgrid and compares different approaches considering the metrics: Mean Squared Error, Mean Absolute Error, and R Squared Value. The results demonstrate that the CNN model performs well compared to other models. This study seeks to advance the application of machine learning in microgrid management.
... And the prior knowledge of these constraints enhances the grid's reliability [58]. In this study, we have considered the UC San Diego microgrid's dataset [59] and computed the hourly generation and consumption forecasts by using the framework provided in section 4.4. The dataset includes power readings for 13 buildings and many distributed energy resources (DERs) tied to a microgrid in the University of California, San Diego campus, taken at 15-minute intervals. ...
... In this paper, we formulate line parameter estimation as a maximum-likelihood problem that has the following distinct difference from previous work: across the network (since these measurements are readily available in most DG's [24], [25]). To solve this problem, we employ an expectationmaximisation (EM) approach [26]- [29] and provide a numerically robust implementation of the resulting EM algorithm. ...
Preprint
Full-text available
This paper presents a method for obtaining distribution grid models based on available measurements from the grid. In particular, we propose a nonlinear approximation of the DistFlow model [1], which includes line losses between nodes and is parameterized by the unknown line parameters, namely the impedance between nodes. Based on measured voltage magnitude at each node, and net injected real and reactive powers at each node, we estimate the unknown line parameter values. To achieve this, we formulate a maximum-likelihood problem that we solve using an expectation-maximisation (EM) approach. In particular, we tailor the EM approach for the proposed nonlinear grid model, and we provide a numerically robust implementation of the resulting algorithm. The proposed method is demonstrated on the IEEE 37-node test feeder network, where we compare to state-of-the-art methods and show that the proposed approach achieves a 70% reduction in voltage error and more than 10, 000 times lower error for state variables. A final comparison uses data from a real network, and we show that the proposed approach achieves parameter estimates that are up to 100 times better than competing approaches.
Article
Full-text available
Reliable data on residential power generation and consumption is vital for effectively integrating renewable energy sources. This is particularly important in the Baltic countries, where climate variability significantly impacts energy production and consumption. Such high-resolution residential usage data is beneficial for various applications, including planning, demand response, consumption behavior analysis, and forecasting. The dataset presented in this study contains one year (2023) of photovoltaic (PV) generation and energy meter power flow data collected at ten-second intervals from a residential dwelling in Estonia. To gather this data, two Camille Bauer PQ1000 power quality monitoring units were installed on the PV and meter side wiring of the house. The paper thoroughly discusses the data collection process, the original dataset, the processed data, and the feature analysis.
Article
Full-text available
This paper describes the release of the detailed building operation data, including electricity consumption and indoor environmental measurements, of the seven-story 11,700-m² office building located in Bangkok, Thailand. The electricity consumption data (kW) are that of individual air conditioning units, lighting, and plug loads in each of the 33 zones of the building. The indoor environmental sensor data comprise temperature (°C), relative humidity (%), and ambient light (lux) measurements of the same zones. The entire datasets are available at one-minute intervals for the period of 18 months from July 1, 2018, to December 31, 2019. Such datasets can be used to support a wide range of applications, such as zone-level, floor-level, and building-level load forecasting, indoor thermal model development, validation of building simulation models, development of demand response algorithms by load type, anomaly detection methods, and reinforcement learning algorithms for control of multiple AC units.
Article
Full-text available
Computational time in optimization models scales with the number of time steps. To save time, solver time resolution can be reduced and input data can be down-sampled into representative periods such as one or a few representative days per month. However, such data reduction can come at the expense of solution accuracy. In this work, the impact of reduction of input data is systematically isolated considering an optimization which solves an energy system using representative days. A new data reduction method aggregates annual hourly demand data into representative days which preserve demand peaks in the original profiles. The proposed data reduction approach is tested on a real energy system and real annual hourly demand data where the system is optimized to minimize total annual costs. Compared to the full-resolution optimization of the energy system, the total annual energy cost error is found to be equal or less than 0.22% when peaks in customer demand are preserved. Errors are significantly larger for reduction methods that do not preserve peak demand. Solar photovoltaic data reduction effects are also analyzed. This paper demonstrates a need for data reduction methods which consider demand peaks explicitly.
Article
Full-text available
Microgrids comprising renewable energy technologies are often modelled and optimised from a theoretical point of view. Verification of theoretical systems with data of actually implemented systems in the field rarely occurs in an open manner, especially on the intermediate scale of research buildings. To enable modelling of the actual microgrid performance of a research environment, we present a multiyear dataset of a microgrid with solar arrays and a battery. The main energy datasets comprise data per second supplemented by hourly solar irradiation data. These may be combined with data concerning the hourly electricity prices from the main grid and the low-electricity-price periods of national holidays. The level of detail of the data per second in combination with the hourly data in these datasets allows for a comparison to the efficiency and weather-parameter correlation of other renewable energy technologies, as well as forecasting future energy generation and consumption.
Article
The prospering Big data era is emerging in the power grid. Multiple world-wide studies are emphasizing the big data applications in the microgrid due to the huge amount of produced data. Big data analytics can impact the design and applications towards safer, better, more profitable, and effective power grid. This paper presents the recognition and challenges of the big data and the microgrid. The construction of big data analytics is introduced. The data sources, big data opportunities, and enhancement areas in the microgrid like stability improvement, asset management, renewable energy prediction, and decision-making support are summarized. Diverse case studies are presented including different planning, operation control, decision making, load forecasting, data attacks detection, and maintenance aspects of the microgrid. Finally, the open challenges of big data in the microgrid are discussed.
Article
Forecast errors can cause sub-optimal solutions in resource planning optimization, yet they are usually modeled simplistically by statistical models, causing unrealistic impacts on the optimal solutions. In this paper, realistic forecast errors are prescribed, and a corrective approach is proposed to mitigate the negative effects of day-ahead persistence forecast error by short-term forecasts from a state-of-the-art sky imager system. These forecasts preserve the spatiotemporal dependence structure of forecast errors avoiding statistical approximations. The performance of the proposed algorithm is tested on a receding horizon quadratic program developed for valley filling the midday net load depression through electric vehicle charging. Throughout one month of simulations the ability to flatten net load is assessed under practical forecast accuracy levels achievable from persistence, sky imager and perfect forecasts. Compared to using day-ahead persistence solar forecasts, the proposed corrective approach using sky imager forecasts delivers a 25% reduction in the standard deviation of the daily net load. It is demonstrated that correcting day-ahead forecasts in real time with more accurate short-term forecasts benefits the valley filling solution.
Chapter
Urban areas suffer from tremendous pressure to cope with increasing population in a city. A smart city is a technological solution that integrates engineering and information systems to assist in managing these scarce resources. A smart city comprises several intelligent services such as smart grids, smart education, smart transportation, smart buildings, smart waste management and so on. Among all these, smart grids are the nucleus of all the facilities because these provide sustainable electrical supply for other smart services to operate seamlessly. Smart grids integrate information and communication technologies (ICT) into traditional energy grids, thereby capturing massive amounts of data from several devices like smart meters, sensors, and other electrical infrastructures. The data collected in smart grids are heterogeneous and require data analytic techniques to extract meaningful information to make informed decisions. We term this enormous amount of data as big energy data. This book chapter discusses progress in the field of big energy data by enlisting different studies that cover several data management aspects such as data collection, data preprocessing, data integration, data storage, data analytics, data visualisation and decision-making. We also discuss various challenges in data management and report recent progress in this field. Finally, we present open research areas in big data management especially in relation to smart grids.
Article
This paper presents a formulation to determine the appropriate power dispatch of an energy storage system, whose available energy is dependent on the charging/discharging pattern from previous time periods. The implementation structure is consistent with current dispatch algorithms used in microgrids, and the algorithm can be used in either grid-connected or islanded modes of operation. The proposed approach employs a backcasting algorithm to estimate the net stored energy value, against which the current cost of energy is compared to determine how the storage system should be used to perform arbitrage. The contribution of this work is a means to include the timedependent resource in traditional economic dispatch algorithms to reduce the cost of energy in a microgrid while enabling the arbitrage algorithm to continuously adapt to changing market conditions. Results show that the backcasting algorithm is able to reduce the average cost of energy by 8.14% and can reduce the average cost of energy by up to 72.3% of the ideal reduction, as determined by a perfect forecasting dispatch.
Article
Smart grids (SGs) have a central role in the development of the global power sector. Cost-benefit analyses and environmental impact assessments are used to support policy on the deployment of SG systems and technologies. However, the conflicting and widely varying estimates of costs, benefits, greenhouse gas (GHG) emission reduction, and energy savings in literature leave policy makers struggling with how to advise regarding SG deployment. Identifying the causes for the wide variation of individual estimates in the literature is crucial if evaluations are to be used in decision-making. This paper (i) summarizes and compares the methodologies used for economic and environmental evaluation of SGs (ii) identifies the sources of variation in estimates across studies, and (iii) point to gap in research on economic and environmental analyses of SG systems. Seventeen studies (nine articles and eight reports published between 2000 and 2015) addressing the economic costs versus benefits, energy efficiency, and GHG emissions of SGs were systematically searched, located, selected, and reviewed. Their methods and data were subsequently extracted and analysed. The results show that no standardized method currently exists for assessing the economic and environmental impacts of SG systems. The costs varied between 0.03 and 1143 M€/yr, while the benefits ranged from 0.04 to 804 M€/yr, suggesting that SG systems do not result in cost savings The primary energy savings ranged from 0.03 to 0.95 MJ/kWh, whereas the GHG emission reduction ranged from 10 to 180 gCO2/kWh, depending on the country grid mix and the system boundary of the SG system considered. The findings demonstrate that although SG systems are energy efficient and reduce GHG emissions, investments in SG systems may not yield any benefits. Standardizing some methodologies and assumptions such as discount rates, time horizon and scrutinizing some key input data will result in more consistent estimates of costs and benefits, GHG emission reduction, and energy savings.
Article
University campuses offer a perfect setting in which to establish a microgrid and maximize its operational benefits. The university setting also offers a unique collection of intellectual resources. In addition to improving operational efficiency, lowering operating costs, and reducing the campus's overall carbon footprint, a university microgrid is an ideal laboratory in which to conduct research to advance modern power system operation and integration of distributed renewable generation for the power delivery industry as a whole. It also provides opportunities for new graduate student research and learning.
Dataport: The World's Largest Energy Data Resource
  • P Street
P. Street, Dataport: The World's Largest Energy Data Resource (Pecan Street Inc, 2015).