Combining Sentinel 1, Sentinel 2 and MODIS data for major winter crop type classification over the Murray Darling Basin in Australia

Full text

Remote Sensing Applications: Society and Environment 34 (2024) 101200
Available online 18 April 2024
2352-9385/© 2024 The Authors. Published by Elsevier B.V. This is an open access article under the CC BY license
(http://creativecommons.org/licenses/by/4.0/).
Contents lists available at ScienceDirect
Remote Sensing Applications: Society and
Environment
journal homepage: www.elsevier.com/locate/rsase
Combining Sentinel 1, Sentinel 2 and MODIS data for major winter
crop type classification over the Murray Darling Basin in Australia
Dhahi Al-Shammari a,*, Ignacio Fuentes b,c, Brett M. Whelan a, Chen Wang d,
Patrick Filippi a, Thomas F.A. Bishop a
aSydney Institute of Agriculture, School of Life & Environmental Science, The University of Sydney, Central Ave, Eveleigh, Sydney, New South Wales,
2015, United Kingdom
bFacultad de Medicina Veterinaria y Agronomía, Universidad de Las Américas, Sede Providencia, Santiago, Chile
cNúcleo de Ciencias Ambientales y Alimentarias (NCAA), Universidad de Las Américas, Santiago, Chile
dCSIRO Data61, Eveleigh, NSW, 2015, Australia
ARTICLE INFO
Keywords:
Radar
Optical remote sensing
Machine learning
Gross primary productivity
Yield monitor data
ABSTRACT
Crop type classification is an essential task in agriculture that has been studied widely since the
emergence of remote sensing technologies. Accurate crop mapping can help assist decision-
making related to storage, marketing, and other production tasks. This study has designed robust
crop classification models to classify two major crop types (cereals and canola) in the Murray
Darling Basin (MDB) in Australia. These models combined Sentinel 1 and 2 and Moderate Resolu-
tion Imaging Spectroradiometer (MODIS) data. Three methods were applied to test classification
quality: the holdout method, leave-one-season-out cross-validation (LOSOCV), and leave-one-
cluster-out-cross-validation (LOCOCV). The holdout method evaluated the model's performance
on data representing the entire population. The LOSOCV method was used to test the model's
ability to extrapolate over time (unseen data from new seasons), and the LOCOCV method was
used to test the ability to extrapolate over space (data from a new site). Crop type labelled data
(n= 193 for cereals and n= 113 for canola) were extracted from high-resolution yield maps de-
rived from grain yield monitors mounted on harvesters, and this was used to train and validate
the classification models. The results showed that the holdout validation approach achieved the
highest accuracy (overall and Kappa scores >0.98), but this might be caused by spatial autocor-
relation of the sampling strategy implemented, which leads to an overoptimistic scenario. The
overall accuracies and Kappa scores achieved from the LOSOCV method varied depending upon
the difference in crop phenology from season to season, with overall accuracy ranging from 0.70
to 0.92 and Kappa scores from 0.61 to 0.90. The overall accuracies and Kappa scores for the LO-
COCV ranged from 0.87 to 0.99 and 0.84 to 0.98, respectively. The gross primary productivity
(GPP) and the chlorophyll red edge index (CIr) were the most important features across the three
models. The outputs of this research can potentially complement models that require crop type
information, such as carbon cycling and crop yield models.
* Corresponding author.
E-mail address: dhahi.al-shammari@sydney.edu.au (D. Al-Shammari).
https://doi.org/10.1016/j.rsase.2024.101200
Received 27 September 2023; Received in revised form 10 April 2024; Accepted 11 April 2024

Remote Sensing Applications: Society and Environment 34 (2024) 101200
2
D. Al-Shammari et al.
1. Introduction
Crop type classification is a challenging task, yet it is valuable in a wide range of agricultural (Burger, 1984;Dey et al., 2021;Lark
and Stafford, 1997;Ugbaje et al., 2017), hydrological (Srivastava et al., 2020;Vervoort et al., 2016), and environmental applications
(Doraiswamy et al., 2007;Gray et al., 2016). In an agricultural context, knowing the location and extent of agricultural lands can help
to estimate crop yield accurately (Rao et al., 2021). This is very important for food security, formulating agricultural policies, and
supply chain logistics (Arias et al., 2020). In a hydrological context, the identification of crop type helps to identify water require-
ments and increases efficiencies in managing water resources since different crop types consume different amounts of water (Dappen
et al., 2008). In a climate such as Australia (semi-arid in most parts of Australia, temperate climate in the southeast and southwest,
and a tropical climate in the north), water resource allocation is a significant issue (Tennakoon and Milroy, 2003). The ecosystem is
complex and variable in space and time (dynamic system), where changes in land cover (especially crops) and land system occur
every season. This requires continuous and accurate monitoring of crop types to help achieve environmental, agricultural, and hydro-
logical goals.
Remote sensing (RS) provides a non-destructive, rapid, and cost-effective method for data collection, especially at the regional and
global scales. Since RS emerged, studies have been increasingly conducted to classify different types of crops. One of the earliest crop
classification studies has been conducted for wheat classification using Landsat 1 data (Misra and Wheeler, 1978). Since then, more
satellites have emerged with different spectral, spatial, and temporal resolutions. Despite satellite-based sensors being able to provide
multi-temporal data at specific spectral and spatial resolutions, some studies have suggested that single-date data can provide suffi-
cient information for crop-type mapping. For example, a single-date image from the SPOT-5 satellite was used for crop type identifica-
tion in southern Texas, United States (Yang et al., 2011). Yang et al. (2011) built five different classification models using a single date
image, and the overall accuracy obtained from these models ranged from 0.68 to 0.87. Other studies have suggested that multitempo-
ral data are required to achieve optimum results (Palchowdhuri et al., 2018;Schultz et al., 2015;Sun et al., 2019;Vuolo et al., 2018).
Multitemporal methods can improve classification results compared to single-date crop-type classification methods (Conese and
Maselli, 1991). However, classification based on a single date might result in errors as some crops have similar spectral reflectance at
a specific stage of plant growth, leading to misclassification. In contrast, phenology-based methods that rely on capturing the changes
in the reflectance trajectory (seasonal dynamic) have great potential since different crops exhibit different reflectance at different
phenological stages.
Phenology-based crop classification methods have gained increasing attention among the agricultural community for providing
reliable results. For example, crop type mapping based on phenology was performed using the 8-day time series of the normalized dif-
ference vegetation index (NDVI) derived from MODIS across the United States (Massey et al., 2017). Massey et al. (2017) mentioned
that MODIS data provided adequate information for capturing variation among crop types. While Landsat offers lower temporal reso-
lution (16 days) compared to MODIS, it provides higher spatial resolution (30 m) that can help to identify fields/parcel boundaries.
An example of a phenology-based model has been built to map cotton in the Murray Darling Basin of Australia (MDB) using phenol-
ogy-based metrics (Al-Shammari et al., 2020). Al-Shammari et al. (2020) found that Landsat 8 provided sufficient information for
suitable mapping cotton fields in the MDB. Another study suggested that Landsat data was sufficient to classify wheat crops at a field
scale (Khan et al., 2018). However, the revisit time of Landsat and cloud cover contamination (which may affect the entire season)
might prevent capturing the unique spectral reflectance of different crop types. One way to overcome these issues is by filling the gap
of missing data using existing gap-filling methods, such as the Fourier transformation technique, which is used in some crop-type
mapping studies (Al-Shammari et al., 2019;Al-Shammari et al., 2020;Moody and Johnson, 2001;Wang et al., 2019).
Since the launch of Sentinel 1 (Synthetic aperture radar: Sentinel 1) in 2014 and Sentinel 2 (optical sensor) in 2015, scientists have
put in significant effort to explore the potential of these two satellites for crop-type mapping. The advantage of these two satellites is
that they provide high spatial resolution (Sentinel 1: 10 m, and Sentinel 2: 10–20 m) with shorter revisit times (Sentinel 1: 12 days,
and Sentinel 2: 5 days) than Landsat satellites. While Landsat provides useful bands for vegetation monitoring in the visible and near-
infrared (NIR) regions, Sentinel 2 provides three bands in the red edge region of the spectrum, which are very important for crop type
mapping because these bands are related to the vegetation biochemical contents (Forkuor et al., 2018). A study showed that using
Sentinel 2 data to classify winter crops in India improved classification accuracy by about 7% compared to the use of Landsat 8 data
(Paul and Kumar, 2019). Moreover, Sentinel 1 provides bands for detecting changes in vegetation structure (Vreugdenhil et al.,
2018). A study showed that Sentinel 1 V V (vertical transmit/vertical receive) and VH (vertical transmit/horizontal receive) backscat-
ter intensity bands were very important in the identification of wheat crops (Song and Wang, 2019).
Even though many studies show the potential of using data from a single satellite for crop classification, some studies have ex-
plored the potential of combining data from different satellites to improve the classification results. For example, Forkuor et al.
(2018) reported an improvement in the classification results by 4% when combining Sentinel 2 red edge bands with Landsat 8 bands.
Another study found that combining data from Sentinel 2 and Landsat 8 improved the classification performance of winter wheat (Xu
et al., 2020). Therefore, incorporating data from different sensors may help improve model performance.
More recently, many studies have been conducted to explore the potential of combining radar Sentinel 1 data with optical data
from other sensors for crop classification. For example, a study confirmed that integrating radar data from Sentinel 1 with Landsat 8
data improved seasonal crop classification (Demarez et al., 2019). In a study on sugarcane mapping, Wang et al. (2020) combined
data from Landsat 8, Sentinel 1, and Sentinel 2. The authors suggested that combining all these data might provide sufficient informa-
tion for classification models, especially in agricultural systems where fields have small sizes, and the area is impacted by frequent
cloud cover contamination (Wang et al., 2020).

Remote Sensing Applications: Society and Environment 34 (2024) 101200
3
D. Al-Shammari et al.
Agricultural lands in Australia are utilised for cropping (mainly cereals) and grazing for livestock production, and one-third of
Australia's food is grown in the MDB. The estimated grazing area is around 727 800 km2, while the dryland cropping is around
133 300 km2in the MDB (Australian Bureau of Meteorology, A. g, 2011). This is problematic when optical sensors are used since pas-
ture can follow the same growing season as crops (Wolfe, 2020) and may show similar greenness and brightness to some cereal crops
(Hill et al., 2005). Furthermore, it has been reported that grass/pasture may be misclassified with wheat because of the similarity in
the plant structure of both systems (Whelen and Siqueira, 2018). Therefore, additional information should be explored to comple-
ment radar and optical data to avoid misclassification issues, especially between pasture and crops in the MDB. The variation in the
terrestrial vegetation gross primary productivity (GPP) has potential as a predictor (Li et al., 2022) since this difference is noticeable
among different vegetation covers. GPP can be a potential predictor that can help mitigate misclassification issues, especially in the
case of pastures and cereals, which are the major crop types in the MDB. GPP refers to the amount of carbon fixation levels plants use
through photosynthesis (Chagas et al., 2019;He et al., 2018). Another study by Gilmanov et al. (2003) showed that winter wheat
recorded a different annual GPP than pasture. A study by et al. (2006) was conducted to evaluate the GPP and net primary productiv-
ity (NPP) products across nine sites around the globe. Turner et al. (2006) suggested that MODIS GPP and NPP products respond to
the general GPP and NPP magnitude trends related to vegetation cover and local climate. Therefore, the temporal variation in GPP
might add valuable information to any classification model. Despite the use of the GPP (which is critical to understanding the carbon
cycle) in many agricultural production systems for vegetation status monitoring (Wolanin et al., 2019) and carbon exchange (Gitelson
et al., 2012), it has not been explored for crop classification. More specifically, the satellite-based GPP has not been used for crop-type
classification. MODIS offers a GPP product at eight days and 500 m spatial resolution. Thus, the potential of this product can be ex-
plored in the context of crop type classification.
Quality and accuracy, spatial and temporal resolution, and representativeness of reference (labelled) data are crucial in crop type
classification models, as the classification models are often limited due to these issues. For example, data collection, labelling, and
georeferencing errors can occur when reference data are collected through field surveys or other methods. Therefore, there is no clear
delineation of boundaries when collecting reference data. Furthermore, labelled data are often collected at a specific point in space
and time, which leads to errors, especially if they were collected at the start- or mid-season given the in-season changes. Field bound-
aries (polygons), and global positioning system (GPS) points sampling have been used to provide representative data to the classifica-
tion models (Conese and Maselli, 1991;Forkuor et al., 2018;Khan et al., 2018;Massey et al., 2017;Misra and Wheeler, 1978;Moody
and Johnson, 2001;Palchowdhuri et al., 2018;Sun et al., 2019;Vuolo et al., 2018;Wang et al., 2019;Yang et al., 2011). Yield moni-
tor maps offer an excellent labelled data source in the classification models as these are the actual crops grown and harvested in a
given area. Yield monitor maps are not commonly used as labelled data in previously published crop classification studies (Al-
Shammari et al., 2020).
Therefore, this study aimed to investigate the potential of multi-source data for wheat, barley, and canola crop classification. High
spatial and temporal resolution Sentinel 1, optical Sentinel 2, and GPP from MODIS will be integrated to classify major winter crops in
the MDB in Australia. The study took advantage of using precision agriculture (PA) data to train and validate the classification (ran-
dom forest) algorithm in Google Earth Engine (GEE) (Gorelick et al., 2017). Three classification methods were followed in this study.
The first method was calibrated, validated, and tested using the holdout method. Moreover, two additional cross-validation strategies
were considered to explore the quality of the model predictions when extrapolating in space and time, namely the leave-one-season-
out-cross-validation (LOSOCV) and the leave-one-cluster-out-cross-validation (LOCOCV) models.
2. Methods
2.1. Study area characteristics and extent
The MDB (Fig. 1) is located in the south-east of Australia, and it covers 1 059 000 km2(14% of Australia's land area). Agricultural
lands cover about 67% of the MDB area, accounting for more than 40% of Australian agricultural production (Kirby et al., 2014). The
MDB is an important production area for crops and pasture for grazing (Scientists, 2017). According to the long-term average records,
the MDB receives 457 mm of mean annual rainfall (Potter et al., 2010), although there is significant variation throughout the basin.
The MDB is characterized by a sub-tropical climate in the north, a semi-arid climate in the west, and a temperate in the south, with
high interannual variability associated mainly with El Niño–Southern Oscillation cycle (Evans and McCabe, 2010).
2.2. Labelled data
Labelled data were collected from different sources based on the extent of the study area. Crop information is derived from high-
resolution (10 m) crop yield maps in different locations across the MDB (Fig. 1). These maps are generated after harvest using ad-
vanced yield monitoring systems equipped with high-precision sensors installed in harvesters using GPS technology. The high accu-
racy of these maps is achieved through accurate calibration of yield sensors for specific crop types, moisture conditions, and other rel-
evant factors. Yield maps of wheat, barley and canola have been retrieved in raster format for each field-season-crop type and then
converted to polygons representing the boundaries of each field (details about fields are shown in Table 1).
According to the crop calendar in the MDB, a temporal window between the 1st of May and the 1st of November can be considered
as the start and end of the growing season. Therefore, this temporal window was used to collate the predictor variables.
Even though the main interest was to classify winter crops, other classes have been added to the classification models to augment
the dataset to improve the predictions of crops. Therefore, the models can be used for agricultural, hydrological, and environmental
applications. Land use data from the Australian Collaborative Land Use and Management Program (ACLUMP) for New South Wales
(NSW) (2017) and Victoria (VIC) (2016) were used to collect polygons of the grazing native vegetation (GNV) and forest classes

Remote Sensing Applications: Society and Environment 34 (2024) 101200
4
D. Al-Shammari et al.
Fig. 1. Study area. Locations of sites where yield monitor data are located. These are shown in yellow, represented by site numbers and locations. Exact field bound-
aries are not shown to preserve landowners' privacy. The background image is the red-green-blue (RGB) composite image for Australia. (For interpretation of the refer-
ences to colour in this figure legend, the reader is referred to the Web version of this article.)
Table 1
Details of sites used in this study, including number of fields (NO. fields) and total area in hectares (ha) for each crop type in every season. NO. fields = Number of
fields.
Site Crop type Area (ha)2017 NO. fields Area (ha) 2018 NO. fields Area (ha) 2019 NO. fields Total area (ha)/site
Site 1 Cereals 4664 31 2996 19 4767 27 12 427
Canola 3534 24 3296 23 2027 16 8857
Site 2 Cereals 7583 10 8033 12 9019 13 24 635
Canola 2278 5 – – 709 1 2987
Site 3 Cereals 1492 12 1812 11 4004 17 7308
Canola 2090 10 3187 13 1059 5 6336
Site 4 Cereals 482 12 – – – – 482
Canola 433 9 – – – – 433
Site 5 Cereals 271 2 1719 9 1990
Canola 253 1 – – 379 6 632
Site 6 Cereals 2441 15 – – 507 2 2948
Canola – – – – – – –
Total area (ha)/season Cereals 16 662 80 13 112 44 20 016 68
Canola 8588 49 6483 36 4174 28
(https://www.agriculture.gov.au/abares/aclump/land-use/mapping-technical-specifications). The ACLUMP datasets have an at-
tribute accuracy of 80 percent or greater, encompassing a range of scales, from fine scales (1:10 000 to 1:25 000) for areas such as ir-
rigated and peri-urban, to coarser scales (1:100 000 for cropping regions and 1:250 000 for semi-arid and arid pastoral zones). The as-
sumption is that this is uniform in time, unlike the cropping classes. Bare soil can be extracted by thresholding NDVI (Vaudour et al.,
2022). Bare soil samples were extracted after setting up an NDVI threshold of 0 ≤0.2, where pixels with a value that did not exceed
this threshold through the growing season were considered bare soil class. Consequently, all bare soil class pixels were filtered across
the entire MDB, and polygons were created based on these filters. All polygons of each class were further buffered with a 100 m inter-
nal buffer to avoid mixed pixel effects around the boundaries. This is a high-resolution pixel-based classification, and all polygons
were sampled to ensure the inclusion of samples from the entire study area. For each class (cereals, canola, forest, GNV and bare soil),
samples were drawn randomly from the polygons for each growing season. These samples were prepared to be used to extract predic-
tor variables for each class (Table 2).
2.3. Satellite data
This phenological-based study used multitemporal Sentinel 1, Sentinel 2 and MODIS data. All data were accessed, processed,
transformed (feature extraction), and analysed in the Google Earth Engine (GEE) platform. The complete workflow is shown in Fig. 2.
The C-band images from a dual-polarization C-band Synthetic Aperture Radar (SAR) instrument provided by the Sentinel 1 mis-
sion were filtered to the Interferometric Wide (IW) mode and a descending (morning) pass. The image collection was set to a single

Remote Sensing Applications: Society and Environment 34 (2024) 101200
5
D. Al-Shammari et al.
Table 2
Number of samples prepared for the classification. The cereals class included samples of barley (5000 samples) and wheat (5000 samples). GNV: Grazing native
vegetation.
Class Cereals Canola Forest GNV Bare soil
2017 100 000 5000 5000 5120 3116
2018 100 000 5000 5000 5246 3134
2019 100 000 5000 5000 5223 3134
Fig. 2. Simple flowchart shows the analytical pathway. VV: vertical transmit/vertical receive, VH vertical transmit/horizontal receive. NSW: New South Wales, VIC:
Victoria. LU: Land use. RF = Random Forest. Models 1, 2, and 3, are the holdout, the leave-one-season-out-cross-validation (LOSOCV), and the leave-one-cluster-out-
cross-validation (LOCOCV) models, respectively.
co-polarization. The backscatter intensity bands, including the VV (vertical transmit/vertical receive) and the VH (vertical transmit/
horizontal receive), were extracted from the 1st of May to the 1st of November for each growing season. In this study, a total number
of 792 (2017), 839 (2018), and 834 (2019) scenes were acquired. The time series of images for each scene were used to calculate the
mean of winter VV and winter VH. The Sentinel 2 level-1C top-of-atmosphere (TOA) data were also obtained from the ESA and ac-
cessed via the GEE platform. A total number of 5316 (2017), 9357 (2018), and 9464 (2019) scenes of Sentinel 2 covered the MDB
area. For this study, Sentinel 2 data were used to extract three types of information, including the monthly mean of EVI, quantiles of
the enhanced vegetation index (EVI) and the chlorophyll red edge index (CIr), and the phase and amplitude (which were extracted us-
ing EVI). A cloud removal algorithm (Hagolle et al., 2010) was applied to mask out images that have more than 10% of cloud contam-
ination (cloud percentage is shown in Fig. 3).
After cloud removal, gaps in the time series are unavoidable; therefore, a gap-filling algorithm or image reduction technique was
applied to fill gaps due to missing pixels. In this case, we constructed seven images (May–November) to represent a time series of

Remote Sensing Applications: Society and Environment 34 (2024) 101200
6
D. Al-Shammari et al.
Fig. 3. Mean cloud probability map in the Murray Darling Basin (MDB) based on the sens2cloudless library for processing Sentinel 2 images (left) and time series
heatmap of the cloud coverage based on the monthly mean of the “CLOUD_PIXEL_PERCENTAGE”property stored on the metadata of scenes (right).
monthly images for each growing season. Images were reduced to monthly mean for each month in the growing season to fill the gap
in the time series. These images were used to calculate the monthly EVI (Equation (1)) (Liu and Huete, 1995), which is similar to the
NDVI. However, the EVI is calculated in a way that improves the sensitivity in high biomass regions and avoids the NDVI saturation
issue. The EVI also incorporates corrections for some atmospheric conditions and canopy background noise. The EVI is reported to be
more helpful in detecting variations in vegetation. The amplitude and phase were extracted from the original time series of EVI by fit-
ting a harmonic function (Bracewell and Bracewell, 1986) that decomposes a periodic signal into sine and cosine frequencies defined
by its unique amplitude and phase angle. In the agricultural context, phase and amplitude are crucial in measuring interannual varia-
tion (Al-Shammari et al., 2020;Jakubauskas et al., 2001;Mingwei et al., 2008). The phase represents the variation in time from the
origin to the wave's peak (e.g., from the lowest value to the maximum greenness of EVI). Amplitude represents the change that occurs,
measured as the distance between the middle of the wave to the peak. The 5th, 50th, and 95th percentiles of the seasonal EVI were
also calculated, corresponding to low, median, and high vegetation cover. As some studies have reported that the red edge bands
could help to increase the accuracy of classification, the CIr (Equation (2)) was included to examine its importance to the model. The
5th, 50th, and 95th percentiles of the CIr (Gitelson et al., 2003) were also calculated for every growing season (May–November) and
added to the space-time cube (STC), which is a data cube that consists of predictors and associated labelled data, for the analysis
(Table 3). For every growing season (May–November), 23 layers (using a cumulative 8-day composite) of MODIS GPP were accumu-
lated for the growing season and added to the STC.
EVI =2.5×NIR −red
NIR +6×red −7.5×blue +1
(1)
CIr =Red edge 3
Red edge 1
−1
(2)
The MODIS GPP data were resampled using the bilinear resampling method to have the spatial resolution (10 m) as Sentinel 2 and
the yield monitor data. Variables described in Table 3 were extracted for each sample within each class and for each growing season.
Variables have been calculated differently to obtain optimal representation of variables (Table 3). For example, VV and VH for each
Table 3
Description of variables used in the space-time cube (STC). Monthly means of EVI are represented by the name of each month (May to November).
Variable Description Source
VV and VH Means of VV and VH for each winter growing season Sentinel 1
Monthly EVI Monthly means of EVI for May, June, July, August, September, October, and November for each winter growing season Sentinel 2
EVI 5th, 50th, 95th percentiles of EVI for each winter growing season Sentinel 2
CIr 5th, 50th, 95th percentiles of EVI for each winter growing season Sentinel 2
GPP Cumulative GPP for each winter growing season MODIS
Amplitude Amplitude for each winter growing season (from EVI) Sentinel 2
Phase Phase for each winter growing season (from EVI) Sentinel 2

Remote Sensing Applications: Society and Environment 34 (2024) 101200
7
D. Al-Shammari et al.
growing season were calculated as the mean of the entire growing season, and this is supposed to give a better representation of the
growing season as it reduces the noise (Dalsasso et al., 2020). To avoid missing pixels in the time series and to provide EVI data for the
entire growing season, EVI was calculated as the monthly mean for each month (Tatsumi et al., 2015). EVI percentiles were calculated
to capture the low, average, and high vegetation cover of classes. CIr percentiles were calculated and added to the STC to explore the
usefulness of red edge index for better capturing low, average, and high vegetation cover. GPP was calculated as the cumulative GPP
for the entire growing season to indicate the total biomass for each class. Amplitude and phase were calculated to represent time and
magnitude of change during the growing season (Al-Shammari et al., 2020).
2.4. Classification procedures
After preparing the STC (Fig. 2), a pixel-based classification using a random forest (RF) approach was performed. The RF algo-
rithm is a widely used supervised machine learning algorithm due to its simplicity, flexibility and speed in model building and predic-
tion (Shaik and Srinivasan, 2019). The RF is based on building multiple decision trees and assembling them to create a more accurate
and stable prediction (Rodriguez-Galiano et al., 2012). In this study, the RF was trained with 250 decision trees, the square root of the
number of variables per split, and a fraction of input to bag per tree of 0.5. Three methods for classification were followed. The first
method, which is the holdout method was to test the classification quality using all data, where data were split into 80% for calibra-
tion and 10% for validation and 10% for testing. However, the holdout method is assumed to cause an overoptimistic performance
caused by random sampling of spatially autocorrelated pixels, which are included in both, the training and testing subsets (Geiβet al.,
2017). Consequently, two different validation strategies, which are more conservative in terms of performance, are included to ac-
count for the spatial autocorrelation of samples. In the LOSOCV method, data from two growing seasons were randomly divided into
80% for calibration and 20% for validation, while the data from the third growing season were left out for testing. The LOSOCV helps
to check classification quality using data from a new growing season that is not included in building the model. The data for each
method was trained and validated with different numbers of samples. Additionally, a LOCOCV method was implemented, in which
data from five out of six clusters (sites) were randomly split into 80% for training and 20% for validation and one cluster was used for
testing. Therefore, in these two cross-validation strategies, the testing subsets are assumed as totally independent from the training
subsets, and no spatial autocorrelation should affect the performance evaluated.
2.5. Accuracy assessment and variable importance evaluation
Assessing the quality of classification was performed by calculating the overall accuracy (OA: Equation (3)), Kappa coefficient
(Kappa: Equation (4)), user's accuracy (UA: Equation (5)), and producer's accuracy (PA: Equation (6)). OA is calculated by the sum of
the correctly classified pixels divided by the total number of pixels. Unlike OA, Kappa calculates the random probability of agree-
ment; thus, it is more conservative than OA. UA reports pixels that the classification model allocated to a specific class, but they be-
longed to another class (wrongly identified). PA calculates the number of pixels accurately classified for a specific class (pixels identi-
fied with one class correctly). In other words, the UA shows the accuracy of the existing map, whereas the PA shows how a model will
predict new data. The OA, Kappa, UA, and PA are ranging from 0 to 1, with 1 indicating optimal results and 0 indicating unsatisfac-
tory results, as shown in Equation (3):6.
OA =Number of samples correctly classified
Total number of samples
(3)
Kappa =Observed agreement −Agreement by chance
1−Agreement by chance
(4)
UA =Number of pixels correctly classified in each class
Total number of classified pixels in that class (row total)
(5)
PA =Number of pixels correctly classified in each class
Total number of classified pixels in that class (column total)
(6)
The F1-score is a widely used metric for evaluating the performance of classification models (Yang and Liu, 1999;Rao et al.,
2021), particularly in scenarios where there is class imbalance. In crop type classification tasks, imbalanced data can be a common is-
sue due to variations in the distribution of different types of crops within a dataset. The F1-score considers both precision and recall,
providing a balanced measure of model performance that is especially useful when dealing with imbalanced classes. By considering
both false positives and false negatives, the F1-score provides a more comprehensive assessment of a classifier's effectiveness in cor-
rectly identifying all classes, including minority classes. Given the imbalanced nature of our dataset, the F1-score was included to
complement other metrics used in this study and ensure a comprehensive evaluation of the model's performance (Equation (7)).
F1 −score =2×precision ×recall
precision +recall
(7)
In this study, the variable importance algorithm embedded within the RF algorithm (Breiman, 2001;Ho, 1995) was used to determine
the influence of each predictor variable in the model. The importance of a predictor variable was calculated by counting the number
of times that the variable was selected for a split within each decision tree (Buston and Elith, 2011). This is important since not all pre-
dictor variables are expected to be essential to achieve a high model performance.

Remote Sensing Applications: Society and Environment 34 (2024) 101200
8
D. Al-Shammari et al.
3. Results
3.1. Evaluation of EVI temporal patterns of all classes
Crops, including cereals and canola, showed different spectral reflectance profiles to the forest and GNV. The monthly EVI time se-
ries are shown in Fig. 4. The EVI temporal patterns of forest class show no increase or decrease in the EVI values during the growing
season for all three growing seasons. The EVI time series showed that the forest always has a higher EVI (median ∼0.30) than GNV,
which had lower EVI values median values ∼0.14–0.26) during the growing seasons (except July and August, which exceeded a me-
dian of ∼0.30). In contrast to the forest class, which was the least variable (i.e., shorter Interquartile range) class in terms of EVI vari-
ability in all growing seasons, GNV was the class that showed most variability (i.e., longer Interquartile range) during the growing
season, especially in August and September.
Furthermore, there was a difference between cereal crops' EVI trajectory and canola's EVI trajectory from June until the end of the
growing season. Canola generally showed higher EVI reflectance than cereals after the start of June until late August, when EVI values
of canola decreased to be similar to the cereal reflectance values, especially in 2017 and 2019 (median ∼0.50–0.70), which corre-
sponds to the flowering stage (production of yellow flowers). On the other hand, cereals (wheat and barley) do not exhibit the same
reflectance characteristics as canola. Therefore, the EVI values of cereals were lower than those of canola from June until the end of
the growing season. It can be noted that forest and GNV are more stable than crops throughout the growing season, which makes it
easier to distinguish from crops.
3.2. Quality of classification
3.2.1. Assessment of the holdout model
Table 4 shows the PA and UA obtained from the test set. The model showed high reliability, which yielded a high OA (0.99) and
Kappa coefficient (0.98). The PA and UA were also high, with a PA of >0.97 and a UA of> 0.96 for all classes (Table 4). Table 11
(appendices) also shows that the model performed well, with all classes separated well, and only a few pixels from each class were
wrongly assigned. The errors mainly occurred because the pixels of other classes were confused with the GNV class, but the error was
Fig. 4. Temporal EVI profile fo rea ch class for growing seasons 2017 (A), 2018 (B), and 2019 (C). GNV: Grazing native vegetation. Every boxplot in the figure shows the
distribution of monthly EVI values of all data for each class.
Table 4
PA and UA of the holdout model. GNV: Grazing native vegetation. Results are reported from the test set.
Class PA UA
Cereals 0.98 0.98
Canola 0.97 0.99
Forest 0.99 0.98
GNV 0.98 0.96
Bare soil 1 0.99

Remote Sensing Applications: Society and Environment 34 (2024) 101200
9
D. Al-Shammari et al.
still marginal (UA = 0.96). Due to how the holdout method splits the data, the model is often evaluated on representative data in the
testing set, resulting in high accuracy. The F1-scores were 0.98, 0.96, 0.99, 0.92, and 0.98 for cereal, canola, forest, GNV and bare
soil, respectively, and macro-averaging for this model was 0.97.
Fig. 5 shows a randomly zoomed-in classified map of the winter crops in the MDB obtained from the 2017 growing season. The
classified map in this figure exemplifies how well the model could be when extrapolating outside the range of the reference data.
Based on the variable importance calculation (Fig. 6), the top five predictor variables that contributed most to the RF model were
the GPP for the winter growing season, which was the most important contributor in the model, followed by the VV backscatter inten-
sity of the winter growing season, the 95th percentile of the Cir, EVI in September, and the VH backscatter intensity of the winter
growing season. The other variables also contributed to the model but with slightly less importance. The amplitude was in the middle
list (bottom five predictor variables in terms of importance) of the variable importance ranking (Fig. 6).
The classification results showed a gradual decline in the winter cereal areas over time (Fig. 7). In 2017, the classified maps
showed that about 8.74 million ha of cereal were grown, whereas, in 2018, the cereal planting areas declined to around 8.46 million
ha. In 2019, there was a decline in these areas again to about 8.06 million ha. On the other hand, canola was predicted to cover
around 1.86 million ha in 2017. Then, there was a dramatic decline in the canola area to 0.6 million ha (down 67%) in 2018. In 2019,
there was a slight increase in the canola area to about 0.76 million ha. The decline in the cereal and canola areas is related to the de-
cline in the rainfall over the MDB during the 2018 and 2019 growing seasons (Fig. 7 box plot) and market forces.
3.2.2. Assessment of the LOSOCV model
This model resulted in different accuracies from the holdout model, which was expected since the validation data comes from a
different season unseen by the model. The LOSOCV model yielded accuracies of 0.82, 0.70, and 0.92, with an overall standard devia-
tion of 0.09and Kappa score of 0.77, 0.61, and 0.90 (with an overall standard deviation of 0.12) when calibrating the LOSOCV models
on data from 2018/2019, 2017/2019, and 2017/2018, respectively, and testing with 2017, 2018, and 2019, respectively (Table 5).
The F1-score was the highest in 2019 and lowest in 2017 for cereal, whereas, for canola, the F1-score was the lowest in 2018 (Table
6). The macro-averaging values were 0.83, 0.71, and 0.91 for 2017, 2018, and 2019, respectively.
The results obtained from the test sets are shown in (Table 7). Many pixels of the cereals class were incorrectly assigned to canola
(∼13% of the cereal pixels) and GNV (∼0.25% of the cereal pixels) when validating the model on the 2017 growing season data set
(appendices: Table 12). Thus, the cereals class achieved the lowest PA (0.60). In the next two growing seasons (2018 and 2019), the
PA of cereals improved significantly, where a PA of 0.85 (2018) and 0.95 (2019) were achieved for the cereals class. However, in
2018, about ∼0.14% of the GNV pixels were classified as cereal. This might indicate some issues when calibrating a model using data
from 2018 to 2019. The model's performance fluctuations could be attributed to an imbalance in training data across classes or
drought conditions in 2018 (Prosser et al., 2021). The model predicted canola well in 2017 (PA = 0.89) and 2019 (PA = 0.83); how-
Fig. 5. Zoomed-in classified map of winter crops versus other cover types of the 2017 growing season in the Murray Darling Basin (MDB), Australia.

Remote Sensing Applications: Society and Environment 34 (2024) 101200
10
D. Al-Shammari et al.
Fig. 6. Variable importance of the holdout model. GPP: Gross primary production; VH (winter): VH of the winter growing season; VV: VV of the winter growing sea-
son. EVI (p5), EVI (p50), and EVI (p95) are the 5, 50, and 95 percentiles of the EVI vegetation index. Cir (p5), Cir (p50), and Cir (p95) are the 5, 50, and 95th per-
centiles of the Cir vegetation index. May to November predictor variables represent the EVI of each month.
Fig. 7. The bar and box plots show the estimated cereals and canola areas obtained from the holdout model. The bar plot shows the area in ha of classified cereals and
canola across three growing seasons. The box plot shows the amount of rainfall (mm) fallen over cereals and canola study sites over the three growing seasons (winter
growing seasons). The left y-axis corresponds to the bar plot where the area is shown in Mega ha (Mha). The right y-axis corresponds to the box plot where the rainfall
is measured by mm of rainfall.
Table 5
Overall accuracy (OA) and kappa score obtained from the leave-one-season-out-cross-validation (LOSOCV) model.
Calibration and validation Test OA Kappa score
2018 and 2019 2017 0.82 0.77
2017 and 2019 2018 0.70 0.61
2017 and 2018 2019 0.92 0.90

Remote Sensing Applications: Society and Environment 34 (2024) 101200
11
D. Al-Shammari et al.
Table 6
F1-score from the leave-one-season-out-cross-validation (LOSOCV) model.
F1-score
Class 2017 2018 2019
Cereals 0.72 0.79 0.94
Canola 0.80 0.63 0.89
Forest 0.97 0.77 0.98
GNV 0.74 0.66 0.88
Bare soil 0.93 0.73 0.87
Table 7
Producer's accuracy (PA) and user's accuracy (UA) of the leave-one-season-out-cross-validation (LOSOCV) model. GNV: Grazing native vegetation. Results are re-
ported from the test set.
Class 2017 2018 2019
PA UA PA UA PA UA
Cereals 0.60 0.95 0.85 0.75 0.95 0.93
Canola 0.89 0.76 0.45 0.99 0.83 0.92
Forest 0.97 0.98 0.61 0.98 0.97 0.99
GNV 0.97 0.60 0.96 0.47 0.96 0.81
Bare soil 0.89 0.98 0.30 0.99 0.80 0.99
ever, the PA decreased sharply in 2018 (PA = 0.45), which means it was difficult to identify canola in that growing season. On the
other hand, the GNV class was well identified in all growing seasons with a PA of 0.97, 0.96 and 0.96 for 2017, 2018, and 2019, re-
spectively. However, the GNV was classified with moderate UA in 2017 (0.60) and low UA (0.47) in 2018.
The variable importance (Fig. 8) shows that the GPP of winter, the 95th percentile of the Cir, the VH and VV backscatter intensity
of winter, and the EVI in September were always in the top six variables in terms of contribution to the model in every growing sea-
son. EVI in May was selected to be among the top six important variables in 2018 and 2019. The contribution of amplitude and phase
was lower than expected, with the amplitude and phase in 2017 where among the six least important variables. Phase remained
among the least important variables in all growing seasons, whereas amplitude importance increased only in 2018.
The estimated areas of cereals and canola obtained from the LOSOCV are shown in Fig. 9. For cereal, the estimated areas in 2017
were about 7.21 million ha. Cereal areas declined to 7.11 million ha in 2017 and later increased in 2019 to around 7.75 million ha.
The increase in cereal areas did not follow the decline in rainfall as in the holdout model (Fig. 7). Canola areas were estimated to be
around 2.43 million ha in 2017. In 2018, these areas declined significantly to about 0.25 million ha; in 2019, the area was estimated
to be about 0.77 million ha. The sharp decrease in canola areas was attributed to the misclassification of canola (Table 7: 2018).
3.2.3. Assessment of the LOCOCV
Table 8 shows that the OA ranges from 0.87 to 0.99, with an overall standard deviation of 0.04, and the Kappa score ranging from
0.84 to 0.98, with an overall standard deviation of 0.05. Sites 1, 3, 5, and 6 have high OA and Kappa scores, indicating that the classi-
fication model performed well at these sites. Site 2 has a lower OA and Kappa score, but site 4 has the lowest OA and Kappa score.
However, the results are still reasonable for all sites, which indicates the reliability of the LOCOCV model in classifying new data out-
side that are unseen and anywhere in the MDB.
The F1-score results of the LOCOCV (Table 9) showed that canola demonstrated variability in classification accuracy, with F1-
scores as low as 0.75 at Site 2, suggesting potential challenges in model performance under site-specific conditions or perhaps indica-
tive of inherent complexities within the data characteristic of that site. At Site 6, there was no data for canola, preventing an evalua-
tion of model performance for this class. For cereals, while generally showing strong performance, indicating a notable decrease in
classification accuracy at Site 4 (F1-score of 0.79), hinting at possible site-specific conditions affecting the model's predictive accu-
racy. According to macro-averaged F1-score results, the model is robust overall. This is evident at Site 5, where the F1-score result is
highest at 0.98 compared to 0.87 at Site 4.
The LOCOCV showed the ability to classify unseen data from new sites (Table 10) correctly. The classifier could effectively iden-
tify cereals in sites 1, 3, 5, and 6, as shown in Table 10 and Figs. 10–15 (Appendices). On the other hand, cereals in site 4 scored the
lowest UA (∼0.63), and this was because cereals were confused with the GNV class (Appendices: Table 11) and this might be because
failure in crop growth in some patches in the fields of this site (Appendices: Fig. 13). Cereals in site 2 were also confused mostly with
the GNV (UA ∼0.75), and this is again could be due to stressed crops or sparse distribution of crops in some patches in the fields of this
site. The classifier could identify canola in sites 1 and 5 with high PA (>0.87) and high UA (>0.92); however, the classifier wrongly
identified canola pixels as cereals in sites 2, 3, and 4, and this might indicate that canola has a very different signature from site to
site. This behaviour was not observed in the cereals, where the ability of the classifier to identify cereals was much better for each site
using other sites’data.
The variable importance of the LOCOCV model (Appendices: Fig. 16) shows that the GPP of winter, the 50th, 95th percentile of
the Cir, and the EVI in September were always in the top four variables in terms of contribution to the model in every growing season.

Remote Sensing Applications: Society and Environment 34 (2024) 101200
12
D. Al-Shammari et al.
Fig. 8. Variable importance of the leave-one-season-out-cross-validation (LOSOCV) model. GPP: Gross primary production; VH (winter): VH of the winter growing sea-
son; VV: VV of the winter growing season. EVI (p5), EVI (p50), and EVI (p95) are the 5, 50, and 95 percentiles of the EVI vegetation index. Cir (p5), Cir (p50), and Cir
(p95) are the 5, 50, and 95 percentiles of the Cir vegetation index. May to October predictor variables represent the EVI of each month.
Fig. 9. Bar and box plot s show the estimated cereals and canola areas obtained from the leave-one-season-out-cross-validation (LOSOCV) model. The bar plot shows the
area in ha of classified cereals and canola across three growing seasons. The box plot shows the amount of rainfall (mm) fallen over cereals and canola study sites over
the three growing seasons. The left y-axis corresponds to the bar plot where the area is shown in Mega ha (Mha). The right y-axis corresponds to the box plot where the
rainfall is measured by mm of rainfall.

Remote Sensing Applications: Society and Environment 34 (2024) 101200
13
D. Al-Shammari et al.
Table 8
Overall accuracy (OA) and kappa scores obtained from the leave-one-cluster-out-cross-validation (LOCOCV) model.
Site OA Kappa
Site 1 0.97 0.96
Site 2 0.89 0.86
Site 3 0.96 0.95
Site 4 0.87 0.84
Site 5 0.99 0.98
Site 6 0.97 0.96
Table 9
F1-score and macro-averaging values from the leave-one-cluster-out-cross-validation (LOCOCV) model.
F1-score
Class Site 1 Site 2 Site 3 Site 4 Site 5 Site 6
Cereals 0.94 0.81 0.93 0.79 0.96 0.95
Canola 0.91 0.75 0.80 0.77 0.96 NA
Forest 0.99 0.99 0.99 0.99 0.99 0.99
GNV 0.95 0.85 0.99 0.81 0.99 0.93
Bare soil 0.99 0.99 0.99 0.99 0.99 0.99
Macro-averaging 0.95 0.88 0.94 0.87 0.98 0.96
Table 10
Producer's accuracy (PA) and user's accuracy (UA) of the leave-one-cluster-out-cross-validation (LOCOCV) model. GNV: Grazing native vegetation. Results are re-
ported from the test set. PA and UA of canola in site 6 is NA indicating no canola at this site in all growing seasons.
Class site 1 site 2 site 3 site 4 site 5 site 6
PA UA PA UA PA UA PA UA PA UA PA UA
Cereals 0.98 0.93 0.88 0.75 0.89 0.98 0.95 0.63 0.93 1 1 0.92
Canola 0.87 0.92 0.87 0.68 0.99 0.66 0.98 0.64 1 0.94 NA NA
Forest 1 1 1 1 1 1 1 1 1 1 1 1
GNV 0.95 1 0.77 1 0.98 1 0.67 1 1 1 0.89 1
Bare soil 1 1 0.98 1 1 1 1 1 1 1 1 1
Surprisingly, the EVI percentiles were at the bottom of variable importance plots for all sites. The EVI in May was also in the top six
variables in all sites, whereas VH winter and VV winter were less important in the LOCOCV than in the LOSOCV and the holdout mod-
els.
4. Discussion
Crop type classification is a common task in agriculture to identify what and where a specific vegetation cover is grown, and there
are many ways to solve this problem. In this study, we implemented three approaches to cover three scenarios where data availability
varies in space and time. The three validation approaches have resulted in different accuracies yet were reasonable for the LOSOCV
results and reasonable for the holdout and the LOCOCV models. The high accuracy achieved from the holdout method was expected
since all the season-to-season variability was considered when calibrating and validating the model (Song et al., 2017). None of the
classes achieved less than 0.97 P A or less than 0.96 UA using the holdout method. These results are reflected in the classified map in
Fig. 5, where most cereals and canola fields were well separated from each other or other classes with excellent delineation of field
boundaries. In contrast, the LOSOCV model's accuracy varied from season to season. This also was expected as the data were collected
from different growing seasons to the growing season being validated (Al-Shammari et al., 2020). The LOCOCV proved that it is possi-
ble to accurately classify unseen data (spatial extrapolation) within the MDB. However, this model showed that canola accuracies var-
ied considerably from site to site, and this was not noticed in cereals, which showed better results than canola. The LOCOCV-based
classifier identified canola as cereals in some sites where canola UA was low, whereas the classifier confused GNV with cereal (Wang
and Tenhunen, 2004). The concern is more related to the canola and cereal confusion as this leads to under/overestimating actual ar-
eas of cereals and canola. In contrast, the confusion of cereals with GNV is less critical as the GNV pixels that the LOCOCV model iden-
tified as cereals can be excluded (masked out) if the GNV boundaries in the MDB are known. In theory, the confusion between classes
can attributed to the natural within-field variability (Maestrini and Basso, 2018) in the fields where some patches may not be as
stressed as others, e.g., higher EVI values compared to the stressed crops, which have low EVI values. Thus, stressed crops are mixed
with e.g., GNV, which always have a relatively low EVI than cereals and canola. It is also possible to observe confusion between cere-
als and canola due to the same issue. Therefore, there may be a limitation in crop yield maps, and samples from these maps should be
carefully selected to avoid selecting unrepresentative samples and to extract the typical cereal samples. However, in this study, the

Remote Sensing Applications: Society and Environment 34 (2024) 101200
14
D. Al-Shammari et al.
aim was to test the ability of these maps to include all the within-field variability in the models and to show a real-world example of
using data from yield maps.
Given that every growing season has unique conditions (different amounts of rainfall, different management, etc.), the accuracies
of the LOSOCV model will depend on the amount of representation of these conditions in the data. For example, there was a notice-
able decline in accuracy when validating the model on data from the 2018 growing season. The decline in accuracy was attributed to
the change in the crop structure and, thus, the reflectance of different cover types. Reflectance and the selection of particular bands to
improve classification accuracy have been addressed in different studies (Peña et al., 2017;Al-Shammari et al., 2020;Kobayashi et
al., 2020;Planque et al., 2021). In this study, the poor crop structure (caused by poor crop density) in the 2018 growing season re-
duced the reflected EVI to a median of ∼0.32 for cereal and of 0.52 for canola in August (mid-season), whereas the EVI values in 2017
and 2019 had increased to higher than 0.50 for both crop types (Fig. 4. B). Therefore, the accuracy decreased when calibrating the
model on 2017 data (which was the wettest growing season among the three growing seasons in this study) and 2019 data (which was
drier than both 2018 and 2017). Furthermore, the variability in 2019 rainfall was higher than both 2017 and 2018, which impacts the
establishment of crops, so areas with adequate rainfall for crop production will be classified correctly. In contrast, areas that suffer
from drought may be misclassified.
The holdout and LOSOCV methods showed different results regarding the estimated areas of cereals and canola; both models re-
ported a fluctuation in the estimated areas of these two land cover types in some growing seasons. In addressing the variability in our
predictions of canola areas across different seasons, particularly the extreme reduction observed in 2018, we recognise the critical in-
fluence of two interrelated factors: the imbalance in data across study years and significant season-to-season variability in key model
features, such as reflectance signatures. The LOSOCV method assumes that data from seasons used in training are relevant and repre-
sentative of the test season. However, extreme events like the drought experienced in 2018 challenge this assumption, leading to pro-
nounced changes in the reflectance properties of canola fields that were not adequately represented in the training data. Factors such
as reduced soil moisture and changes in plant health during the drought directly affected the spectral signatures used for analysis,
contributing to the model's reduced accuracy for that year. While we initially attributed the varied performance across seasons in the
LOSOCV model to the drought, we acknowledge that issues in data distribution also likely impacted model performance since few
clusters were selected for validation, but this is related to data scarcity issues. This challenge requires a refined modelling approach
that integrates data balance and adaptive mechanisms to accommodate variation in critical features influenced by environmental
conditions, resulting in robust predictions across diverse agricultural scenarios. A study was conducted to measure the sensitivity of
wheat, barley, canola, chickpea, and field pea to water stress and temperature across Australia (Dreccer et al., 2018). They found that
canola is the most sensitive crop to water stress compared to the other crops. The results of the holdout method show the highest accu-
racies in this study. However, the outperformance obtained using the holdout method is a result of an oversimplistic sampling strat-
egy which overlooks the effect of spatial autocorrelation in the data. Therefore, LOSOCV and LOCOCV more realistically depict the
performance of the models evaluated.
Despite the effective separability that these multi-temporal predictor variables offered, some of these predictor variables were
more important than others. For example, according to the variable importance plot (Fig. 6), the most critical predictor variable in the
holdout model was the GPP of the winter growing season.
Furthermore, the VV and VH of the winter growing season in the holdout and LOSOCV were at the top five predictor variables in
terms of contribution to the model. The difference between crop types in terms of structure and during phenological stages has an in-
fluence on the backscattering intensities (Forkuor et al., 2014). This is because the VV is influenced by the volume scattering and VH
is influenced by the double-bouncing scattering between stem and ground (Planque et al., 2021). Therefore, the high importance of
the VV and VH predictors proved that these two predictors can discriminate different crop types with different structures. In addition,
the LOCOCV proved that VV and VH predictors are site-specific, as the variable importance of these two predictors changed from site
to site (Fig. 16). That means VV and VH predictors might not be as useful as the other predictors, e.g., the 95th percentile of the CIr
and the EVI in September, which were the highest-ranking predictors in all models. As mentioned, the EVI in September was at the
top 5 important predictor variables and the most stable predictor among all other predictors. This is very clear because the EVI of the
forest and the GNV classes remained stable in September, whereas the highest variation between canola and cereals was noticed in
September. The 95th percentile of the CIr index also ranked among the top five predictors. This is because the CIr is a red edge-based
index containing information that should help improve a crop classifier's accuracy (Kang et al., 2021). The contribution of the GPP
(winter), VH (winter), VV (winter), EVI September, and the 95th percentile of the CIr to the LOSOCV model also remained high when
classifying each growing season.
Lastly, the variation in the estimation of class areas was due to the variation inaccuracies produced by the two different models.
The areas estimated by the holdout model are more accurate than those estimated by the LOSOCV model because the holdout model
had a better discriminative ability.
The results of both LOCOCV and LOSOCV models are satisfactory, yet they highlight distinct challenges in classification accuracy
due to various factors. For LOCOCV, inconsistencies in agricultural practices across different sites, such as variations in planting and
harvesting, alongside diverse environmental conditions and soil types, can impact spectral signatures, affecting the model's precision.
Similarly, LOSOCV's performance is influenced by seasonal phenological changes and weather conditions, impacting RS data quality.
Notably, drought conditions in the north of the MDB present a unique challenge that may lead to the misclassification of cereals and
canola in failed fields. Therefore, the classification models cannot accurately classify cereals and canola in those fields. Therefore, dif-
ferent validation methods may be required depending on how the model will be used, such as for retrospective mapping or for map-
ping current or future seasons. For example, the hold-out method might be suitable for retrospective mapping, while LOSOCV might
be more appropriate for mapping current or future growing seasons. Including more growing seasons of data can improve the model's

Remote Sensing Applications: Society and Environment 34 (2024) 101200
15
D. Al-Shammari et al.
performance across different environmental conditions. Furthermore, yield monitor maps as reference data (ground truthing) are
beneficial but also must be used with caution, and this is because within-field variability of biomass is commonplace in the field. This
implies that field pixels have different reflectance across growing seasons. One way to deal with the within-field variability is by tak-
ing the mean reflectance of a field, but this requires a large number of yield maps. Lastly, it is preferable to know the field boundaries,
so a percentage of pixels belonging to a specific crop type within a field boundary can be used to identify the entire field as this spe-
cific crop type. It is possible to identify field boundaries using a model that have been developed to map field extent, boundaries, and
individual fields using a single monthly composite image from Sentinel 2 (Waldner and Diakogiannis, 2020).
5. Conclusions
This study examined the potential of combining multi-temporal, multi-source data (obtained from Sentinel 1, Sentinel 2, and
MODIS) to classify two major crop types (cereals and canola) in the MDB in Australia. Three methods have been implemented: the
holdout model, where all data from all growing seasons were selected for training, validating and testing the model; the LOSOCV,
where new data from a new growing season was used to test model quality; and the LOCOCV, where each cluster (site) was left out for
testing. The results showed that the highest separability between classes was achieved from the holdout method. The LOSOCV
method is subjected to season-to-season variation in data; subsequently, accuracy depends on that variation. The results revealed that
in the 2018 growing season, the accuracy decreased mainly due to drought in the MDB. This suggests that more data from different
growing seasons and areas in the MDB would be needed to improve classification quality. The LOCOCV showed promising results
where high separability between classes was achieved using this model. According to LOCOCV results, it is possible to extrapolate the
model to classify unseen cereals and canola fields within the MDB area. The results also showed that the MODIS-GPP of winter grow-
ing seasons provides a significant data layer for crop type classification (Gilmanov et al., 2003;Turner et al., 2006). The radar data
(VH and VV) also provided high-quality information for discriminating crop types, as different crop types have different structure. Re-
constructing the EVI by averaging its monthly values helped in two ways. Firstly, averaging the EVI for each month helped fill the
missing data gaps. Secondly, due to the slight temporal variation in signature, which results from different sowing dates, averaging
the signal for each month helped to average that variation and created a unique signature for each land cover type. The contribution
of the CIr index was also evident, where it was always among the top six predictors in the two methods. This indicated the importance
of incorporating the red edge information in the classification models. The proposed model and outputs in this study could be incor-
porated in future studies for carbon modelling and crop yield prediction. This leads to more precise and valuable predictions for grow-
ers, policymakers, and other stakeholders in the agricultural sector.
Ethical_statement
I declare that all ethical practices have been followed in relation to the development, writing, and publication of the article.
CRediT authorship contribution statement
Dhahi Al-Shammari: Writing –review & editing, Writing –original draft, Methodology, Formal analysis, Data curation, Con-
ceptualization. Ignacio Fuentes: Writing –review & editing, Methodology, Formal analysis. Brett M. Whelan: Writing –review &
editing, Supervision. Chen Wang: Writing –review & editing, Supervision. Patrick Filippi: Writing –review & editing. Thomas
F.A. Bishop: Writing –review & editing, Supervision.
Declaration of competing interest
The authors declare the following financial interests/personal relationships which may be considered as potential competing in-
terests: Dhahi Al-Shammari reports financial support was provided by Commonwealth Scientific and Industrial Research Organisa-
tion & Data61. Dhahi Al-Shammari reports a relationship with Commonwealth Scientific and Industrial Research Organisation &
Data61 that includes: funding grants.
Data availability
The data that has been used is confidential.
Acknowledgements
The corresponding author would like to gratefully acknowledge the financial support (CSIRO/Data61 Postgraduate Research
Stipend and Supplementary Scholarship in Digital Agriculture) from the Commonwealth Scientific and Industrial Research Organisa-
tion (CSIRO).

Remote Sensing Applications: Society and Environment 34 (2024) 101200
16
D. Al-Shammari et al.
Appendices
Fig. 10. Classified map of winter crops versus other cover types of the 2019 growing season in site 1.
Fig. 11. Classified map of winter crops versus other cover types of the 2019growing season in site 2.

Remote Sensing Applications: Society and Environment 34 (2024) 101200
17
D. Al-Shammari et al.
Fig. 12. Classified map of winter crops versus other cover types of the 2019 growing season in site 3.
Fig. 13. Classified map of winter crops versus other cover types of the 2017 growing season in site 4.
Fig. 14. Classified map of winter crops versus other cover types of the 2019 growing season in site 5.

Remote Sensing Applications: Society and Environment 34 (2024) 101200
18
D. Al-Shammari et al.
Fig. 15. Classified map of winter crops versus other cover types of the 2019 growing season in site 6.

Remote Sensing Applications: Society and Environment 34 (2024) 101200
19
D. Al-Shammari et al.
Fig. 16. Variable importance of the leave-one-cluster-out-cross-validation (LOCOCV) model. GPP: Gross primary production; VH (winter): VH of the winter growing
season; VV: VV of the winter growing season. EVI (p5), EVI (p50), and EVI (p95) are the 5, 50, and 95 percentiles of the EVI vegetation index. CIr (p5), CIr (p50 ), and CIr

Remote Sensing Applications: Society and Environment 34 (2024) 101200
20
D. Al-Shammari et al.
◀
Fig. 16. (continued)
(p95) are the 5, 50, and 95 percentiles of the CIr vegetation index. May to October predictor variables represent the EVI of each month. Plots 1 to 6 corresponds to sites
1 to 6, respectively.
Table 11
Confusion matrix with the producer's accuracy (PA) and user's accuracy (UA) of the holdout model. GNV: Grazing native vegetation. Results are reported from the
test set.
Labelled data
Cereals Canola Forest GNV Bare soil UA
Classified data Cereals 5931 51 0 21 0 0.98
Canola 14 2969 0 3 0 0.99
Forest 0 1 3038 33 0 0.98
GNV 47 31 18 2960 0 0.96
Bare soil 2 0 0 1 1868 0.99
PA 0.98 0.97 0.99 0.98 1
Table 12
Confusion matrix with producer's accuracy (PA) and user's accuracy (UA) of the leave-one-season-out-cross-validation (LOSOCV) model. GNV: Grazing native vege-
tation.
Labelled data 2017
Cereals Canola Forest GNV Bare soil UA
Classified data 2017 Cereals 6060 280 0 12 0 0.95
Canola 1331 4462 0 32 0 0.76
Forest 0 2 4869 93 0 0.98
GNV 2573 254 131 4978 337 0.60
Bare soil 36 2 0 5 2779 0.98
PA 0.60 0.89 0.97 0.97 0.89
Labelled data 2018
Cereals Canola Forest GNV Bare soil UA
Classified data 2018 Cereals 8572 2645 0 157 5 0.75
Canola 3 2260 0 2 0 0.99
Forest 0 0 3066 43 0 0.98
GNV 1425 95 1934 5038 2170 0.47
Bare soil 0 0 0 5 959 0.99
PA 0.85 0.45 0.61 0.96 0.30
Labelled data 2019
Cereals Canola Forest GNV Bare soil UA
Classified data 2019 Cereals 9567 374 0 61 231 0.93
Canola 277 4151 0 46 0 0.92
Forest 0 0 4878 45 0 0.99
GNV 151 475 122 5066 376 0.81
Bare soil 0 0 0 5 2527 0.99
PA 0.95 0.83 0.97 0.96 0.80
Table 13
Confusion matrix with producer's accuracy (PA) and user's accuracy (UA) of the leave-one-cluster-out-cross-validation (LOCOCV) model. GNV: Grazing native vege-
tation.
Site 1
Labelled data
Classified data Cereals Canola Forest GNV Bare soil UA
Cereals 20 943 983 0 574 0 0.93
Canola 453 6871 0 176 0 0.92
Forest 0 0 14 494 0 0 1
GNV 0 0 0 15 589 0 1
Bare soil 0 0 0 0 14 792 1
PA 0.98 0.87 1 0.95 1
(continued on next page)

Remote Sensing Applications: Society and Environment 34 (2024) 101200
21
D. Al-Shammari et al.
Table 13 (continued)
Site 2
Labelled data
Classified data Cereals Canola Forest GNV Bare soil UA
Cereals 16 941 768 0 4514 277 0.75
Canola 2259 5077 0 161 0 0.68
Forest 0 0 14 493 1 0 1
GNV 0 0 2 15 586 1 1
Bare soil 0 0 0 2 14 790 1
PA 0.88 0.87 1 0.77 0.98
Site 3
Labelled data
Classified data Cereals Canola Forest GNV Bare soil UA
Cereals 19 696 42 0 262 0 0.98
Canola 2487 4929 0 82 0 0.66
Forest 0 0 14 493 1 0 1
GNV 4 0 6 15 579 0 1
Bare soil 0 0 0 0 14 792 1
PA 0.89 0.99 1 0.98 1
Site 4
Labelled data
Classified data Cereals Canola Forest GNV Bare soil UA
Cereals 3132 31 0 1837 0 0.63
Canola 182 1599 12 707 0 0.64
Forest 0 0 4664 0 0 1
GNV 0 0 0 5120 0 1
Bare soil 0 0 0 0 4901 1
PA 0.95 0.98 1 0.67 1
Site 5
Labelled data
Classified data Cereals Canola Forest GNV Bare soil UA
Cereals 4994 0 0 6 0 1
Canola 388 7081 0 31 0 0.94
Forest 0 0 14 494 0 0 1
GNV 3 0 2 15 584 0 1
Bare soil 0 0 0 0 14 792 1
PA 0.93 1 1 1 1
Site 6
Labelled data
Classified data Cereals Canola Forest GNV Bare soil UA
Cereals 15 011 –1 1243 11 0.92
Canola – – – – – NA
Forest 0 –9663 1 0 1
GNV 0 –0 10 342 1 1
Bare soil 0 –0 0 9846 1
PA 1 NA 1 0.89 1
References
Al-Shammari, D., Bishop, T., Fuentes, I., Filippi, P., 2019. Mapping cotton fields using phenology-based metrics derived from a time series of Landsat imagery. In:
Proceedings of the 2019 Agronomy Australia Conference. Wagga Wagga, Australia.
Al-Shammari, D., Fuentes, I., M. Whelan, B., Filippi, P., Fa Bishop, T., 2020. Mapping of cotton fields within-season using phenology-based metrics derived from a time
series of landsat imagery. Rem. Sens. 12 (18), 3038. https://doi.org/10.3390/rs12183038.
Arias, M., Campo-Bescós, M.Á., Álvarez-Mozos, J., 2020. Crop classification based on temporal signatures of Sentinel-1 observations over Nava rre province, Spain. Rem.
Sens. 12 (2), 278. https://doi.org/10.3390/rs12020278.
Australian Bureau of Meteorology, A. g, 2011. Murray-darling basin physical information. Retrieved July 10 from. http://www.bom.gov.au/water/nwa/2011/mdb/
index.shtml.
Bracewell, R.N., Bracewell, R.N., 1986. The Fourier Transform and its Applications, vol. 31999. McGraw-Hill, New York. https://doi.org/10.1119/1.1973431.
Breiman, L., 2001. Random forests. Mach. Learn. 45, 5–32. https://doi.org/10.1023/A:1010933404324.
Burger, A., 1984. Crop classification. Physiol. Basis of Crop Growth and Dev. 1–12. https://doi.org/10.2135/1984.physiologicalbasis.c1.

Remote Sensing Applications: Society and Environment 34 (2024) 101200
22
D. Al-Shammari et al.
Buston, P.M., Elith, J., 2011. Determinants of reproductive success in dominant pairs of clownfish: a boosted regression tree analysis. J. Anim. Ecol. 80 (3), 528–538.
https://doi.org/10.1111/j.1365-2656.2011.01803.x.
Chagas, M.C., Delgado, R.C., de Souza, L.P., de Carvalho, D.C., Pereira, M.G., Teodoro, P.E., Junior, C.A.S., 2019. Gross primary productivity in areas of different land
cover in the western Brazilian Amazon. Remote Sens. Appl.: Soc. Environ. 16, 100259. https://doi.org/10.1016/j.rsase.2019.100259.
Conese, C., Maselli, F., 1991. Use of multitemporal information to improve classification performance of TM scenes in complex terrain. ISPRS J. Photogrammetry
Remote Sens. 46 (4), 187–197. https://doi.org/10.1016/0924-2716(91)90052-W.
Dalsasso, E., Yang, X., Denis, L., Tupin, F., Yang, W., 2020. SAR image despeckling by deep neural networks: from a pre-trained model to an end-to-end training
strategy. Rem. Sens. 12 (16), 2636. https://doi.org/10.3390/rs12162636.
Dappen, P.R., Ratcliffe, I.C., Robbins, C.R., Merchant, J.W., 2008. Mapping Agricultural Land Cover for Hydrologic Modeling in th ePl atte River Watershed of Nebraska.
Great Plains Research. pp. 39–52.
Demarez, V., Helen, F., Marais-Sicre, C., Baup, F., 2019. In-season mapping of irrigated crops using Landsat 8 and Sentinel-1 time series. Rem. Sens. 11 ( 2), 118. https://
doi.org/10.3390/rs11020118.
Dey, S., Chaudhuri, U., Bhogapurapu, N., Lopez-Sanchez, J.M., Banerjee, B., Bhattacharya, A., Mandal, D., Rao, Y.S., 2021. Synergistic use of TanDEM-X and landsat-8
data for crop-type classification and monitoring. IEEE J. Sel. Top. Appl. Earth Obs. Rem. Sens. 14, 8744–8760. https://doi.org/10.1109/JSTARS.2021.3103911.
Doraiswamy, P., McCarty, G., Hunt Jr, E., Yost, R., Doumbia, M., Franzluebbers, A., 2007. Modeling soil carbon sequestration in agricultural lands of Mali. Agric. Syst.
94 (1), 63–74. https://doi.org/10.1016/j.agsy.2005.09.011.
Dreccer, M.F., Fainges, J., Whish, J., Ogbonnaya, F.C., Sadras, V.O., 2018. Comparison of sensitive stages of wheat, barley, canola, chickpea and field pea to
temperature and water stress across Australia. Agric. For. Meteorol. 248, 275–294. https://doi.org/10.1016/j.agrformet.2017.10.006.
Evans, J., McCabe, M., 2010. Regional climate simulation over Australia’s Murray-Darling basin: a multitemporal assessment. J. Geophys. Res. Atmos. 115 (D14).
https://doi.org/10.1029/2010JD013816.
Forkuor, G., Conrad, C., Thiel, M., Ullmann, T., Zoungrana, E., 2014. Integration of optical and synthetic aperture radar imagery for improving crop mapping in
northwestern Benin, west africa. Rem. Sens. 6 (7), 6472–6499. https://doi.org/10.3390/rs6076472.
Forkuor, G., Dimobe, K., Serme, I., Tondoh, J.E., 2018. Landsat-8 vs. Sentinel-2: examining the added value of sentinel-2’s red-edge bands to land-use and land-cover
mapping in Burkina Faso. GIScience Remote Sens. 55 (3), 331–354. https://doi.org/10.1080/15481603.2017.1370169.
Geiß, C., Pelizari, P.A., Schrade, H., Brenning, A., Taubenböck, H., 2017. On the effect of spatially non-disjoint training and test samples on estimated model
generalization capabilities in supervised classification with spatial features. Geosci. Rem. Sens. Lett. IEEE 14 (11), 2008–2012.
Gilmanov, T.G., Verma, S.B., Sims, P.L., Meyers, T.P., Bradford, J.A., Burba, G.G., Suyker, A.E., 2003. Gross primary production and light response parameters of four
Southern Plains ecosystems estimated using long-term CO2-flux tower measurements. Global Biogeochem. Cycles 17 (2). https://doi.org/10.1029/2002GB002023.
Gitelson, A.A., Peng, Y., Masek, J.G., Rundquist, D.C., Verma, S., Suyker, A., Baker, J.M., Hatfield, J.L., Meyers, T., 2012. Remote estimation of crop gross primary
production with Landsat data. Rem. Sens. Environ. 121, 404–414. https://doi.org/10.1016/j.rse.2012.02.017.
Gitelson, A.A., Viña, A., Arkebauer, T.J., Rundquist, D.C., Keydan, G., Leavitt, B., 2003. Remote estimation of leaf area index and green leaf biomass in maize canopies.
Geophys. Res. Lett. 30 (5). https://doi.org/10.1029/2002GL016450.
Gorelick, N., Hancher, M., Dixon, M., Ilyushchenko, S., Thau, D., Moore, R., 2017. Google Earth engine: planetary-scale geospatial analysis for everyone. Rem. Sens.
Environ. 202, 18–27. https://doi.org/10.1016/j.rse.2017.06.031.
Gray, J.M., Bishop, T.F., Smith, P.L., 2016. Digital mapping of pre-European soil carbon stocks and decline since clearing over New South Wales, Australia. Soil Res. 54
(1), 49–63. https://doi.org/10.1071/SR14307.
Hagolle, O., Huc, M., Pascual, D.V., Dedieu, G., 2010. A multi-temporal method for cloud detection, applied to FORMOSAT-2, VENμS, LANDSAT and SENTINEL-2
images. Rem. Sens. Environ. 114 (8), 1747–1755. https://doi.org/10.1016/j.rse.2010.03.002.
He, M., Kimball, J.S., Maneta, M.P., Maxwell, B.D., Moreno, A., Beguería, S., Wu, X., 2018. Regional crop gross primary productivity and yield estimation using fused
landsat-MODIS data. Rem. Sens. 10 (3), 372. https://doi.org/10.3390/rs10030372.
Hill, M.J., Ticehurst, C.J., Lee, J.-S., Grunes, M.R., Donald, G.E., Henry, D., 2005. Integration of optical and radar classifications for mapping pasture type in Western
Australia. IEEE Trans. Geosci. Rem. Sens. 43 (7), 1665–1681. https://doi.org/10.1109/TGRS.2005.846868.
Ho, T.K., 1995. Random decision forests. Proc. 3rd Int. Conf. Document Anal. Recog.
Jakubauskas, M.E., Legates, D.R., Kastens, J.H., 2001. Harmonic analysis of time-series AVHRR NDVI data. Photogramm. Eng. Rem. Sens. 67 (4), 461–470.
Kang, Y., Meng, Q., Liu, M., Zou, Y., Wang, X., 2021. Crop classification based on red edge features analysis of GF-6 WFV data. Sensors 21 (13), 4328. https://doi.org/
10.3390/s21134328.
Khan, A., Hansen, M.C., Potapov, P.V., Adusei, B., Pickens, A., Krylov, A., Stehman, S.V., 2018. Evaluating Landsat and RapidEye data for winter wheat mapping and
area estimation in Punjab, Pakistan. Rem. Sens. 10 (4), 489. https://doi.org/10.3390/rs10040489.
Kirby, M., Bark, R., Connor, J., Qureshi, M.E., Keyworth, S., 2014. Sustainable irrigation: how did irrigated agriculture in Australia’s murray–darling basin adapt in the
millennium drought? Agric. Water Manag. 145, 154–162. https://doi.org/10.1016/j.agwat.2014.02.013.
Kobayashi, N., Tani, H., Wang, X., Sonobe, R., 2020. Crop classification using spectral indices derived from Sentinel-2A imagery. J. Inf. Telecommun. 4 (1), 67–90.
Lark, R., Stafford, J., 1997. Classification as a first step in the interpretation of temporal and spatial variation of crop yield. Ann. Appl. Biol. 130 (1), 111–121. https://
doi.org/10.1111/j.1744-7348.1997.tb05787.x.
Li, M., Wu, T., Wang, S., Sang, S., Zhao, Y., 2022. Phenology–gross primary productivity (GPP) method for crop information extraction in areas sensitive to non-point
source pollution and its influence on pollution intensity. Rem. Sens. 14 (12), 2833. https://doi.org/10.3390/rs14122833.
Liu, H.Q., Huete, A., 1995. A feedback based modification of the NDVI to minimize canopy background and atmospheric noise. IEEE Trans. Geosci. Rem. Sens. 33 (2),
457–465. https://doi.org/10.1109/TGRS.1995.8746027.
Maestrini, B., Basso, B., 2018. Drivers of within-field spatial and temporal variability of crop yield across the US Midwest. Sci. Rep. 8 (1), 14833. https://doi.org/
10.1038/s41598-018-32779-3.
Massey, R., Sankey, T.T., Congalton, R.G., Yadav, K., Thenkabail, P.S., Ozdogan, M., Meador, A.J.S., 2017. MODIS phenology-derived, multi-year distribution of
conterminous US crop types 198, 490–503. https://doi.org/10.1016/j.rse.2017.06.033.
Mingwei, Z., Qingbo, Z., Zhongxin, C., Jia, L., Yong, Z., Chongfa, C., 2008. Crop discrimination in Northern China with double cropping systems using Fourier analysis
of time-series MODIS data. Int. J. Appl. Earth Obs. Geoinf. 10 (4), 476–485. https://doi.org/10.1016/j.jag.2007.11.002.
Misra, P.N., Wheeler, S., 1978. Crop classification with LANDSAT multispectral scanner data. Pattern Recogn. 10 (1), 1–13. https://doi.org/10.1016/0031-3203(78)
90042-0.
Moody, A., Johnson, D.M., 2001. Land-surface phenologies from AVHRR using the discrete Fourier transform. Rem. Sens. Environ. 75 (3), 305–323. https://doi.org/
10.1016/S0034-4257(00)00175-9.
Palchowdhuri, Y, Valcarce-Diñeiro, R, King, P, Sanabria-Soto, M, 2018. Classification of multi-temporal spectral indices for crop type mapping: a case study in Co alville,
UK. J. Agri. Sci. 156 (1), 24–36. https://doi.org/10.1017/S0021859617000879.
Paul, S., Kumar, D.N., 2019. Comparison of landsat-8 and sentinel-2 data for classification of rabi crops over Karnataka, India. Int. Arch. Photogram. Rem. Sens. Spatial
Inf. Sci. https://doi.org/10.5194/isprs-archives-XLII-3-W6-579-2019.
Peña, M.A., Liao, R., Brenning, A., 2017. Using spectrotemporal indices to improve the fruit-tree crop classification accuracy. ISPRS J. Photogrammetry Remote Sens.
128, 158–169.
Planque, C., Lucas, R., Punalekar, S., Chognard, S., Hurford, C., Owers, C., Horton, C., Guest, P., King, S., Williams, S., 2021. National crop mapping using sentinel-1
time series: a knowledge-based descriptive algorithm. Rem. Sens. 13 (5), 846. https://doi.org/10.3390/rs13050846.
Potter, N., Chiew, F., Frost, A.J. J.o. h., 2010. An Assessment of the Severity of Recent Reductions in Rainfall and Runoff in the Murray–Darling Basin, vol. 381. pp.
52–64. https://doi.org/10.1016/j.jhydrol.2009.11.025. 1-2.
Prosser, I.P., Chiew, F.H., Stafford Smith, M., 2021. Adapting water management to climate change in the Murray–Darling Basin, Australia. Water 13 (18), 2504.
https://doi.org/10.3390/w13182504.
Rao, P., Zhou, W., Bhattarai, N., Srivastava, A.K., Singh, B., Poonia, S., Lobell, D.B., Jain, M., 2021. Using Sentinel-1, Sentinel-2, and Planet imagery to map crop type of

Remote Sensing Applications: Society and Environment 34 (2024) 101200
23
D. Al-Shammari et al.
smallholder farms. Rem. Sens. 13 (10), 1870. https://doi.org/10.3390/rs13101870.
Rodriguez-Galiano, V.F., Ghimire, B., Rogan, J., Chica-Olmo, M., Rigol-Sanchez, J.P., 2012. An assessment of the effectiveness of a random forest classifier for land-
cover classification. ISPRS J. Photogrammetry Remote Sens. 67, 93–104. https://doi.org/10.1016/j.isprsjprs.2011.11.002.
Schultz, B., Immitzer, M., Formaggio, A.R., Sanches, I.D.A., Luiz, A.J.B., Atzberger, C.J.R.S., 2015. Self-guided Segmentation and Classification of Multi-Temporal
Landsat 8 Images for Crop Type Mapping in Southeastern Brazil, vol. 7. pp. 14482–14508. https://doi.org/10.3390/rs71114482. 11.
Scientists, W.G.o.C., 2017. Review of Water Reform in the Murray Darling Basin.
Shaik, A.B., Srinivasan, S., 2019. A brief survey on random forest ensembles in classification model. In: International Conference on Innovative Computing and
Communications: Proceedings of ICICC 2018, vol. 2.
Song, Q., Hu, Q., Zhou, Q., Hovis, C., Xiang, M., Tang, H., Wu, W., 2017. In-season crop mapping with GF-1/WFV data by combining object-based image analysis and
random forest. Rem. Sens. 9 (11), 1184. https://doi.org/10.3390/rs9111184.
Song, Y., Wang, J., 2019. Mapping winter wheat planting area and monitoring its phenology using Sentinel-1 backscatter time series. Rem. Sens. 11 (4), 449. https://
doi.org/10.3390/rs11040449.
Srivastava, A., Kumari, N., Maza, M., 2020. Hydrological response to agricultural land use heterogeneity using variable infiltration capacity model. Water Resour.
Manag. 34 (12), 3779–3794. https://doi.org/10.1007/s11269-020-02630-4.
Sun, C., Bian, Y., Zhou, T., Pan, J., 2019. Using of multi-source and multi-temporal remote sensing data improves crop-type mapping in the subtropical agriculture
region. Sensors 19 (10), 2401. https://doi.org/10.3390/s19102401.
Tatsumi, K., Yamashiki, Y., Torres, M.A.C., Taipe, C.L.R., 2015. Crop classification of upland fields using Random forest of time-series Landsat 7 ETM+data. Comput.
Electron. Agric. 115, 171–179. https://doi.org/10.1016/j.compag.2015.05.001.
Tennakoon, S., Milroy, S., 2003. Crop water use and water use efficiency on irrigated cotton farms in Australia. Agric. Water Manag. 61 (3), 179–194. https://doi.org/
10.1016/S0378-3774(03)00023-4.
Turner, D.P., Ritts, W.D., Cohen, W.B., Gower, S.T., Running, S.W., Zhao, M., Costa, M.H., Kirschbaum, A.A., Ham, J.M., Saleska, S.R., 2006. Evaluation of MODIS NPP
and GPP products across multiple biomes. Rem. Sens. Environ. 102 (3–4), 282–292. https://doi.org/10.1016/j.rse.2006.02.017.
Ugbaje, S.U., Odeh, I.O., Bishop, T.F., Li, J., 2017. Assessing the spatio-temporal variability of vegetation productivity in Africa: quantifying the relative roles of climate
variability and human activities. Int. J. Digit. Earth 10 (9), 879–900. https://doi.org/10.1080/17538947.2016.1265017.
Vaudour, E., Gholizadeh, A., Castaldi, F., Saberioon, M., Borůvka, L., Urbina-Salazar, D., Fouad, Y., Arrouays, D., Richer-de-Forges, A.C., Biney, J., 2022. Satellite
imagery to map topsoil organic carbon content over cultivated areas: an overview. Rem. Sens. 14 (12), 2917. https://doi.org/10.3390/rs14122917.
Vervoort, R., Guillaume, J., Bishop, T., Kundu, D., van Ogtrop, F., 2016. Using an Object and Pattern Oriented Approach to Hydrological Modelling Teaching and
Research.
Vreugdenhil, M., Wagner, W., Bauer-Marschallinger, B., Pfeil, I., Teubner, I., Rüdiger, C., Strauss, P., 2018. Sensitivity of Sentinel-1 backscatter to vegetation dynamics:
an Austrian case study. Rem. Sens. 10 (9), 1396. https://doi.org/10.3390/rs10091396.
Vuolo, F., Neuwirth, M., Immitzer, M., Atzberger, C., Ng, W.-T., 2018. How much does multi-temporal Sentinel-2 data improve crop type classification? 72, 122–130.
https://doi.org/10.1016/j.jag.2018.06.007.
Waldner, F., Diakogiannis, F.I., 2020. Deep learning on edge: extracting field boundaries from satellite images with a convolutional neural network. Rem. Sens. En viron.
245, 111741. https://doi.org/10.1016/j.rse.2020.111741.
Wang, J., Xiao, X., Liu, L., Wu, X., Qin, Y., Steiner, J.L., Dong, J., 2020. Mapping sugarcane plantation dynamics in Guangxi, China, by time series Sentinel-1, Sentinel-2
and Landsat images. Rem. Sens. Environ. 247, 111951. https://doi.org/10.1016/j.rse.2020.111951.
Wang, Q., Tenhunen, J.D., 2004. Vegetation mapping with multitemporal NDVI in North Eastern China transect (NECT). Int. J. Appl. Earth Obs. Geoinf. 6 (1), 17–31.
https://doi.org/10.1016/j.jag.2004.07.002.
Wang, S., Azzari, G., Lobell, D.B., 2019. Crop type mapping without field-level labels: random forest transfer and unsupervised clustering techniques. Rem. Sens.
Environ. 222, 303–317. https://doi.org/10.1016/j.rse.2018.12.026.
Whelen, T., Siqueira, P., 2018. Time-series classification of Sentinel-1 agricultural data over North Dakota. Rem. Sens. Lett. 9 (5), 411–420. https://doi.org/10.1080/
2150704X.2018.1430393.
Wolanin, A., Camps-Valls, G., Gómez-Chova, L., Mateo-García, G., van der Tol, C., Zhang, Y., Guanter, L., 2019. Estimating crop primary productivity with Sentinel-2
and Landsat 8 using machine learning methods trained with radiative transfer simulations. Rem. Sens. Environ. 225, 441–457. https://doi.org/10.1016/
j.rse.2019.03.002.
Wolfe, E., 2020. Country pasture profile Australia. Retrieved July 10 from. https://cdn.csu.edu.au/__data/assets/pdf_file/0008/3376970/Australian-Pasture-Profile-
6.pdf.
Xu, F., Li, Z., Zhang, S., Huang, N., Quan, Z., Zhang, W., Liu, X., Jiang, X., Pan, J., Prishchepov, A.V., 2020. Mapping winter wheat with combinations of temporally
aggregated Sentinel-2 and Landsat-8 data in Shandong Province, China. Rem. Sens. 12 (12), 2065. https://doi.org/10.3390/rs12122065.
Yang, C., Everitt, J.H., Murden, D.J.C., Agriculture, E.i., 2011. Evaluating High Resolution SPOT 5 Satellite Imagery for Crop Identification, vol. 75. pp. 347–354.
https://doi.org/10.1016/j.compag.2010.12.012. 2.
Yang, Y., Liu, X., 1999. A re-examination of text categorization methods. In: Proceedings of the 22nd Annual International ACM SIGIR Conference on Research and
Development in Information Retrieval.