Content uploaded by Bernhard Jenny
Author content
All content in this area was uploaded by Bernhard Jenny on Nov 04, 2020
Content may be subject to copyright.
Full Terms & Conditions of access and use can be found at
https://www.tandfonline.com/action/journalInformation?journalCode=tcag20
Cartography and Geographic Information Science
ISSN: (Print) (Online) Journal homepage: https://www.tandfonline.com/loi/tcag20
Elevation models for reproducible evaluation of
terrain representation
Patrick J. Kennelly , Tom Patterson , Bernhard Jenny , Daniel P. Huffman ,
Brooke E. Marston , Sarah Bell & Alexander M. Tait
To cite this article: Patrick J. Kennelly , Tom Patterson , Bernhard Jenny , Daniel P. Huffman ,
Brooke E. Marston , Sarah Bell & Alexander M. Tait (2020): Elevation models for reproducible
evaluation of terrain representation, Cartography and Geographic Information Science, DOI:
10.1080/15230406.2020.1830856
To link to this article: https://doi.org/10.1080/15230406.2020.1830856
Published online: 04 Nov 2020.
Submit your article to this journal
View related articles
View Crossmark data
Elevation models for reproducible evaluation of terrain representation
Patrick J. Kennelly
a
‡
, Tom Patterson
b
‡
, Bernhard Jenny
c
‡
, Daniel P. Human
d
, Brooke E. Marston ,
Sarah Bell
e
and Alexander M. Tait
f
a
Department of Biological and Environmental Science, Long Island University, Brookville, NY, USA;
b
U.S. National Parks Service (Ret.), Harpers
Ferry, WV, USA;
c
Faculty of Information Technology, Monash University, Melbourne, Australia;
d
somethingaboutmaps, Madison, WI, USA;
e
Esri
Inc., Redlands, CA, USA;
f
National Geographic Society, Washington, DC, USA
ABSTRACT
This paper proposes elevation models to promote, evaluate, and compare various terrain repre-
sentation techniques. Our goal is to increase the reproducibility of terrain rendering algorithms
and techniques across dierent scales and landscapes. We introduce elevation models of varying
terrain types, available to the user at no cost, with minimal common data imperfections such as
missing data values, resampling artifacts, and seams. Three multiscale elevation models are
available, each consisting of a set of elevation grids, centered on the same geographic location,
with increasing cell sizes and spatial extents. We also propose a collection of single-scale elevation
models of archetypal landforms including folded ridges, a braided riverbed, active and stabilized
sand dunes, and a volcanic caldera. An inventory of 78 publications with a total of 155 renderings
illustrating terrain visualization techniques guided the selection of landform types in the elevation
models. The benets of using the proposed elevation models include straightforward comparison
of terrain representation methods across dierent publications and better documentation of the
source data, which increases the reproducibility of terrain representations.
ARTICLE HISTORY
Received 10 July 2020
Accepted 28 September 2020
KEYWORDS
Reproducibility; terrain
rendering; terrain
visualization; digital
elevation model; shaded
relief
1. Elevation models for evaluating and
comparing cartographic visualizations
Evaluating and comparing cartographic visualizations
with quantitative metrics is rarely successful. Because
cartographic visualization methods vary widely and are
highly subjective, a single ground truth reference image
is often unavailable. Therefore, qualitative visual inspec-
tion and comparison are the preferred methods for
evaluating cartographic representations. Visual inspec-
tion and comparison are also common for evaluating
information visualizations (Isenberg et al., 2013) or
computer graphics renderings where researchers fre-
quently use a relatively small set of standard three-
dimensional models, such as the Utah teapot
(Lehmann, 2012) or the Stanford bunny (Turk, 2000).
By using a small set of models, researchers can compare
different rendering methods across one or more pub-
lications and simplify the verification and reproduction
of results by others. In the field of cartography however,
there are no similar sets of data models that researchers,
software developers, and mapmakers can use to evaluate
and compare visualization methods. In this paper, we
propose elevation models for comparing terrain render-
ing techniques.
Many studies introducing new or improved terrain
rendering techniques have evaluated results by visually
comparing a new rendering with well-known and
sometimes historical hand-produced renderings.
Examples include comparing digital rock drawing
methods with Swiss topographic maps (Geisthövel &
Hurni, 2018), comparing hand-drawn relief shading
with historical masterpieces (Bell, 2018), and compar-
ing plan oblique relief with Raisz’s landform maps
(Jenny & Patterson, 2007). Comparing new terrain
rendering techniques with existing and commonly
used algorithms, such as comparing custom relief
shading techniques with algorithms found in geo-
graphic information system (GIS) software (e.g.,
Huffman, 2017; Kennelly & Stewart, 2014; Marston &
Jenny, 2015), is an alternative approach. These studies
involve locating appropriate high-quality elevation
data, an oftentimes tedious task that could be simpli-
fied with a predefined collection of elevation models.
Additionally, the elevation model used in one study
may not be available again in the future as improved or
higher resolution elevation data tend to supersede
older data, making it difficult for future authors to
reproduce methods and results.
CONTACT Bernhard Jenny bernie.jenny@monash.edu
‡
shared co-first authorship
CARTOGRAPHY AND GEOGRAPHIC INFORMATION SCIENCE
https://doi.org/10.1080/15230406.2020.1830856
© 2020 Cartography and Geographic Information Society
Standard geometry models from computer graphics
take a decidedly object-based approach to representing
one item, such as a bunny or teapot. Using computer
graphics as an example, we propose elevation models at
single predefined scales of archetypal, isolated land-
forms. We also provide elevation models that include
a variety of landforms and vary in spatial extent and cell
size to make our model as utilitarian as possible.
Reusing data models is particularly useful for geo-
graphic information science (GIScience), which strives
for reproducible research. In computer science, Peng
(2011) considers reproducible research as a continuum
from publication to full replication where the associated
components move from code, to code plus data, to
linked and executable code plus data. Much of the
current discussion focuses on reproducible modeling
(Gahegan, 2019) in the geosciences (Gil et al., 2016),
specifically in GIScience (Brunsdon & Comber, 2020;
Kedron et al., 2019, 2020; Nüst et al., 2018; Shannon &
Walker, 2018). The associated workflows (Cerutti et al.,
2019), figures and maps (Konkol & Kray, 2019), and
associated code (Giraud & Lambert, 2017) focus on the
code side of Peng’s continuum. This work contributes to
the data side of the continuum in order to fill this gap in
reproducible research.
Our primary goal is to introduce free, easily accessi-
ble elevation models that facilitate reproducible com-
parisons of terrain renderings varying in scale and
landform type. The models should minimize erroneous
artifacts, which can be visually distracting or interrupt
the visual continuity of a terrain rendering.
To achieve our primary goal, we aim to (a) facilitate
the comparison of rendering techniques across different
publications and software; (b) reduce ambiguity around
data sources, terrain model resolution (cell size), and
accuracy inherent in current cartographic literature; (c)
improve reproducibility of terrain rendering results; and
(d) facilitate the development of new terrain rendering
techniques by providing models that are easily accessi-
ble, even as newer data becomes available.
Our research was motived by the potential benefits it
could provide to at least three different groups of users. The
first group of users are software developers, who need to
verify the correctness of a terrain rendering algorithm they
are implementing. They code a published algorithm for
their software and visually verify their implementation by
comparing the results to a figure published by the authors
of the algorithm. The second group of users are researchers
developing new terrain visualizations. They can easily com-
pare a new visualization method with methods published
in the literature. If they use the same elevation models as the
authors of previous research, they are not required to go
through the error-prone process of re-coding existing
visualization algorithms and can instead refer to published
figures. Alternatively, they may re-code an algorithm to
reproduce and verify published results, which is simplified
if the author of the original algorithm included a figure
rendered with a well-known elevation model. The third
group of users are cartographers, who may require models
with a variety of landform types and map scales when
selecting software and rendering parameters for creating
multiscale web maps.
Producing the proposed elevation models consisted
of two parts. First, we identified what the authors of new
terrain rendering algorithms and methods used when
publishing their results. To understand the utility and
requirements for elevation models, we identified map
scales, cell sizes, and landform types commonly used to
illustrate new techniques. We inventoried 78 articles
that included 155 terrain renderings. Besides map
scale, cell size, and landform types, we also analyzed
how authors evaluated their terrain rendering techni-
ques and how they indicated, if at all, the information
necessary to reproduce the visualization.
The second part of the methodology was selecting and
compiling elevation models based on our findings from
part one. The idea for a collection of reusable elevation
models was introduced to the cartographic community
in July 2019 at the International Cartographic Conference
ICC in Tokyo (Kennelly et al., 2019b). Early results
were presented at the October 2019 North American
Cartographic Information Society (NACIS) conference
in Tacoma, Washington (Kennelly et al., 2019a). The
selection of the elevation models introduced in this arti-
cle was influenced in part by feedback received at these
conferences.
2. An inventory of elevation models used in
publications on terrain visualization
2.1. Goals and scope of inventory
To guide the selection of elevation models, we inventor-
ied publications from the cartographic literature that
introduced new digital terrain visualization techniques
or improved existing techniques. A small number of
studies evaluating or comparing multiple terrain visua-
lization techniques were also included. The goal of this
inventory was to identify the type of elevation models
used in earlier publications. We were interested in the
display scale of the sample renderings, as well as the cell
size, landform type, and geographic location of the
digital elevation models. We inventoried how authors
evaluated their renderings, how scale was indicated on
the renderings, and whether the cell size and source
of elevation data was included. We collected this
2P. J. KENNELLY ET AL.
information to identify any publication conventions
within the cartographic community, particularly in
regard to reproducibility of presented results.
The inventory sampled a diverse collection of ren-
derings including colored aspect, contour lines, hyp-
sometric tints, plan oblique relief, relief shading, rock
and scree representation, and spot heights. We
excluded publications discussing rarely used terrain
visualization techniques (such as hachures or illumi-
nated and shaded contours), physical terrain models,
non-orthographic terrain visualizations (such as per-
spective maps and panorama maps), animated fly-
throughs, maps in virtual or augmented reality, or
maps created with hologram technology or autoster-
eoscopic displays. We included works published in
journals and conference proceedings that are accessi-
ble through university libraries or online. A journal’s
citation index or the reputation of a conference did
not influence our selection.
To limit the publications to a number we could
reasonably handle, we refined the criteria further.
Although many pre-digital publications on terrain
visualization are still relevant, such as Imhof’s seminal
book on Cartographic Relief Presentation (Imhof,
1982) or Tanaka’s contouring method (Tanaka,
1950), we excluded publications from the pre-digital
era since these works did not use digital elevation
models. We only inventoried English-language publi-
cations. We restricted the survey to publications that
introduced new methods or techniques for terrain
mapping, and excluded reports that focused on apply-
ing existing methods to a particular geographic area,
tutorials, software manuals, or discussions of historic
aspects. We excluded publications about digital tech-
niques that were not derived from a digital elevation
model such as digital rock drawing tools (Hurni et al.,
2001) or digital-manual relief shading (Bell, 2018;
Tóth, 2011). Publications discussing the creation,
enhancement, simplification, or geomorphologic ana-
lysis of elevation models were also excluded because
the goal of these works was not to create cartographic
visualizations.
The inventory recorded 155 figures from 78 pub-
lications, of which 72 were peer-reviewed. We
included journal articles (55), conference proceedings
(15), webpages (3), book chapters (2), theses (2), and
one technical report. The earliest publication is from
1965 (Yoéli, 1965). In total, 17 publications were pub-
lished in 2000 or earlier, 25 between 2001 and 2010,
and 36 between 2011 and 2019. To avoid potential
bias, each author contributed to the encoding process
of the publication figures. We will refer to the pub-
lication figures as “renderings” to avoid confusing
them with the figures presented in this paper.
Table 1 lists the publications by visualization type.
The relatively large number of publications on relief
shading is due to two reasons. First, the cartographic
research community seems to publish more academic
articles on relief shading than other terrain visualiza-
tion methods and second, shaded relief is the primary
research interest of the authors of this study. The goal
of the inventory was not to quantify research efforts of
academic cartographers, but to identify the type of
elevation models used and understand current prac-
tices in regard to reproducibility.
2.2. Inventory results
The inventory data is publicly available (Kennelly
et al., 2020b). Figure 1 shows the geographic locations
of the elevation models in the 155 inventoried render-
ings. There are clusters in the Rocky Mountains, the
Swiss Alps, and several other mountainous places
across the world.
Table 1. Inventoried publications by visualization type. Numbers in parentheses indicate the publication count for each
visualization type.
Visualization Type Publications
Colored aspect (1) Moellering & Kimerling, 1990
Contour lines (6) Jaara & Lecordix, 2011; Kettunen et al., 2017; Mackaness & Steven, 2006; Oksanen et al., 2017; Samsonov et al., 2019; Touya
et al., 2019
Hypsometric tinting (4) Eyton, 1990; Huffman & Patterson, 2013; Patterson & Jenny, 2011; Samsonov, 2011
Plan oblique relief (5) Jenny, 2006; Jenny et al., 2015; Jenny & Patterson, 2007; Kettunen et al., 2009; Willett et al., 2015
Relief shading (37) Batson et al., 1975; Brassel, 1974; Brown, 2014; Chiba et al., 2008; Ding & Densham, 1994; Edwards & Davis, 1994; Horn, 1981;
Jenny, 2001; Jenny, 2006; Katzil & Doytsher, 2003; Kennelly, 2008; Kennelly & Kimerling, 2004; Kennelly & Stewart, 2006,
Kennelly & Stewart, 2014; Leonowicz et al., 2010a, Leonowicz et al., 2010b; Leonowicz & Jenny, 2010; Loisios et al., 2007;
Lukas & Weibel, 1995; Mark, 1992; Marston & Jenny, 2015; Nighbert, 2000; Patterson, 1997, Patterson, 2001, Patterson,
2002, Patterson, 2004, Patterson, 2014, Patterson, 2016; Pingel & Clarke, 2014; Podobnikar, 2012b; Tait, 2002; Veronesi &
Hurni, 2014, Veronesi & Hurni, 2015; White, 1985; Yoéli, 1965; Yoëli, 1967; Yokoyama et al., 2002
Rock and scree (12) Christophe et al., 2016; Dahinden & Hurni, 2007; Geisthövel, 2017; Geisthövel & Hurni, 2015, Geisthövel & Hurni, 2018; Gondol
et al., 2008; Hurtut & Lecordix, 2011; Jenny et al., 2010; Jenny & Hutzler, 2008; Loi, 2015; Lysák, 2016; Yang et al., 2015
Spot heights (5) Baella et al., 2007; Jaara & Lecordix, 2011; Palomar-Vázquez & Pardo-Pascual, 2007; Podobnikar, 2012a; Rocca et al., 2017
User studies (8) Biland & Çöltekin, 2017; Buddeberg et al., 2017; Carbonell-Carrera & Hess-Medler, 2019; Murakoshi & Higashi, 2015; Patterson
& Jenny, 2013; Patton & Crawford, 1977; Phillips, 1982; Wilson & Worth, 1985
CARTOGRAPHY AND GEOGRAPHIC INFORMATION SCIENCE 3
2.2.1. Map scale
We recorded the scale of the renderings using scale
indications from the publication when available. If the
scale was not indicated, it was computed from distances
measured on the rendering. Scale was not determined
for maps that were intended to be viewed at multiple
scales. If the rendering was clearly designed to be dis-
played at a different scale than the scale used in the
publication, then the intended scale was recorded.
Figure 2 shows that most renderings were at medium
or large scales; the most commonly used scales were
around 1:100,000. Relatively few renderings were at
a scale of 1:1 million or smaller.
2.2.2. Grid cell size
For 98 renderings, the elevation model’s grid cell size was
either indicated in the text or communicated by the
authors of the publications. A small number of render-
ings were not considered because they were computed
from triangulated irregular networks. Figure 3 shows the
distribution of the renderings’ cell sizes with respect to
map scale. There are clusters for spatial resolutions of 10,
25, and 30 meters, which are the most common cell sizes
of elevation models. As researchers use increasingly
higher resolution elevation data (with cell sizes between
1 and 10 meters) at medium and even small scales, the
scalability of the proposed elevation models will be an
important consideration.
2.2.3. Landform types
From a subset of the renderings, we identified the five
most common landform types: (1) plain or glacial field;
(2) low to medium slope hills; (3) steep slope (alpine)
mountain range; (4) volcano, caldera, or volcanic field;
and (5) tableland or mesas. Using these categories, we
encoded what landforms each inventoried rendering
showed. Of the 155 renderings, 105 showed a mountain
range with steep slopes, 69 showed hills with low to
medium slope, and 50 showed plains, plains with small
hills, or flat glacial fields (Figure 4). The prevalence of
alpine landscape is not surprising since this terrain is
particularly challenging and dramatic to map. Hills and
flat lowlands were also common. We were surprised by
the popularity of cone-shaped calderas and volcanic fields
shown in the renderings (17). Tableland or mesas with
flat mountain tops were shown in 14 renderings.
2.2.4. Indication of data source and cell size
Of the 79 publications, 31 did not indicate the source
of the elevation data used for the renderings. Four
publications indicated only the source for some
Figure 1. Location of elevation models in the inventoried sample
renderings.
million
million
million
million
million
Figure 2. Map scale for 142 renderings. Thirteen renderings
were excluded because their scale was either indeterminable
or they were meant to be viewed at multiple scales. Figure 3. Distribution of map scale and elevation model cell size.
4P. J. KENNELLY ET AL.
renderings. Only 31 publications explicitly stated the
grid cell size. For eight publications, the cell size was
derived from the stated data source. To create Figure
3, we received the cell size from authors or we
extracted the cell size from other sources for 17
publications that lacked this information.
2.2.5. Evaluation method
Of the 79 publications, 58 evaluated the presented ren-
derings while 21 conducted no evaluation (Figure 5).
Twenty publications visually compared the presented
renderings to manually created renderings in adjacent
displays of the same elevation data. Eleven conducted
a user study to evaluate their results. Four visually
inspected the presented results, but did not compare
them to other renderings. The authors of one publica-
tion collected feedback from expert cartographers.
3. Methodology
3.1. Requirements
The inventory of publications introducing new or
improved terrain visualization techniques helped guide
the selection of the elevation models. The inventory
showed that authors create renderings across a range of
map scales, but rarely at scales smaller than 1:1 million.
There is also a need to adapt terrain visualization techni-
ques to multiscale web mapping applications, which is
addressed by a number of publications (Jenny et al., 2015;
Patterson, n.d.; Raposo, 2020; Samsonov, 2011). The
inventory revealed that grids with a cell size greater
than 100 meters are rarely used. In recent years, high-
resolution models with cell sizes between 1 and
10 meters – mainly created with light detection and
ranging [LiDAR] – are being used more frequently,
even at medium and small map scales. Therefore, the
proposed elevation models need to be multiscale and
include high-quality data with small cell sizes. From the
inventory, it became clear that the elevation models
should include diverse landform types, but priority
should be given to models including steep mountain
ranges because these landforms are commonly used.
The inventory also clearly showed that renderings
often include archetypal landforms. Whereas alpine
mountain ranges are often the focus of relief representa-
tion studies, there is still a need for elevation models of
other terrain types. For example, the subtle topography of
a braided riverbed requires a completely different repre-
sentational approach than an alpine landscape.
Specialized visualization methods have been designed
for specific landforms types, such as colored aspect for
steep mountains (Moellering & Kimerling, 1990) or rela-
tive elevation models for historic riverbeds (Olson et al.,
2014). As a result, we decided to compile single-scale
elevation models of archetypal landforms as well. Here,
the focus is on landform types that differ from alpine
landscapes, which the multiscale models cover well.
Based on these criteria, we searched for suitable pub-
lic domain datasets that would be sharable with
a permissive license. Additionally, we required high
quality accurate data with minimal artifacts.
3.2. Data processing
Elevation data was processed in Natural Scene Designer
Pro, QGIS, and Adobe Photoshop and involved switch-
ing between NSD, IMG, and ASCII Grid formats. We
used the GeoCrop tool in Geographic Imager–a GIS
plugin for Photoshop–to crop each of the grids in their
respective multi-resolution series to the same center
point. Slight elevation variations, usually within a few
decimeters, exist between grids in each of the multi-
resolution series. We used bicubic interpolation for
downsampling and did not filter or generalize the eleva-
tion data, even at the smallest scales. The proposed
elevation models are mostly free of artifacts and seams,
but a few exist; we did not attempt to fix these
imperfections.
4. Elevation models for reproducible
evaluation
We assembled elevation data from a variety of sources.
For the United States, we used the 3D Elevation Program
Plain
Hills
l l l
ll
l
l l l
Figure 4. Landform types shown in the inventoried renderings.
l
l
l
l
Figure 5. Evaluation method used in inventoried publications.
CARTOGRAPHY AND GEOGRAPHIC INFORMATION SCIENCE 5
(3DEP), formerly known as National Elevation Dataset
(NED), by the U.S. Geological Survey (USGS) (USGS,
n.d.). The 3DEP combines data created with a variety of
techniques such as LiDAR and interferometric synthetic
aperture radar (IFSAR) (Gesch, 2007). All the datasets we
used are in the public domain. The three multiscale
elevation models and the collection of single-scale arche-
typal models that we propose are described in detail
below. Table 2 indicates the center point, coordinate
reference system, and dimensions of the three multiscale
models, as well as the cell size and the data source for each
scale. Table 3 lists the corresponding information for the
single-scale models of archetypal landforms: the location,
UTM projection zone (all single-scale models use the
NAD83 datum), model dimensions, and cell size.
4.1. Gore Range, Colorado, USA multiscale model
The first multiscale elevation model centers on Gore
Range, Colorado, a chain of fault-block mountains that
trend 100 kilometers northwest to southeast (Figure 6).
Rocky spires, talus slopes, and glacier-carved lake basins
characterize central parts of the range. Mount Powell is
the highest peak at 4,135 meters.
Grid cells in the multiscale model range from 1 meter
to 5 kilometers. Each of the grids measure 1,500 × 1,500
height samples, share the same center point (9.7378 N,
106.3282 W), and are in the Albers conic equal-area
projection. The three most detailed grids (1, 5, and 15-
meter cell size) are derived from 3DEP LiDAR data at
1-meter resolution. The 30-meter resolution grid
derives primarily from 3DEP LiDAR data at 1-meter
resolution with 3.3-meter 3DEP LiDAR data in
peripheral areas. The grids at 90, 250, 500, and 1,000-
meter resolutions derive from the above-mentioned
LiDAR and other 3DEP sources. The 2,000, 2,500, and
5,000-meter grids originated with 3 arc-second data
downloaded from viewfinderpanoramas.org, which
combines USGS NED, Canadian government, and
SRTM sources.
Our motivation for selecting this multiscale elevation
model was that the fractured rocks of the Gore Range
coupled with the detailed LiDAR data from which the
elevation model derives presents challenges on how best
to visualize this complex and intricate topography.
4.2. Valdez, Alaska, USA multiscale model
The second multiscale elevation model centers on
Valdez, a small port on Prince William Sound, Alaska
(Figure 7). The intricate fjords and islands of Prince
William Sound open to the Pacific Ocean in the south.
Numerous tidewater glaciers descend from the Chugach
Mountains forming an arc to the north.
The elevation model has nine grids with cell sizes
ranging from 3.3 meters to 2 kilometers. All grids mea-
sure 1,500 × 1,500 height samples, share the same center
point (61.1428 N, 146.3717 W), and are in the Albers
conic equal-area projection. The three most detailed
grids are derived from 3DEP LiDAR data starting at
3.3-meter resolution. The 7.5 and 15-meter grids are
downsampled with bicubic interpolation in a single
step, as is the case with subsequent elevation models.
The next three coarser resolutions are 30, 90, and
250 meters, and these are all derived from 3DEP
IFSAR data originally distributed at a resolution of
Table 2. Multiscale models: geographic location, coordinate reference system, dimensions, cell size, and data source for each scale.
Gore Range, Colorado, USA Valdez, Alaska, USA Churfirsten, Switzerland
Center point 39.7378 N/106.3282 W 61.1428 N/146.3717 W 47.1436 N/9.3486 E
Projection and datum Albers conic, NAD83
Standard parallels: 29.5 N, 45.5 N
Latitude of origin: 23 N
Central meridian: 106 W
Albers conic, NAD83
Standard parallels: 55 N, 65 N
Latitude of origin: 50 N
Central meridian: 146.5 W
Bonne, WGS84
Latitude of origin: 47.25 N
Central meridian: 9.4 E
Dimensions 1,500 × 1,500 1,500 × 1,500 1,500 × 1,500
Cell size Data source with cell size (in meter [m] or arc-second [“])
1 m 3DEP LiDAR 1 m – –
3.3 m – 3DEP LiDAR 3.3 m –
5 m 3DEP LiDAR 1 m 3DEP LiDAR 3.3 m –
15 m 3DEP LiDAR 1 m 3DEP LiDAR 3.3 m –
30 m 3DEP LiDAR 1 m (3.3 m at periphery) 3DEP IFSAR 5 m ViewFinderPanoramas 1”
60 m – – ViewFinderPanoramas 1”
90 m Other 3DEP sources 3DEP IFSAR 5 m –
120 m – – ViewFinderPanoramas 1”
250 m Other 3DEP sources 3DEP IFSAR 5 m ViewFinderPanoramas 1”
500 m Other 3DEP sources 3DEP IFSAR 5 m ViewFinderPanoramas 3”
1,000 m Other 3DEP sources ViewFinderPanoramas 3” ViewFinderPanoramas 3”
2,000 m ViewFinderPanoramas 3” ViewFinderPanoramas 3” ViewFinderPanoramas 3”
2,500 m ViewFinderPanoramas 3” – –
5,000 m ViewFinderPanoramas 3” – –
6P. J. KENNELLY ET AL.
5 meters. The three grids covering the greatest extent are
comprised of 500, 1,000 and 2,000-meter grid cells.
These grids are downsampled from 2 arc-second 3DEP
data in the United States and less detailed elevation data
in Canada, downloaded from viewfinderpanoramas.org.
Valdez on Prince William Sound, where mountains,
glaciers, and the sea meet, is a sub-polar landscape that
is severely affected by global warming, including de-
glaciation and sea level rise. Another motivation for
selecting the Valdez area is that it includes flat river
plains and water surfaces with steep adjacent mountain
slopes. The resulting sharp transitions between flat and
steep areas are often blurred by filter operations that
generalize terrain models. The models with small and
medium cell sizes included in the Valdez multiscale
model can be used effectively to evaluate whether terrain
generalization methods preserve sharp transitions (for
an example of such an application, see Jenny, 2020).
4.3. Churrsten, Switzerland multiscale model
The third multiscale elevation model is centered on the
Churfirsten, a crescent-shaped mountain chain in
Switzerland rising out of Walensee, a fjord-like lake
(Figure 8). Seven “sawtooth” peaks form the crest with
a maximum elevation of 2,306 meters. The multiscale
model includes an alpine landscape with characteristic
pointed peaks, narrow valleys, and steep slopes, but also
rolling hills that transition into plains with large lakes.
The elevation model has seven resolutions with grid
cells ranging from 30 meters to 2 kilometers. All grids
measure 2,500 × 1,500 height samples, share the same
center point (47.1436 N, 9.3486E), and are in the Bonne
equal-area projection. They are derived from data
downloaded from viewfinderpanoramas.org. The 30,
60, 120, and 250-meter grids derive from 1 arc-second
data and the 500, 1,000, and 2,000-meter grids are
derived from 3 arc-second data.
The 30-meter model, which shows the crescent-
shaped Churfirsten mountain chain at the center, is
well suited for experimenting with multi-directional
lighting for relief shading because of the arcing shape
of its ridgeline. For this reason, the Churfirsten
mountains are the subject of several illustrations
demonstrating light adjustment for relief shading in
Eduard Imhof’s book Cartographic Relief Presentation
(Imhof, 1982). The models with a cell size of 250 m and
larger are challenging to visualize, because the Alps
appear as an intricate and unstructured amalgamation
of peaks that dwarf the surrounding topography.
4.4. Single-scale models of archetypal landforms
We are proposing elevation models of archetypal land-
form types derived from 3DEP LiDAR sources with cell
sizes ranging from 2 to 10 meters. The five archetypal
landform models now available include a volcanic cal-
dera (Crater Lake, Oregon, USA, Figure 9), active sand
dunes (Great Sand Dunes, Colorado, USA, Figure 10),
a braided riverbed (Jackson Hole, Wyoming, USA,
Figure 11), folded ridges (Massanutten Mountain,
Virginia, USA, Figure 12), and stabilized sand dunes
(Sandhills, Nebraska, USA, Figure 13). Other models
will be added in the future as new LiDAR data becomes
available.
5. Conclusion
This article introduces three multiscale elevation
models and a collection of single-scale elevation
models of archetypal landforms to simplify the eva-
luation and comparison of various terrain render-
ing techniques. The models can be accessed via
http://shadedrelief.com/SampleElevationModels/. All
models are available under the Creative Commons
CC BY Attribution license.
The three multiscale elevation models show mainly
alpine terrain and allow mapmakers to create and
evaluate terrain renderings at a broad range of map-
ping scales, from a small, isolated landform to large
areas. The collection of archetypal landform elevation
models allows a user to evaluate the suitability of
rendering techniques with specific landforms. The col-
lection currently contains models of folded ridges,
a braided riverbed, active and stabilized sand dunes,
and a volcanic caldera.
Table 3. Single-scale models of archetypal landforms: location, UTM projection zone, dimensions, and cell size. All single-scale models
use the NAD83 datum and are derived from 3DEP LiDAR data with original resolutions between 2 and 10 meters.
Landform Location UTM zone Dimensions Cell size
Volcanic caldera Crater Lake, Oregon, USA 10 N 5,200 × 5,200 3.33 m
Active sand dunes Great Sand Dunes, Colorado, USA 13 N 5,200 × 5,200 3.33 m
Braided riverbed Jackson Hole, Wyoming, USA 12 N 4,200 × 4,200 2 m
Folded ridges Massanutten Mountain, Virginia, USA 17 N 3,900 × 3,900 10 m
Stabilized sand dunes Sandhills, Nebraska, USA 14 N 4,500 × 4,500 10 m
CARTOGRAPHY AND GEOGRAPHIC INFORMATION SCIENCE 7
Figure 6. Gore Range, Colorado, USA multiscale model: shaded relief renderings of all 11 grids with cell sizes between 1 and
5,000 meters.
8P. J. KENNELLY ET AL.
In the future, we plan to expand the collection of
elevation models for archetypal landforms as suitable
data becomes available. The best results are achieved
with LiDAR elevation data containing very few artifacts
and minimal evidence of human activity, such as roads
and building footprints. Potential data sources are the
USGS 3DEP and the Land Information New Zealand
(LINZ) LiDAR initiatives. Canyons, drumlins, hills,
karst, tablelands, and stratovolcanoes are some of the
landform types we hope to include. We anticipate hav-
ing elevation models for the majority of archetypal land-
forms in the next few years.
Based on our inventory of published renderings,
many authors do not document the source or cell size
of elevation data, or the scale of example visualizations.
Omitting this information makes it considerably more
difficult for others to verify implementation, replicate
the published work, or compare new or improved algo-
rithms with existing methods. Using a proposed eleva-
tion model can reduce ambiguity if authors neglect to
include this information.
We also observed and experienced through our
own research in terrain visualization that a number
of previous publications are imprecise or lack
Figure 7. Valdez, Alaska, USA multiscale model: shaded relief renderings of all nine grids with cell sizes between 3.3 and 2,000 meters.
CARTOGRAPHY AND GEOGRAPHIC INFORMATION SCIENCE 9
algorithmic details, parameters to create sample
renderings, or steps for pre-processing input data
or post-processing resulting renderings. The small
cartographic research community should aim to
Figure 8. Churfirsten, Switzerland multiscale model: shaded relief renderings of all seven grids with cell sizes between 30 and
2,000 meters.
10 P. J. KENNELLY ET AL.
raise documentation standards to improve the
reproducibility of publications.
The proposed terrain models have already been used
for illustrating recently developed rendering techniques
(Jenny, 2020; Jenny et al., 2020; Jenny & Patterson, 2020)
and we hope that developers of new algorithms will find
them useful for comparing their results to existing meth-
ods. This work may inspire researchers in other areas of
cartography and GIScience to develop similar collections
of data models suitable for their domain.
Data Availability Statement
We recommend accessing the proposed elevation models
via http://shadedrelief.com/SampleElevationModels/. The three
multiscale elevation models are openly available in the Zenodo
repository at Kennelly et al. (2020c), “Elevation Models for
Reproducible Evaluation of Terrain Representation – Multiscale
Models” (https://doi.10.5281/zenodo.3938012). The elevation
models of archetypal landforms are openly available in the
Zenodo repository at Kennelly et al. (2020b), “Elevation Models
for Reproducible Evaluation of Terrain Representation –
Archetypal Landforms” (https://doi.10.5281/zenodo.3938020).
All elevation models are in georeferenced GeoTIFF and Esri
ASCII grid file formats, and are available under the Creative
Commons CC BY Attribution license. The elevation models
were derived from different resources available in the public
domain; see Section 4 for details. The inventory of 155 figures
from 78 publications on terrain visualization is openly available
in the Zenodo repository at Kennelly et al. (2020a), “Elevation
Models for Reproducible Evaluation of Terrain Representation –
Inventory of Renderings” (https://doi.10.5281/zenodo.3937894).
Acknowledgments
The authors thank Jonathan de Ferranti for the permission to
use the elevation models at viewfinderpanoramas.org under
the Creative Commons Attribution 4.0 International. The
authors also thank the anonymous reviewers for their valuable
comments.
Figure 9. Volcanic caldera (Crater Lake, Oregon, USA): 5,200 × 5,200
height samples with a cell size of 3.33 meters.
Figure 10. Active sand dunes (Great Sand Dunes, Colorado,
USA): 5,200 × 5,200 height samples with a cell size of
3.33 meters.
Figure 11. Braided riverbed (Jackson Hole, Wyoming, USA):
4,200 × 4,200 height samples with a cell size of 2 meters.
Figure 12. Folded ridges (Massanutten Mountain, Virginia, USA):
3,900 × 3,900 height samples with a cell size of 10 meters.
Figure 13. Stabilized sand dunes (Sandhills, Nebraska, USA):
4,500 × 4,500 height samples with a cell size of 10 meters.
CARTOGRAPHY AND GEOGRAPHIC INFORMATION SCIENCE 11
Disclosure statement
No potential conflict of interest was reported by the authors.
ORCID
Patrick J. Kennelly http://orcid.org/0000-0003-1687-7547
Tom Patterson http://orcid.org/0000-0003-4813-895X
Bernhard Jenny http://orcid.org/0000-0001-6101-6100
Daniel P. Huffman http://orcid.org/0000-0001-6467-2672
Brooke E. Marston http://orcid.org/0000-0001-8848-5513
Sarah Bell http://orcid.org/0000-0001-5300-9701
Alexander M. Tait http://orcid.org/0000-0003-2745-9164
References
Baella, B., Palomar-Vázquez, J., Pardo-Pascual, J. E., & Pla, M.
(2007). Spot heights generalization: Deriving the relief of
the topographic database of Catalonia at 1:25,000 from the
master database. Proceedings of the 7th ICA Workshop on
Progress in Automated Map Generalization. Paris, France.
Batson, R. M., Edwards, K., & Eliason, E. M. (1975).
Computer-generated shaded-relief maps. Journal of
Research of the U. S. Geological Survey, 3(4), 401–408.
Bell, S. (2018). Drawing hillshade: A tutorial (with time lapse
videos). Petrichor Studio. http://bit.ly/drawinghillshade
Biland, J., & Çöltekin, A. (2017). An empirical assessment of
the impact of the light direction on the relief inversion
effect in shaded relief maps: NNW is better than NW.
Cartography and Geographic Information Science, 44(4),
358–372. https://doi.org/10.1080/17538947.2018.1447030
Brassel, K. (1974). A model for automatic hill-shading. The
American Cartographer, 1(1), 15–27. https://doi.org/10.
1559/152304074784107818
Brown, L. (2014). Texture shading: A new technique for
depicting terrain relief. Proceedings of the 9th ICA
Mountain Cartography Workshop, 1–14. Banff, Canada.
Brunsdon, C., & Comber, A. (2020). Opening practice:
Supporting reproducibility and critical spatial data science.
Journal of Geographical Systems. Advance online publication.
https://doi.org/10.1007/s10109-020-00334-2
Buddeberg, J., Jenny, B., & Willett, W. (2017). Interactive
shearing for terrain visualization: an expert study.
GeoInformatica, 21(3), 643–665. https://doi.org/10.1007/
s10707-016-0283-9
Carbonell-Carrera, C., & Hess-Medler, S. (2019). Interactive
visualization software to improve relief interpretation skills:
Spatial data infrastructure geoportal versus augmented rea-
lity. The Professional Geographer, 71(4), 725–737. https://
doi.org/10.1080/00330124.2019.1595064
Cerutti, V., Bellman, C., Both, A., Duckham, M., Jenny, B.,
Lemmens, R. L. G., & Ostermann, F. O. (2019). Improving
the reproducibility of geospatial scientific workflows: The
use of geosocial media in facilitating disaster response.
Journal of Spatial Science. Advance online publication.
https://doi.org/10.1080/14498596.2019.1654944
Chiba, T., Kaneta, S., & Suzuki, Y. (2008). Red relief image
map: New visualization method for three dimensional
data. The International Archives of the Photogrammetry,
Remote Sensing and Spatial Information Sciences, 37(B2),
1071–1076.
Christophe, S., Duménieu, B., Turbet, J., Hoarau, C., Mellado, N.,
Ory, J., Loi, H., Masse, A., Arbelot, B., Vergne, R., Brédif, M.,
Hurtut, T., Thollot, J., & Vanderhaeghe, D. (2016). Map style
formalization: Rendering techniques extension for cartogra-
phy. In P. Bénard & H. Winnemöller (Eds.), Non-photorealistic
animation and rendering. The Eurographics Association.
https://doi.org/10.2312/exp.20161064
Dahinden, T., & Hurni, L. (2007). Development and qual-
ity assessment of analytical rock drawings. Proceedings
of the 23rd International Cartographic Conference (ICC).
Moscow, Russia.
Ding, Y., & Densham, P. J. (1994). A loosely synchronous, parallel
algorithm for hill shading digital elevation models.
Cartography and Geographic Information Systems, 21(1),
5–14. https://doi.org/10.1559/152304094782563957
Edwards, K., & Davis, P. A. (1994). The use of intensity-hue-
saturation transformation for producing color shaded-relief
images. Photogrammetric Engineering & Remote Sensing, 60
(11), 1369–1374.
Eyton, J. R. (1990). Color stereoscopic effect cartography.
Cartographica: The International Journal for Geographic
Information and Geovisualization, 27(1), 20–29. https://
doi.org/10.3138/K213-2288-7672-U72T
Gahegan, M. (2019). Reproducible geocomputation: An open
or shut case? Proceedings of GeoComputation, Auckland,
New Zealand. https://doi.org/10.17608/k6.auckland.
9870089.v2
Geisthövel, R. (2017). Automatic Swiss style rock depiction
[Doctoral dissertation, ETH Zürich]. ETZ Zürich Research
Collection. https://doi.org/10.3929/ethz-b-000201368
Geisthövel, R., & Hurni, L. (2015). Automatic rock depiction
via relief shading. Proceedings of the 27th International
Cartographic Conference, Washington, DC.
Geisthövel, R., & Hurni, L. (2018). Automated Swiss-style
relief shading and rock hachuring. The Cartographic
Journal, 55(4), 341–361. https://doi.org/10.1080/00087041.
2018.1551955
Gesch, D. B. (2007). The national elevation dataset. In
D. Maune (Ed.), Digital elevation model technologies and
applications—the DEM users manual (2nd ed., pp. 99–118).
American Society for Photogrammetry and Remote
Sensing.
Gil, Y., David, C. H., Demir, I., Essawy, B. T., Fulweiler, R. W.,
Goodall, J. L., Karlstrom, L., Lee, H., Mills, H. J., Oh, J.,
Pierce, S. A., Pope, A., Tzeng, M. W., Villamizar, S. R., &
Yu, X. (2016). Toward the geoscience paper of the future:
Best practices for documenting and sharing research from
data to software to provenance. Earth and Space Science, 3
(10), 388–415. https://doi.org/10.1002/2015EA000136
Giraud, T., & Lambert, N. (2017). Reproducible cartography.
M. Peterson (Ed.), Advances in Cartography and GIScience,
Selections from the International Cartographic Conference
2017 (pp. 173–183). Washington, D.C., USA, Springer.
https://doi.org/10.1007/978-3-319-57336-6_13
Gondol, L., Le, B. A., & Lecordix, F. (2008). A new approach
for mountain areas cartography. In A. Ruas & C. Gold
(Eds.), Headway in spatial data handling: Lecture notes in
geoinformation and cartography (pp. 315–333). Springer.
https://doi.org/10.1007/978-3-540-68566-1_18
12 P. J. KENNELLY ET AL.
Horn, B. K. P. (1981). Hill shading and the reflectance map.
Proceedings of the Institute of Electrical and Electronics
Engineers IEEE, 69(1), 14–47. https://doi.org/10.1109/
PROC.1981.11918
Huffman, D. P. (2017). Creating shaded relief in blender.
somethingaboutmaps. https://somethingaboutmaps.word
press.com/2017/11/16/creating-shaded-relief-in-blender/
Huffman, D. P., & Patterson, T. (2013). The design of gray earth:
A monochrome terrain dataset of the world. Cartographic
Perspectives, 74, 61–70. https://doi.org/10.14714/CP74.580
Hurni, L., Dahinden, T., & Hutzler, E. (2001). Digital cliff drawing
for topographic maps: Traditional representations by means of
new technologies. Cartographica: The International Journal for
Geographic Information and Geovisualization, 38(1–2), 55–65.
https://doi.org/10.3138/6R12-2GV6-X501-2306
Hurtut, T., & Lecordix, F. (2011). Cartography of mountain
rocky areas, a statistical modeling and drawing of element
arrangements. Proceedings of the 25th International
Cartographic Conference (ICC), Paris, France.
Imhof, E. (1982). Cartographic relief presentation. De Gruyter.
Isenberg, T., Isenberg, P., Chen, J., Sedlmair, M., & Moller, T.
(2013). A systematic review on the practice of evaluating
visualization. IEEE Transactions on Visualization and
Computer Graphics, 19(12), 2818–2827. https://doi.org/10.
1109/TVCG.2013.126
Jaara, K., & Lecordix, F. (2011). Extraction of cartographic
contour lines using digital terrain model (DTM). The
Cartographic Journal, 48(2), 131–137. https://doi.org/10.
1179/1743277411Y.0000000011
Jenny, B. (2001). An interactive approach to analytical relief
shading. Cartographica: The International Journal for
Geographic Information and Geovisualization, 38(1&2),
67–75. https://doi.org/10.3138/F722-0825-3142-HW05
Jenny, B. (2020). Terrain generalization with line integral
convolution. Cartography and Geographic Information
Science. Advance online publication. https://doi.org/10.
1080/15230406.2020.1833762
Jenny, B., Buddeberg, J., Hoarau, C., & Liem, J. (2015). Plan
oblique relief for web maps. Cartography and Geographic
Information Science, 42(5), 410–418. https://doi.org/10.
1080/15230406.2015.1015169
Jenny, B., Heitzler, M., Singh, D., Farmakis-Serebryakova, M.,
Liu, J. C., & Hurni, L. (2020). Cartographic relief shading
with neural networks. IEEE Transactions on Visualization
and Computer Graphics. https://doi.org/10.1109/TVCG.
2020.3030456
Jenny, B. (2006). Design of a panorama map with plan oblique
and sphericalprojection. Proceedings of the 5th ICA
Mountain Cartography Workshop, Bohinj, Slovenia (pp.
121–128).doi.org/10.1179/000870406X158164
Jenny, B., & Hutzler, E. (2008). Automatic scree representation
for topographic maps. Proceedings of the 6th ICA Mountain
Cartography Workshop (pp. 99–103). Lenk, Switzerland.
Jenny, B., Hutzler, E., & Hurni, L. (2010). Scree representation on
topographic maps. The Cartographic Journal, 47(2), 141–149.
https://doi.org/10.1179/000870409X12525737905006
Jenny, B., & Patterson, T. (2007). Introducing plan oblique
relief. Cartographic Perspectives, 57, 21–40. https://doi.org/
10.14714/CP57.279
Jenny, B., & Patterson, T. (2020). Aerial perspective for shaded
relief. Cartography and Geographic Information Science.
https://doi.org/10.1080/15230406.2020.1813052
Katzil, Y., & Doytsher, Y. (2003). A logarithmic and sub-pixel
approach to shaded relief representation. Computers &
Geosciences, 29(9), 1137–1142. https://doi.org/10.1016/
S0098-3004(03)00135-3
Kedron, P., Frazier, A. E., Trgovac, A. B., Nelson, T., &
Fotheringham, A. S. (2019). Reproducibility and replicabil-
ity in geographical analysis. Geographical Analysis, 1–13.
https://doi.org/10.1111/gean.12221
Kedron, P., Li, W., Fotheringham, S., & Goodchild, M. (2020).
Reproducibility and replicability: Opportunities and chal-
lenges for geospatial research. International Journal of
Geographical Information Science, 1–19. https://doi.org/10.
1080/13658816.2020.1802032
Kennelly, P. J. (2008). Geomorphology terrain maps display-
ing hill-shading with curvature. Geomorphology, 102(3–4),
567–577. https://doi.org/10.1016/j.geomorph.2008.05.046
Kennelly, P. J., & Kimerling, A. J. (2004). Hillshading of
terrain using layer tints with aspect-variant luminosity.
Cartography and Geographic Information Science, 31(2),
67–77. https://doi.org/10.1559/1523040041649416
Kennelly, P. J., Patterson, T., Jenny, B., Huffman, D. P.,
Marston, B. E., Bell, S., & Tait, A. M. (2020a). Elevation
models for reproducible evaluation of terrain representa-
tion – archetypal landforms [Data set]. Zenodo. https://
doi.10.5281/zenodo.3938020
Kennelly, P. J., Patterson, T., Jenny, B., Huffman, D. P.,
Marston, B. E., Bell, S., & Tait, A. M. (2020b). Elevation
models for reproducible evaluation of terrain representa-
tion – inventory of renderings [Data set]. Zenodo. https://
doi.10.5281/zenodo.3937894
Kennelly, P. J., Patterson, T., Jenny, B., Huffman, D. P.,
Marston, B. E., Bell, S., & Tait, A. M. (2020c). Elevation
models for reproducible evaluation of terrain representa-
tion – multiscale models [Data set]. Zenodo. https://doi.10.
5281/zenodo.3938012
Kennelly, P. J., Patterson, T., Tait, A., Jenny, B., Huffman, D.,
Bell, S., & Marston, B. (2019a). Standard elevation models
for evaluating terrain representations. NACIS. https://
youtu.be/Cl7lmEYX_qQ
Kennelly, P. J., Patterson, T., Tait, A., Jenny, B., Huffman, D.,
Bell, S., & Marston, B. (2019b). Standard elevation models
for evaluating terrain representation. Abstracts of the
International Cartographic Association, 1. https://doi.org/
10.5194/ica-abs-1-168-2019
Kennelly, P. J., & Stewart, A. J. (2006). A uniform sky illumi-
nation model to enhance shading of terrain and urban
areas. Cartography and Geographic Information Science,
33(1), 21–36. https://doi.org/10.1559/152304006777323118
Kennelly, P. J., & Stewart, A. J. (2014). General sky models for
illuminating terrains. International Journal of Geographical
Information Science, 28(2), 383–406. https://doi.org/10.
1080/13658816.2013.848985
Kettunen, P., Koski, C., & Oksanen, J. (2017). A design of
contour generation for topographic maps with adaptive
DEM smoothing. International Journal of Cartography, 3
(1), 19–30. https://doi.org/10.1080/23729333.2017.1300998
Kettunen, P., Sarjakoski, T., Sarjakoski, L. T., & Oksanen, J.
(2009). Cartographic portrayal of terrain in oblique parallel
projection. Proceedings of the 24th International Cartographic
Conference, 15–21, Santiago, Chile.
Konkol, M., & Kray, C. (2019). In-depth examination of
spatiotemporal figures in open reproducible research.
CARTOGRAPHY AND GEOGRAPHIC INFORMATION SCIENCE 13
Cartography and Geographic Information Science, 46(5),
412–427. https://doi.org/10.1080/15230406.2018.1512421
Lehmann, A.-S. (2012). Taking the lid off the Utah teapot:
Towards a material analysis of computer graphics. Zeitschrift
Für Medien- Und Kulturforschung, 3(1), 169–184. https://doi.
org/10.28937/1000106362
Leonowicz, A. M., & Jenny, B. (2010). Automated small-scale
relief shading: A new method and software application.
Geographica Technica, [Special issue], 90–95. https://doi.
org/10.5167/uzh-45721
Leonowicz, A. M., Jenny, B., & Hurni, L. (2010a). Automated
reduction of visual complexity in small-scale relief shading.
Cartographica: The International Journal for Geographic
Information and Geovisualization, 45(1), 64–74. https://
doi.org/10.3138/carto.45.1.64
Leonowicz, A. M., Jenny, B., & Hurni, L. (2010b). Terrain
sculptor: Generalizing terrain models for relief shading.
Cartographic Perspectives, 67, 51–60. https://doi.org/10.
14714/CP67.114
Loi, H. (2015). Programmable synthesis of element textures
and application to cartography [Doctoral dissertation,
Université de Grenoble]. HAL. https://hal.archives-
ouvertes.fr/tel-01462816v1
Loisios, D., Tzelepis, N., & Nakos, B. (2007). A methodology
for creating analytical hill-shading by combining different
lighting directions. Proceedings of the 23rd International
Cartographic Conference (ICC). Moscow, Russia.
Lukas, K., & Weibel, R. (1995). Assessment and improvement
of methods for analytical hillshading. Proceedings of the 17th
International Cartographic Conference (ICC), 2231–2240.
Barcelona, Spain.
Lysák, J. (2016). An algorithm for automated digital rock
drawing in the style used in Czech topographic maps.
AUC Geographica, 51(1), 5–16. https://doi.org/10.14712/
23361980.2016.1
Mackaness, W., & Steven, M. (2006). An algorithm for loca-
lised contour removal over steep terrain. The Cartographic
Journal, 43(2), 144–156. https://doi.org/10.1179/
000870406X114630
Mark, R. (1992). Multidirectional, oblique-weighted, shaded-
relief image of the Island of Hawaii. No. 92-422. US Dept. of
the Interior, US Geological Survey.
Marston, B. E., & Jenny, B. (2015). Improving the representation
of major landforms in analytical relief shading. International
Journal of Geographical Information Science, 29(7),
1144–1165. https://doi.org/10.1080/13658816.2015.1009911
Moellering, H., & Kimerling, A. J. (1990). A new digital
slope-aspect display process. Cartography and Geographic
Information Systems, 17(2), 151–159. https://doi.org/10.
1559/152304090783813853
Murakoshi, S., & Higashi, H. (2015). Cognitive characteristics
of navigational map use by mountaineers. International
Journal of Cartography, 1(2), 210–231. https://doi.org/10.
1080/23729333.2016.1158490
Nighbert, J. S. (2000). Using remote sensing imagery to tex-
turize layer tinted relief. Cartographic Perspectives, 36,
94–98. https://doi.org/10.14714/CP36.827
Nüst, D., Granell, C., Hofer, B., Konkol, M.,
Ostermann, F. O., Sileryte, R., & Cerutti, V. (2018).
Reproducible research and GIScience: An evaluation
using AGILE conference papers. PeerJ, 6(e5072), 1–23.
https://doi.org/10.7717/peerj.5072
Oksanen, J., Koski, C., Kettunen, P., Sarjakoski, T. (2017).
Automatic generation of cartographic features for relief pre-
sentation based on LIDAR DEMs. FIG Working Week.
Surveying the World of Tomorrow - From Digitalisation to
Augmented Reality.
Olson, P. L., Legg, N. T., Abbe, T. B., Reinhart, M. A., &
Radloff, J. K. (2014). A methodology for delineating
planning-level channel migration zones (Appendix E. Methods
for generating relative elevation models).
Palomar-Vázquez, J., & Pardo-Pascual, J. (2007). Automated
spot heights generalisation in trail maps. International
Journal of Geographical Information Science, 22(1),
91–110. https://doi.org/10.1080/13658810701349003
Patterson, T. (1997). A desktop approach to shaded relief
production. Cartographic Perspectives, 28, 38–40. https://
doi.org/10.14714/CP28.694
Patterson, T. (2001). See the light: How to make illuminated
shaded relief in Photoshop 6.0. http://www.shadedrelief.
com/illumination/
Patterson, T. (2002). Getting real: Reflecting on the new look
of National Park Service maps. Cartographic Perspectives,
43(43), 43–56. https://doi.org/10.14714/CP43.536
Patterson, T. (2004). Creating Swiss-style shaded relief in
Photoshop. http://www.shadedrelief.com/shading/Swiss.html
Patterson, T. (2014). Enhancing shaded relief with terrain
texture shader. Proceedings of the 9th ICA Mountain
Cartography Workshop. Banff, AB.
Patterson, T. (2016). Producing manual small-scale shaded
relief. Proceedings of the 10th ICA Mountain Cartography
Workshop, 1–11. Berchtesgaden, Germany.
Patterson, T. (n.d.). Creating web map shaded relief. http://
www.shadedrelief.com/web_relief/
Patterson, T., & Jenny, B. (2011). The development and rationale
of cross-blended hypsometric tints. Cartographic Perspectives,
69, 31–45. https://doi.org/10.14714/CP69.20
Patterson, T., & Jenny, B. (2013). Evaluating cross-blended
hypsometric tints: A user study in the United States,
Switzerland, and Germany. Cartographic Perspectives,
(75), 5–17. https://doi.org/10.14714/CP75.578
Patton, J. C., & Crawford, P. V. (1977). The perception of hypso-
metric colours. The Cartographic Journal, 14(2), 115–127.
https://doi.org/10.1179/caj.1977.14.2.115
Peng, R. D. (2011). Reproducible research in computational
science. Science, 334(6060), 1226–1227. https://doi.org/10.
1126/science.1213847
Phillips, R. J. (1982). An experimental investigation of
layer tints for relief maps in school atlases.
Ergonomics, 25(12), 1143–1154. https://doi.org/10.1080/
00140138208925071
Pingel, T., & Clarke, K. (2014). Perceptually shaded slope
maps for the visualization of digital surface models.
Cartographica: The International Journal for Geographic
Information and Geovisualization, 29(4), 225–240. https://
doi.org/10.3138/carto.49.4.2141
Podobnikar, T. (2012a). Detecting mountain peaks and delineat-
ing their shapes using digital elevation models, remote sensing
and geographic information systems using autometric meth-
odological procedures. Remote Sensing, 4(3), 784–809. https://
doi.org/10.3390/rs4030784
Podobnikar, T. (2012b). Multidirectional visibility index for ana-
lytical shading enhancement. The Cartographic Journal, 49(3),
195–207. https://doi.org/10.1179/1743277412Y.0000000012
14 P. J. KENNELLY ET AL.
Raposo, P. (2020). Variable DEM generalization using
local entropy for terrain representation through scale.
International Journal of Cartography, 6(1), 99–120.
https://doi.org/10.1080/23729333.2019.1687973
Rocca, L., Jenny, B., & Puppo, E. (2017). A continuous
scale-space method for the automated placement of spot
heights on maps. Computers & Geosciences, 109, 216–227.
https://doi.org/10.1016/j.cageo.2017.09.003
Samsonov, T. (2011). Multiscale hypsometric mapping. In A.
Ruas. (Ed.), Advances in cartography and GIScience.
Volume 1: Selection from ICC 2011, Paris, Lecture notes in
geoinformation and cartography (pp. 497–520). Springer.
https://doi.org/10.1007/978-3-642-19143-5_28
Samsonov, T., Koshel, S., Walther, D., & Jenny, B. (2019).
Automated placement of supplementary contour lines.
International Journal of Geographical Information Science, 33
(10), 2072–2093. https://doi.org/10.1080/13658816.2019.
1610965
Shannon, J., & Walker, K. (2018). Opening GIScience:
A process-based approach. International Journal of
Geographical Information Science, 32(10), 1911–1926.
https://doi.org/10.1080/13658816.2018.1464167
Tait, A. (2002). Photoshop 6 tutorial: How to create basic
colored shaded relief. Cartographic Perspectives, 42, 12–17.
https://doi.org/10.14714/CP42.550
Tanaka, K. (1950). The relief contour method of representing
topography on maps. Geographical Review, 40(3), 444–456.
https://doi.org/10.2307/211219
Tóth, T. (2011). Accidental cARTographer. Cartographic
Perspectives, 67, 19–28. https://doi.org/10.14714/CP67.111
Touya, G., Boulze, H., Schleich, A., & Quinquenel, H. (2019).
Contour lines generation in karstic plateaus for topographic
maps. Proceedings of the 29th International Cartographic
Conference (ICC), 1–8. Tokyo, Japan. https://doi.10.5194/ica-
proc-2-133-2019
Turk, G. (2000). The Stanford bunny. https://www.cc.gatech.
edu/~turk/bunny/bunny.html
USGS. (n.d.). 3D elevation program. https://www.usgs.gov/
core-science-systems/ngp/3dep/
Veronesi, F., & Hurni, L. (2014). Changing the light azimuth
in shaded relief representation by clustering aspect. The
Cartographic Journal, 51(4), 291–300. https://doi.org/10.
1179/1743277414Y.0000000100
Veronesi, F., & Hurni, L. (2015). A GIS tool to increase the
visual quality of relief shading by automatically changing
the light direction. Computers & Geosciences, 74, 121–127.
https://doi.org/10.1016/j.cageo.2014.10.015
White, D. (1985). Relief modulated thematic mapping by
computer. The American Cartographer, 12(1), 62–67.
https://doi.org/10.1559/152304085783914640
Willett, W., Jenny, B., Isenberg, T., & Dragicevic, P. (2015).
Lightweight relief shearing for enhanced terrain percep-
tion on interactive maps. Proceedings of the 33rd Annual
ACM Conference on Human Factors in Computing
Systems (pp. 3563–3572). https://doi.org/10.1145/
2702123.2702172
Wilson, K., & Worth, C. (1985). The effects of shaded relief
and hypsometric tints on map information accessibility.
The Bulletin of the Society of University Cartographers, 19
(1), 13–19.
Yang, N., Wan, L., Zheng, G., & Yang, J. (2015). Using
hachures to construct a 3D doline model automatically.
Cartographica: The International Journal for Geographic
Information and Geovisualization, 50(2), 86–93. https://
doi.org/10.3138/carto.50.2.2793
Yoéli, P. (1965). Analytical hill shading (a cartographic
experiment). Surveying and Mapping, 25(4), 573–579.
Yoëli, P. (1967). The mechanisation of analytical hill shading.
The Cartographic Journal, 4(2), 82–88. https://doi.org/10.
1179/caj.1967.4.2.82
Yokoyama, R., Shlrasawa, M., & Pike, R. J. (2002). Visualizing
topography by openness: A new application of image pro-
cessing to digital elevation models. Photogrammetric
Engineering & Remote Sensing, 68(3), 257–265.
CARTOGRAPHY AND GEOGRAPHIC INFORMATION SCIENCE 15