Content uploaded by Thomas Schwotzer
Author content
All content in this area was uploaded by Thomas Schwotzer on Apr 12, 2017
Content may be subject to copyright.
Open Historical Data Map - a prototype
Thomas Schwotzer
HTW Berlin
University of Applied Sciences
Germany, Berlin +49 30 5019-2604
Email: thomas.schwotzer@htw-berlin.de
Abstract—Open Street Map (OSM) is one example of free and
open map service. Volunteers chart geographic objects and enter
those information into the systems. OHDM is going to adopt
that idea but for historic maps and data. OHDM will offer
historic data as Web Map Service and interfaces to down- and
upload historic spatial information. A technical prototype can be
presented during the workshop. That paper describes the used
components and some major obstacles which had to be overcome.
I. OHDM DATA
The OHDM data model is already published [OHDM16]
and will not be discussed here in detail. In general, OHDM
stores geographic objects, geometries and their temporal rela-
tions. Quite often, same geometry describes different objects
when time passes: Streets are renamed for instance and us-
age of buildings can change. Same objects can also change
geometry for instance if a building is expanded.
PostGIS was our choice for the OHDM data base. Postgres
is free and open software. The PostGIS plugin makes it a fast
spatial database. We use a Debian server with 40 Terrabyte
hard drive, 32 kernels and 30 GByte Memory. Planet wide
OSM data from Jan, 19th 2017 (3.7 billion nodes, 250 million
ways) require less than 1 Terrabyte. That hardware is sufficient
for that amount of data.
II. WE B MAP SE RVICE
Web Map Service is the OGC standard for offering digital
maps [WMS]. We chose GeoServer1as WMS implementation.
It is well documented any easy to administrate. These are sig-
nificant criteria in a university environment in which students
often enter and leave the project.
We produce rendering tables out of the OHDM core
database, see figure below. Each line of those tables contains
object description, its geometry and validity. The renamed
street would appear in two lines which differ in description
but not geometry. That data model is apparently redundant but
optimized for rendering. With a spatial index on those tables,
Geoserver is able to render tables with about 10 million lines
under 10 seconds. Geoserver is an appropriate technology for
the project.
There is a pleasant side effect: We can offer about 100 WMS
layers each containing an OSM feature. For instance, there is a
layer called railway_lines containing all railways. Other
applications can use those layers for their own purpose.
1www.geoserver.org
It took our server about 5 hours to produce rendering tables
covering whole Germany. We calculate that production of
worldwide rendering tables are done within a day on our
hardware.
III. OSM DATA IMP ORT /U PDATE
OHDM requires data. OSM with its huge amount of data is
a perfect choice to test our hardware and software components.
The current planet.osm file has a size of about 850
GByte. Those data are imported into an intermediate database
what we call the OSM import, see figure below.
In a second step (OSM extraction), semantic information
(like names), geometries and validity are extracted from OSM
data and imported into the OHDM database. An annual update
is planned. Changes in OSM are saved with the OHDM
database which remember that history. We expect an annual
growth rate of less than a Gigabyte each year.
That initial OSM importer was written in Java which is
the usual language in our teaching programs. It communicates
with the OHDM PostGIS database via JDBC and issues
INSERT statements for each OSM entity. The whole import
takes about 10 days on our hardware. It is sufficient for a
prototype when having in mind that this process will only
be performed once a year. Nevertheless, is worth considering
alternatives.
PSQL is a PostGIS tool which can process these statement
a fifth faster than JDBC. There is another alternative, though.
PostGIS offers another structure and API supporting fast
import and export of tables (pgdump) for backup and recov-
ery. That API is documented and could be used for our project.
Some first measurements indicate that using that API would
require a hundredth (!) of time. We are optimistic to import
worldwide OSM data within 15 hours.
IV. CON CL US IO N AN D OUTLOOK
PostGIS is a stable and fast spatial database server that fits
perfectly to our requirements.
Creating optimized rendering tables and creating spatial
indexes are basis of a fast rendering process. Geoserver is a
reliable and well-documented WMS server that is fast enough
for our needs.
Importing other sources than OSM into OHDM is most
relevant now. We are in close contact to libraries especially
in Berlin. Our next task is to incorporate historic data into
OHDM and to offer it to interested parties.
REF ER EN CE S
[WMS] Jeff de la Beaujardiere (editor): OpenGIS Web
Map Server Implementation Specification 1.3.0, 2006
http://www.opengeospatial.org/standards/wms
[OHDM16] Thomas Schwotzer: Open Historic Data Map (work in progress)
First International Workshop on Exploring Old Maps, June 2016,
Luxembourg
https://www.researchgate.net/publication/303818952
Open Historical Data Map OHDM - work in progress
Fig. 1. OHDM Components and processes