Conference PaperPDF Available

The SARoNGS Project: Providing Web Processing Services and Federated Access on the National Grid Service to the Landmap Service

Authors:

Abstract and Figures

In recent years, there has been an increased demand on Remotely Sensed Data (RSD) for multipurpose usage including industrial, commercial, military and academic re search. RSD is huge and growing rapidly. RSD handling, management, storage, processing, accessing and online delivery is a challenge. RSD is usually processed by individual researchers, except for the likes of ESA and other big agencies, on local PCs, this is reflected in the usual availability of image processing software for workstations and the distinct lack of such software for High Performance Computing (HPC) environments. Image processing of RSD is time consuming and as such ripe for implementation in such an environment. Grid Computing is an environment that makes use of many computing resources large and small, accessible through a common interface or set of protocols. It has the potential to handle large datasets and perform image processing by proc essing images in parallel. Two barriers prevent the spread of implementation of HPC, the hurdle of applying for computing time on the services and consequently the authentication - authorisation process and the skill needed to create and manage the p rocessin g workflow on the grid. Firstly, The National Grid Service (NGS) is one of the UK grids, with access to more than 2000 CPUs of HPC resources. The NGS offers the UK academic community access to these facilities, many of which are free at the point of use. These facilities are now accessible through the Shibboleth Access to Resources on National Grid Service (SARoNGS) Project, which allows users to securely login and use grid resources using their university’s central logging obviating the ne ed for complex g rid credentials. Secondly: Landmap has been working on implementing the outputs of the SARoNGS Project to provide a use case of grid image processing.
Content may be subject to copyright.
Proceedings of RSPSoc 2009 Annual Conference, 8-11th September 2009, Leicester, UK
509
The SARoNGS Project: Providing Web Processing
Services and Federated Access on the National Grid
Service to the Landmap Service
Amer Alroichdi, Andrew Rowley, Anja Le Blanc, Mike Jones, Gail Millin,
James Schumm and Kamie Kitmitto
The University of Manchester, Mimas, Oxford Rd. Manchester, M13 9Pl
Email: amer.alroichdi@manchester.ac.uk
Summary
In recent years, there has been an increased demand on Remotely Sensed Data (RSD) for
multipurpose usage including industrial, commercial, military and academic research. RSD is
huge and growing rapidly. RSD handling, management, storage, processing, accessing and
online delivery is a challenge. RSD is usually processed by individual researchers, except for
the likes of ESA and other big agencies, on local PCs, this is reflected in the usual availability
of image processing software for workstations and the distinct lack of such software for High
Performance Computing (HPC) environments. Image processing of RSD is time consuming
and as such ripe for implementation in such an environment. Grid Computing is an
environment that makes use of many computing resources large and small, accessible through
a common interface or set of protocols. It has the potential to handle large datasets and
perform image processing by processing images in parallel. Two barriers prevent the spread
of implementation of HPC, the hurdle of applying for computing time on the services and
consequently the authentication-authorisation process and the skill needed to create and
manage the processing workflow on the grid.
Firstly, The National Grid Service (NGS) is one of the UK grids, with access to more
than 2000 CPUs of HPC resources. The NGS offers the UK academic community
access to these facilities, many of which are free at the point of use. These facilities
are now accessible through the Shibboleth Access to Resources on National Grid
Service (SARoNGS) Project, which allows users to securely login and use grid
resources using their university’s central logging obviating the need for complex grid
credentials.
Secondly: Landmap has been working on implementing the outputs of the SARoNGS
Project to provide a use case of grid image processing.
1 Introduction
Landmap is a Mimas Data Centre remotely sensed data based service providing access to all
UK academic institutions. It provides Optical, Radar and Elevation data acquired from both
airborne and space borne platforms. The service implements Open Geospatial Consortium
(OGC) compliant Web Mapping Service (WMS), and Web Coverage Service (WCS).
LANDMAP data set has recently grown substantially, with the new acquisition of the UK
Cities Revealed aerial photography, Lidar data and TopSat. This growth is set to continue due
to continued JISC investment in spatial data acquisition at least for the next year. New targets
for acquisitions are new data such Disaster Monitoring Constellation (DMC). This increase in
data volume has led LANDMAP to investigate the way we serve data to the academic
community in the UK. Landmap recognises the need to allow users to perform processing on
Proceedings of RSPSoc 2009 Annual Conference, 8-11th September 2009, Leicester, UK
510
its data arachive through the introduction of an OGC compliant Web Processing Service
(WPS). LANDMAP has investigated new solutions to improve its quality of service and meet
the challenge of serving a huge amount of data in flexibly and intuitively. This has been
approached by introducing ERDAS Apollo Image Manager (EAIM) software as an online
data catalogue and image server. This software was further modified to enable OGC -
compliant Grid-enabled Web Processing Service (G-WPS).
The LANDMAP G-WPS requires authentication and Authorisation to gain access on the
National Grid Service (NGS), this was done through the work of the SARoNGS project
(Shibboleth Access to Resource on the National Grid Service). SARoNGS solution allows
users to gain access to NGS Resources via their own institutions usernames and passwords.
2 Data Archive and Online Delivery
The EAIM web client provides the access mechanisms for the various capabilities of the
EAIM. The client has a customisable interface that provides an interface to the data catalogue
allowing for visualisation and download of the various datasets (Figure 1). It allows users to
search by keyword, browse, navigate and download archived data in different format. The
download function is provided through an FTP link to send to a user’s email account. In
addition, it allows saving image web context as an XML file for future use or for sharing with
other users.
The EAIM web Client was modified to allow for access to the landmap implemented Grid-
enabled WPS (G-WPS). This allows users to perform geo-processing on the selected data
over the NGS resources. This is done through the implementation of Devolved Authentication
better known as Shibboleth in the EAIM web Client to allow access to the data, the same
credentials are then used to acquire a Grid Certificate to gain access to the NGS resources.
The web client is further enhanced through the addition WPS capabilities though the
implementation of open Source 52North WPS solution. We will be expanding on each of
these elements later in this document.
Figure 1. ERDAS Apollo Image Manager (EAIM) web client after a login was performed.
Proceedings of RSPSoc 2009 Annual Conference, 8-11th September 2009, Leicester, UK
511
3 Shibboleth Logging onto EAIM Web Client
Upon clicking on log in via Shibboleth radio button user will be prompted to select their
institute name and will be allowed to select for how long browser should remember the
password, Figure 3. Once the institution is selected, the user will be redirected to their home
institution federated access page to login in the usual manner. Once their institution verifies
their credentials they are redirected to an NGS web page confirming their role (Figure 3). If
the user affirms their role they will be brought back to the modified EAIM web client with a
new NGS ID as shown previously in Figure 1.
Figure 2. Shibboleth Log in screen.
Figure 3. NGS Certificate presented allowing for processing on the NGS.
Proceedings of RSPSoc 2009 Annual Conference, 8-11th September 2009, Leicester, UK
512
4 Grid Computing and Gridification
The term Grid computing first appeared in the mid 1990s to stand for distributed online-
interconnected computers that accessible via open and standardized interfaces. The internet
allows users to share information, while the Grid allows users to share computing power, with
an ultimate aims to make the global network of computing resource as one supercomputer
(CERN, 2008). NGS in the UK has over 2000 networked computer processors with more than
8 TB storage space distributed over five UK institutes. The term gridification means
developing or modifying and existed application or a service to work in a grid environment
(Kruger, 2008). NSG is one of the UK computing grids which aims to provide electronic
access for UK academic communities to all computational resources and facilities that are
required to perform their research independently from their geographical location or their
institute resource. The NGS provide can now provide access to its resources through Single
Sign on Federated Access, Shibboleth, implemented through the SARoNGS project.
5 NGS Authentication and Authorisation using SARoNGS
The idea behind SARoNGS is to use one federated access to its services and facilities without
going throw registration forms for each individual. For example, logging to NGS resources to
access the G-WPS via EAIM web client will perform many behind the scene operation that
users do not need to know or to see. Figure 4 illustrates how SARoNGS works.
Figure 2. A graphical illustration of how SARoNGS project architecture works.
In our case the browser would be EAIM web client trying to access an NGS resource.
According to SARoNGS project plan published in JISC (2009) SARoNGS uses the results
from SHEBANGS project which translate an Identity Provider’s SAML assertion obtained
through browser based Shibboleth methods, into Globus Security Infrastructure (GSI)
credentials and provide it Credential Translation Service (CTS). During the authentication
process of an individual, the CTS obtains a short term X.509 credential from the MyProxy
CA (WP 2). Using this credential, it obtains a VOMS Attribute Certificate from the VOMS
server specified by the individual. A GSI credential incorporating the VOMS Attribute
Certificate is then manufactured and this is delegated to the UK NGS MyProxy server where
it is accessible to the Individual directly or via a Portal. A portal able to obtain this credential
Proceedings of RSPSoc 2009 Annual Conference, 8-11th September 2009, Leicester, UK
513
is then able to access grid resources on behalf of an individual, enabling existing VOMS-
aware grid middleware to make authorisation decisions. Figure 03 shows the basic
architecture: Step 1 provides the Portal-user the ability to use the SARoNGS mechanism to
login. Steps 2, 3 and 4a are the familiar Shibboleth authentication and attribute passing
mechanisms. Step 4b shows the ShibGrid style certificate issuing process. Step 4c shows the
gathering of VO credentials (VOMS Attribute Certificates). Step 4d shows the SHEBANGS
style movement of the credentials into a controlled public credential store (MyProxy). Steps
5, 6 and 7 show the familiar grid portal access mechanisms.
6 Web Processing Service (WPS)
The OGC compliant WPS describes standardized specifications for web interface to carry out
processing and publishing geodata. These specifications allow users to interact with available
online image processing exposed by a service provider. The WPS image processing could be
as simple as adding or subtracting images and could be very complicated calculations models
(OGC, 2007; Baranski, 2009). LANDMAP adapted the 52north Grid-enabled WPS (G-WPS),
which is compliant with OGC specifications, within the EAIM web client. The 52north G-
WPS is a java based program which uses the open source Sextant geo-processing algorithm
library and runs in tomcat server (www.52north.org). The library contains over 250
algorithms that can be performed on raster and vector data including image supervised and
unsupervised classification, such as NDVI etc.
Figure 5. The overall G-WPS architecture of the 52north.
There are three main steps for the WPS interface (Figure 5):
1. A GetCapabilities operation provides details of the geoprocessing algorithms that the
WPS can perform.
2. Then a DescribeProcess operation is performed, which is commonly done utilising an
XML (Extended Mark-UP Language) file providing a total description of the geo-
processing algorithm including the input and output parameters, data type and format.
3. Finally an Execute operation performs the execution of the process according to the
XML file description for the specific algorithm on the grid (Brauner and Schaeffer,
2008) (Figure 02). Upon the execution of the process three sub-processing operations
are performed implementing IGridAlgorithm interface which will run the process
Proceedings of RSPSoc 2009 Annual Conference, 8-11th September 2009, Leicester, UK
514
logic, split data into smaller data chunks and join data results coming from the parallel
executed for each data chunk
52north developed their own web client, Udig, that performs dynamic retrieval of the WPS
results from the server performing the WPS. LANDMAP plugged the 52north G-WPS to
EAIM web client in a non-dynamic where the G-WPS results are emailed to the user’s
account as an FTP link once the operation is finished. This was done to cater for the
asynchronous nature of Grid Operations, which schedule jobs on the Grid according to
various priorities. Processing of G-WPS requests are not guaranteed to be instantaneous as in
a regular WPS fashion (Figure 6).
Figure 6. Illustration of how 52north G-WPS is integrated to EAIM web client. The Get Capabilities button
allows for the selection of processing algorithms.
In the Landmap application the selected data for processing are copied to the NGS database
before processing. As proof of concept Landmap run a WPS NDVI algorithm on Landsat 7
data successfully. The fact that the obtained results were performed faster than on a desk top,
is not interesting, however, the fact that you could schedule such a task for a large number of
images is of interest.
Two things should be noted here, the G-WPS are not truly Grid Enabled as most Geo-
processing algorithms are not parallelized and the splitting of data sets into smaller subsets
that are processed in parallel, then the result re-assembled is not utilized. The nature of
Proceedings of RSPSoc 2009 Annual Conference, 8-11th September 2009, Leicester, UK
515
algorithms available for WPS though interesting are not those would lend themselves to huge
requirement in processing power.
In the context of Remote Sensing data it is not advisable for the data and the processing
power to be separated from each other. Though technically OGC standards cater for the
separation, practically the shifting of great amounts of data from one server to other in order
to do the processing is not advisable.
EAIM is expected to provide WPS capabilities through their continuous improvement and
development of their software. Landmap will be investigating the possible deployment of an
experimental service utilising these new capabilities.
7 Conclusions
SARoNGS provides an easy way for academic users to gain access to computing power that
is the preserve of the few. However, access to computing power alone does not guarantee that
users will utilise this resource. There is a need for access to a suite of software and data
resources that academics can utilise to conduct research and explore ideas. The OGC
framework of services provide access to data resources in an interoperable way. In the mean
while OGC WPS specifications allows for the tools necessary to carry on research and
exploration.
There still exist a gap between the theory of grid enabling geo-resources and the practicality
of carrying useful applied research using these tools. However, the gap is closing and the set
of tools necessary to do such work is nearing.
References
BARANSKI, B., 2008. Grid Computing Enabled Web Processing Service, GI-Days, Münster,
Germany June 16-17, 2008
BRAUNER, J., and SCHAEFFER, B., 2008. Integration of grass functionality in Web based
SDI service chains. Proceedings of the Academic Track of the 2008 Free and Open
Source Software for Geospatial (FOSS4G) Conference, incorporating the GISSA 2008
Conference, 29 September-3 October 2008, Cape Town, South Africa.
CERN, 2008. http://gridcafe.web.cern.ch/gridcafe/ (last access 27.06.2009).
JISC, 2009, Shibboleth Access to Resources on the National Grid Service.
http://www.jisc.ac.uk/media/documents/programmes/einfrastructure/sarongsprojectplan
web.pdf (last accessed 26.06.2009).
KRUGER, A., 2008. Web processing Service (WPS): grid-enabled spatial processing
capabilities, GC-OGF Collaboration Workshop. 22nd Open Grid Forum. OGF 22.
Open Geospatial Consortium Inc, 2007, OpenGIS® Web Processing Service,
http://www.opengeospatial.org/standards/wps (last accessed 26.06.2009).
ResearchGate has not been able to resolve any citations for this publication.
Article
Full-text available
The OGC Web Processing Service (WPS) specification defines a standardized interface to publish and perform geospatial processes over the web. Such a process can range from a simple geometric calculation to a complex simulation model. When computation large amounts of data and do complex calculations, the use of Grid Computing and related methods and technologies are a good choice for achieving high performance. At first glance, the WPS and its needs of computation resources is a perfect candidate to benefit from Grid Computing. This paper presents an architecture of a grid­enabled WPS. The presented proof­of­concept implementation demonstrates that Grid Computing and related methods and technologies increase calculation performance and improve service availability. Furthermore this paper presents an detailed outlook of interesting topics for further research activities.
Conference Paper
Full-text available
The national grid service (NGS) provides access to compute and data resources for UK academics. Currently users are required to have an X.509 certificate from the UK e-science certification authority (CA) or one of its international peers to access the NGS. The CA must satisfy the requirements for internationally agreed assurance levels and some users find the processes of obtaining and managing certificates difficult. Shibboleth, an implementation of federation identity based authentication, has been widely deployed in academic environments in the UK. The SARoNGS project, was proposed to integrate the Shibboleth and X.509 based infrastructures, to deliver a production level service for accessing the NGS in a user friendly way. This paper describes an architecture by which users are authenticated by the UK access management federation to acquire low assurance credentials to access Grid resources on the NGS. Users can login to NGS resources via NGS portal, using their local institution's authentication system.
Integration of grass functionality in Web based SDI service chains
  • J Brauner
BRAUNER, J., and SCHAEFFER, B., 2008. Integration of grass functionality in Web based SDI service chains. Proceedings of the Academic Track of the 2008 Free and Open Source Software for Geospatial (FOSS4G) Conference, incorporating the GISSA 2008
Web processing Service (WPS): grid-enabled spatial processing capabilities, GC-OGF Collaboration Workshop. 22 nd Open Grid Forum
  • A Kruger
KRUGER, A., 2008. Web processing Service (WPS): grid-enabled spatial processing capabilities, GC-OGF Collaboration Workshop. 22 nd Open Grid Forum. OGF 22.