Technical ReportPDF Available

Quality Assurance in Marine Biological Monitoring. A report prepared for the Healthy and Biologically Diverse Seas Evidence Group and the National Marine Biological Analytical Quality Control scheme

Authors:
  • Berks Bucks and Oxon Wildlife Trust

Abstract

The importance of quality assurance in marine biological monitoring is a priority of the UK Marine Monitoring and Assessment Strategy and is implicit in several European directives, such as the Water Framework Directive, the Marine Strategy Framework Directive and the OSPAR Joint Assessment Monitoring Programme. Quality Assurance in marine biology is the systematic examination and evaluation of all aspects of a monitoring programme (from survey design, field methods, laboratory methods, data analysis and storage) to ensure that standards of data quality and comparability between organisations are being met. This in turn provides confidence in the evidence base for policy and decision making. The following paper presents the principles of the current best practice Quality Assurance system which the UK’s government agencies, their contractors, partners and data providers are expected to adhere to.
Quality Assurance in Marine Biological Monitoring
A report prepared for the Healthy and Biologically Diverse Seas Evidence Group
and the National Marine Biological Analytical Quality Control scheme
Prue Addison, Environment Agency/Joint Nature Conservation Committee,
January 2010
Cover Note:
The importance of quality assurance in marine biological monitoring is a priority of
the UK Marine Monitoring and Assessment Strategy and is implicit in several
European directives, such as the Water Framework Directive, the Marine Strategy
Framework Directive and the OSPAR Joint Assessment Monitoring Programme.
Quality Assurance in marine biology is the systematic examination and evaluation of
all aspects of a monitoring programme (from survey design, field methods, laboratory
methods, data analysis and storage) to ensure that standards of data quality and
comparability between organisations are being met. This in turn provides confidence
in the evidence base for policy and decision making.
The following paper presents the principles of the current best practice Quality
Assurance system which the UK’s government agencies, their contractors, partners
and data providers are expected to adhere to.
Quality Assurance in Marine Biological Monitoring, January 2010
1
1 Documentation
The following types of documentation are expected for an organisation undertaking
any type of marine biological monitoring:
1.1 Standard Operating Procedure
Standard Operating Procedure(s) (SOP) should be produced/adopted for all
monitoring methods which an organisation is engaged in. In order to ensure that all
aspects of Quality Assurance have been met for a particular monitoring method, a
SOP should include:
1. A clear statement of the qualitative and quantitative goals of the
monitoring programme, which demonstrates that the specified methods are
fit for purpose.
2. Methodological details of all steps performed in a monitoring programme,
including all monitoring methods (e.g. both field and laboratory aspects),
details of equipment/instruments used and their calibration and
maintenance schedule.
3. Details of the procedures related to the generation, analysis and archiving
of data.
4. The quality assurance system in place.
1.2 Quality Manual
A Quality Manual details an organisation’s quality assurance system/policy (i.e. all
aspects of quality assurance described in this document), which allows systematic
audits of a monitoring programme to ensure that all aspects of quality assurance are
being met. It should also detail the frequency at which different aspects of quality
assurance must be audited and the standards/limits that should be met and what
remedial action is necessary if the standards/limits are not met.
2 Adherence to standardised methods
Development of and adherence to national, European or international standardised
methods (where they exist) is a statutory requirement in the Water Framework and the
Marine Strategy Framework Directives. The adoption of standardised methods
ensures consistency between organisations and will allow for the production of data
which is of an equivalent scientific quality and comparability.
Certification/accreditation can also be obtained for an organisation which follows a
national, European or International standard through accreditation schemes such as
Good Laboratory Practice (GLP) or United Kingdom Accreditation Service (UKAS).
If no standard exists, it is also possible to become accredited in an organisation’s own
method.
Section 10 ‘Standards for Marine Monitoring’ shows the current list (as of January
2010) of all British Standards (BSi), International Organization for Standardisation
(ISO) standards and European Committee for Standardisation (CEN) standards which
exist for marine biological monitoring.
Quality Assurance in Marine Biological Monitoring, January 2010
2
Many of the UK government agencies protocols and standards for UK and European
marine monitoring programmes are also easily accessible on the Marine Monitoring
Protocols Database.
3 Adherence to data standards and guidelines
Adherence to data standards and guidelines (where they exist) on a national, European
or international scale is necessary to demonstrate that the data produced from an
organisation’s monitoring programme is of an equivalent scientific quality and
comparability at a national, European or international scale. Such data standards and
guidelines include:
1. Species lists which contain current taxonomic names and synonyms (e.g.
World Register of Marine Species (WoRMS), which is now endorsed by the
Healthy and Biologically Diverse Seas Evidence Group (HBDSEG)).
2. Data and metadata guidelines which define data and metadata formats to
ensure data can be easily and shared in the future (e.g. Marine Environmental
Data and Information Network (MEDIN) data guidelines and Mapping
European Seabed Habitats (MESH) standards.
4 Calibration and maintenance of instrumentation and equipment
Instrumentation and equipment used in an organisations laboratory and field aspects
of monitoring must be subject to routine calibration and maintenance to ensure against
bias in results. Maintenance includes the general servicing of equipment and
instruments, appropriate cleaning and careful use. Calibration is the operational
checks of equipment and instrumentation at specified intervals to ensure the accuracy
of data collected.
5 Voucher/Reference collection
For any biological work, a voucher/reference collection containing examples of taxa
encountered should be compiled and retained for each monitoring programme. The
benefits of a voucher collection are two-fold, as they provide evidence of species
identification from a particular monitoring programme (e.g. if any identification
queries are raised in the future), and the voucher collection can be used as a reference
for future identification of taxa.
A voucher collection can consist of preserved specimens or images of
specimens/habitats. Specimens/images should be fully labelled stating at least the
taxon/habitat name, sample location and date, and the identifier. Ideally the details
recorded (including the taxon/habitat identification) should be confirmed by a second
biologist.
6 Taxonomic resources
A standard collection of taxonomic resources should be used to aid in the
identification of marine biological taxa. This, along with the use of a standard species
list, will ensure that the identification of taxa follow the current literature. Taxonomic
resources can include:
Quality Assurance in Marine Biological Monitoring, January 2010
3
1. A collection of up to date literature and identification/reference books.
2. A comprehensive and regularly updated reference and taxonomic key library.
3. Expert second opinion.
7 Staff competency
A minimum competency (education, training, work experience and/or other
demonstrated skills) necessary for staff to undergo different types of marine
monitoring work must be defined by the organisation. This competency must be
checked and documented within a Training Manual. For less experienced staff who
are undergoing training, appropriate supervision of work should be provided until the
required competency in the method is achieved. It is important for an organisation to
consider staff competency in order to plan for future succession to maintain a skilled
work force. Consideration should be given to ensure at least two staff members are
competent in each type of marine monitoring activity.
Individual competency can be improved/maintained through participation in internal
or external training in all relevant aspects of marine monitoring work:
7.1 Training
Training ensures that staff are competent in specific monitoring methods, and that
staff are kept up to date with the current literature and best-practice methods.
Training can be done internally within an organisation, however participation in
external training through quality assurance, accreditation or qualification schemes
should also be done to demonstrate an organisation’s competency is similar to a wider
group of organisations to ensure the quality and comparability of data at a larger
scale. Training can generally take the form of workshops, proficiency tests or
qualifications:
7.1.1 Workshops
A workshop involves the training/teaching of new techniques or taxonomic
identification to staff and the sharing of good practice. Workshops can be run
internally or externally through universities, museums or marine
organisations/associations.
7.1.2 Proficiency tests
A proficiency test involves a sample, of a known content, being given to a staff
member to assess. The staff member is tested on their assessment of the sample,
which for biological samples can include their extraction, identification and
enumeration of taxa from the sample. For chemical or physical samples a staff
member would be tested in the accuracy of their chemical and physical measurements
from the sample.
Proficiency tests can be run through an external quality assurance scheme, where they
are often referred to as Interlaboratory Comparisons or Ring Tests.
Quality Assurance in Marine Biological Monitoring, January 2010
4
7.1.3 Qualifications
Qualifications (additional to tertiary qualifications) such as an Identification
Qualification awarded by a professionally recognised institute can demonstrate an
individuals proficiency has reached an externally recognised standard at the time of
the test.
8 Quality Control
8.1 Internal Quality Control
Internal Quality Control (QC) is a system of routine technical checks to measure and
control the quality of operational techniques and equipment and data and analysis
within an organisation.
QC of operational techniques and equipment is where techniques and equipment are
checked against certified reference materials, or repeat analyses are conducted and
compared using a control chart (where the precision of measurements are compared
against an expected standard/limit of error). QC also includes details of remedial
action in cases where expected standards/limits are not met.
QC of data and analysis is where samples are re-checked and compared against the
original and compared to an expected standard/limit of error/deviation from the
original. This measures the repeatability of the method of analysis, and highlights
factors such as inter-worker variability. QC must also include details of remedial
action in cases where expected standards/limits are not met.
In general internal QC should be done within an organisation by a staff member who
is different from the staff member who conducted the original sample analysis.
Recommended approaches to internal QC can be found in BSi/ISO/CEN standards
such as those listed in Section 10 ‘Standards for Marine Monitoring’.
8.2 External Quality Control
Participation in an External Quality Control (QC) or Analytical Quality Control
(AQC) scheme is mandatory for organisations involved national monitoring programs
such as CSEMP and WFD. External QC involves a selection of samples from an
organisation being re-analysed by an external QA scheme. The similarity of the
original is compared to the QC sample and a pass or fail flag is given based on a
minimum expected standard of similarity. The data from the samples under test may
be qualitative, quantitative, continuous or discrete, and derived from laboratory
analysis or field survey. By working to a standard set by the external QA scheme,
participation in external QC will ensure comparability of data quality across multiple
organisations.
The batch of samples associated with sample(s) which are subject to external QC
must be retained until QC samples are deemed to have passed (or remedial action of
failed samples has been completed satisfactorily). All reports and Statement of
Performance certificates provided by the external QA scheme should be filed in the
Quality Manual and available for inspection upon request. Samples should be
Quality Assurance in Marine Biological Monitoring, January 2010
5
flagged as pass/fail prior to being put onto a national database, with failed samples not
databased until remedial action has been taken.
In the case of the National Marine Biological Analytical Quality Control (NMBAQC)
scheme’s Invertebrate component, benthic infaunal samples are re-analysed and the
similarity (e.g. Bray-Curtis Similarity Index (BCSI)) of the original sample and the
QC sample is compared. A pass or fail flag is given to the participating organisation’s
sample based on a minimum acceptable similarity (e.g. >90% BCSI). The
organisations are required to achieve the scheme’s quality standards and complete any
required remedial actions.
9 Quality Management System
An organisation should maintain an appropriate quality management system which
provides an audit trail of the entire quality assurance system (as outlined in the
Quality Manual). The frequency at which audits should be done will be specified in
the Quality Manual. Audits ensure that any biases (e.g. inter-worker variability) are
controlled so that quality outputs are maintained and confidence in the outputs can be
demonstrated. Audits can take many forms, and it is likely that there will be different
types of audits for an organisation and all of its marine monitoring practices. The
following are the main types of audits that an organisation should complete to ensure
they have a working QA system in place:
9.1 Quality Control Audit
Examination of in-house QC data, check of appropriate control limits, and corrective
action is documented where QC data has breached control limits.
9.2 Vertical Audit
A random selection of data points, where each item is traced to the Method,
Instrument/Equipment, Calibration and internal QC, and internal or external AQC.
9.3 Calibration Audit
Check of equipment calibration certificates, in-house maintenance and checks comply
with calibration schedule and actual results documented.
9.4 Method Witnessing
Comparison of written procedure to observed procedure under real conditions. Any
differences must be resolved through corrective action where needed.
9.5 Training Manual
Criteria stating the minimum competency (education, training, work experience
and/or other demonstrated skills) necessary for staff to undergo different types of
marine monitoring work must be defined in the Training Manual. The competency of
each staff member must be signed off initially against all criteria and evidence of
ongoing competency must be maintained on a regular basis.
Quality Assurance in Marine Biological Monitoring, January 2010
6
10 Standards for Marine Biological Monitoring
10.1 Fish
BS EN 14757:2005. Water quality - Sampling of fish with multi-mesh gillnets. 30pp.
BS EN 14962:2006. Water quality - Guidance on the scope and selection of fish
sampling methods. 28pp.
10.2 Phytoplankton
BS EN 15204:2006. Water quality - Guidance standard on the enumeration of
phytoplankton using inverted microscopy (Utermöhl technique). 46 pp.
BS EN 15972:2009. Water quality - Guidance on quantitative and qualitative
investigations of marine phytoplankton. 29 pp.
10.3 Benthic Infauna
EN ISO 16665:2005 (Under Review). Water quality - Guidelines for quantitative
sampling and sample processing of marine soft-bottom macrofauna.
10.4 Epibiota
CEN/TC230 N 646 (Draft). Water quality – Visual seabed surveys using remotely
operated and towed observation gear for collection of environmental data.
EN ISO 19493:2007. Water quality - Guidance on marine biological surveys of hard-
substrate communities. 32 pp.
10.5 General
BS EN ISO 5667-1:2006. Water quality - Sampling - Part 1: Guidance on the design
of sampling programmes and sampling techniques. 42 pp.
BS EN ISO 5667-3:2003. Water quality -Sampling - Part 3: Guidance on the
preservation and handling of water samples. 31 pp.
10.6 Marine Sediment
BS EN ISO 5667-19:2004. Water quality - Part 19: Guidance on sampling in marine
sediments. 26 pp.
10.7 Quality Assurance and Quality Control
BS EN 14996:2006. Water quality – Guidance on assuring the quality of biological
and ecological assessments in the aquatic environment. 14pp.
BS EN ISO/IEC 17025: 2005. General requirements for the competence of testing
and calibration laboratories. 28pp.
Quality Assurance in Marine Biological Monitoring, January 2010
7
BS EN ISO 19011:2002. Guidelines for quality and/or environmental management
systems auditing. 32 pp.
CEN/TC 230/WG 2/TG 6: N 59 (Working document). Water Quality – Guidance
standard on the design and analysis of interlaboratory comparison studies for
ecological assessment. 8pp.
ICES, 2004. Biological monitoring: General guidelines for quality assurance. Ed. by
H. Rees. ICES Techniques in Marine Environmental Sciences, No. 32. 44 pp.
... The next challenge to be tackled is how to encourage and facilitate the sharing of environmental data collected by industry, such as commercial fisheries and proponents undertaking environmental impact assessment, where access to these data is commonly restricted by commercialin-confidence clauses. Responsible and effective use of shared monitoring data will require: strict data quality assurance/quality control procedures to ensure the quality of data prior to sharing (Addison, 2010), standardized metadata specifying essential details of the data to minimize potential for mis-use (Vitolo et al., 2015), approaches to protect commercial-in-confidence elements, and a robust data sharing policy that benefits both the user and data provider by supporting the ongoing funding of the data collection (Juffe-Bignoli et al., 2016). ...
Article
Sustainable management and conservation of the world’s oceans requires effective monitoring, evaluation, and reporting (MER). Despite the growing political and social imperative for these activities, there are some persistent and emerging challenges that marine practitioners face in undertaking these activities. In 2015, a diverse group of marine practitioners came together to discuss the emerging challenges associated with marine MER, and potential solutions to address these challenges. Three emerging challenges were identified: (i) the need to incorporate environmental, social and economic dimensions in evaluation and reporting; (ii) the implications of big data, creating challenges in data management and interpretation; and (iii) dealing with uncertainty throughout MER activities. We point to key solutions to address these challenges across MER activities: (i) integrating models into marine management systems to help understand, interpret, and manage the environmental and socio-economic dimensions of uncertain and complex marine systems; (ii) utilizing big data sources and new technologies to collect, process, store, and analyze data; and (iii) applying approaches to evaluate, account for, and report on the multiple sources and types of uncertainty. These solutions point towards a potential for a new wave of evidence-based marine management, through more innovative monitoring, rigorous evaluation and transparent reporting. Effective collaboration and institutional support across the science–management–policy interface will be crucial to deal with emerging challenges, and implement the tools and approaches embedded within these solutions.
... The next challenge to be tackled is how to encourage and facilitate the sharing of environmental data collected by industry, such as commercial fisheries and proponents undertaking environmental impact assessment, where access to these data is commonly restricted by commercialin-confidence clauses. Responsible and effective use of shared monitoring data will require: strict data quality assurance/quality control procedures to ensure the quality of data prior to sharing (Addison, 2010), standardized metadata specifying essential details of the data to minimize potential for mis-use (Vitolo et al., 2015), approaches to protect commercial-in-confidence elements, and a robust data sharing policy that benefits both the user and data provider by supporting the ongoing funding of the data collection (Juffe-Bignoli et al., 2016). ...
Article
Full-text available
Sustainable management and conservation of the world's oceans requires effective monitoring, evaluation, and reporting (MER). Despite the growing political and social imperative for these activities, there are some persistent and emerging challenges that marine practitioners face in undertaking these activities. In 2015, a diverse group of marine practitioners came together to discuss the emerging challenges associated with marine MER, and potential solutions to address these challenges. Three emerging challenges were identified: (i) the need to incorporate environmental, social and economic dimensions in evaluation and reporting; (ii) the implications of big data, creating challenges in data management and interpretation; and (iii) dealing with uncertainty throughout MER activities. We point to key solutions to address these challenges across MER activities: (i) integrating models into marine management systems to help understand, interpret, and manage the environmental and socio-economic dimensions of uncertain and complex marine systems; (ii) utilizing big data sources and new technologies to collect, process, store, and analyze data; and (iii) applying approaches to evaluate, account for, and report on the multiple sources and types of uncertainty. These solutions point towards a potential for a new wave of evidence-based marine management, through more innovative monitoring, rigorous evaluation and transparent reporting. Effective collaboration and institutional support across the science-management-policy interface will be crucial to deal with emerging challenges, and implement the tools and approaches embedded within these solutions.
Book
Full-text available
Spillages of oil and chemicals at sea can be high profile events and can result in significant environmental damage. Effective response to marine spills is essential if risk to the public and the marine environment is to be minimised and effective clean up and recovery options initiated. In general, requirements for key response activities such as the initiation of counter pollution measures, situational awareness, clean up and recovery are established within international conventions and implemented through national contingency plans. However, the key element of environmental monitoring and impact assessment is rarely included and it is to facilitate best scientific practice and management in post-spill monitoring that these guidelines have been developed. The importance of prompt and effective environmental monitoring is an important part of an integrated spill response as it is only through this that the risks and impacts to the human food chain, the marine ecosystem and commercial marine resources can be ascertained. Furthermore, it is only through monitoring that we can gather the data necessary to establish the effectiveness of the response operations and any subsequent actions taken to mitigate impacts or promote recovery.It is therefore imperative that scientifically robust approavhes, methods, and processes are considered in developing a monitoring plan so that it can be implemented in a prompt and cost-effective manner and to ensure that the results are fit for purpose and adhere to scientific standards. In the United Kingdon, the cross-government Premiam (Pollution Response in Emergencies: Marine Impact Assessment and Monitoring) initiative works to promote effective post-spill monitoring and these guidelines, now referred to in the UK National Contingency Plan, for a key deliverable. The guidelines begin by outlining the key principles of an environmental monitoring programme by establishing the key aims and the fundamental answers to why, where, what and how we need to monitor. Understanding these key aims is essential to the planning process outlined in this guide which then proceeds to provide the necessary detail on sample collection and management, key monitoring techniques and strategies for a range of ecological habitats and marine resources. Finally, the guide highlights the importance of data management, communication and reporting in order to enable effective dissemination from the programme. While these guidelines have been developed from a UK perspective the key aims, principles and many of the approaches are applicable to the implementation of marine monitoring plans for any spill. They aim to provide essential guidance in the event of a spill but should also be considered as an important source of reference for pre-planning and preparedness so that appropriate authorities can implement appropriate monitoring as effectively as possible.
Technical Report
Full-text available
This report focuses on the seabed habitat monitoring and assessments conducted by the UK Country Nature Conservation Bodies (CNCBs) during the period from 1999 - 2013 in inshore1 marine protected areas (MPAs), namely SSSIs, ASSIs, SACs, Ramsar sites and SPAs. It aims to identify: •how Common Standards Monitoring (CSM) Guidance has been implemented; •how existing monitoring could inform a UK-wide monitoring programme to address all marine biodiversity obligations; and •recommendations for future UK seabed habitat monitoring and assessments. The outputs of this report will help to inform the development of options for monitoring seabed habitats both within MPAs and the wider environment to be presented to the UK Government and the Devolved Administrations. The options will consider whether existing monitoring programmes are sufficient to meet our marine biodiversity obligations and management requirements or whether additional monitoring is needed.
Working document) Water Quality – Guidance standard on the design and analysis of interlaboratory comparison studies for ecological assessment
  • Cen Tc
CEN/TC 230/WG 2/TG 6: N 59 (Working document). Water Quality – Guidance standard on the design and analysis of interlaboratory comparison studies for ecological assessment. 8pp.
Water quality-Sampling of fish with multi-mesh gillnets
  • B S En
BS EN 14757:2005. Water quality-Sampling of fish with multi-mesh gillnets. 30pp.
Water quality-Guidance on the scope and selection of fish sampling methods
  • B S En
BS EN 14962:2006. Water quality-Guidance on the scope and selection of fish sampling methods. 28pp.
Water quality -Sampling -Part 1: Guidance on the design of sampling programmes and sampling techniques
  • Bs
  • Iso
BS EN ISO 5667-1:2006. Water quality -Sampling -Part 1: Guidance on the design of sampling programmes and sampling techniques. 42 pp.
Water quality -Sampling -Part 3: Guidance on the preservation and handling of water samples
  • Bs
  • Iso
BS EN ISO 5667-3:2003. Water quality -Sampling -Part 3: Guidance on the preservation and handling of water samples. 31 pp.
Biological monitoring: General guidelines for quality assurance
ICES, 2004. Biological monitoring: General guidelines for quality assurance. Ed. by H. Rees. ICES Techniques in Marine Environmental Sciences, No. 32. 44 pp.
Water quality -Part 19: Guidance on sampling in marine sediments
  • Bs
  • Iso
BS EN ISO 5667-19:2004. Water quality -Part 19: Guidance on sampling in marine sediments. 26 pp.
Water quality-Guidance on quantitative and qualitative investigations of marine phytoplankton
  • B S En
BS EN 15972:2009. Water quality-Guidance on quantitative and qualitative investigations of marine phytoplankton. 29 pp.
Guidelines for quality and/or environmental management systems auditing
  • Bs
  • Iso
BS EN ISO 19011:2002. Guidelines for quality and/or environmental management systems auditing. 32 pp.
Water quality -Guidance on marine biological surveys of hardsubstrate communities
  • En Iso
EN ISO 19493:2007. Water quality -Guidance on marine biological surveys of hardsubstrate communities. 32 pp. 10.5 General