Content uploaded by Claudia Magdalena Fabian
Author content
All content in this area was uploaded by Claudia Magdalena Fabian on Oct 04, 2024
Content may be subject to copyright.
Available via license: CC BY 4.0
Content may be subject to copyright.
Eur. Phys. J. Plus (2024) 139:868
https://doi.org/10.1140/epjp/s13360-024-05313-w
Review
Citizen science, accessibility, art & science, critical thinking, policy
and engagement: thoughts and lessons learned from the REINFORCE experience
Stylianos Angelidakis8, Theodore Avgitas12 , Emmanouil Chaniotakis4, Johanna Casado13 , Paschal Coyle15,
Gwenhaël de Wasseige5, Francesco Di Renzo12 , Claudia Magdalena Fabian6, Dimitrios Fassouliotis8,
Francesco Fidecaro2,3, Beatriz Garcia7, Gary Hemming1,a, Christine Kourkoumelis8,b,RémyLeBreton
17 ,
Jacques Marteau12 , Francesco Mureddu9, Vincenzo Napolano1, Francesco Osimanti10, Enzo Oukacha16 ,
Maria Panagopoulou4, James Pearson11 , Massimiliano Razzano2,3,c, Sofoklis Sotiriou4, Stephen Serjeant11 ,
Francesca Spagnuolo1, Elisabeth Unterfrauner6, Stylianos Vourakis14
1European Gravitational Observatory (EGO), Cascina, Pisa 56021, Italy
2Università di Pisa, Pisa 56127, Italy
3INFN, Sezione di Pisa, Pisa 56127, Italy
4Ellinogermanki Agogi, 15351 Attica, Pallini, Greece
5CP3, Institut de Mathématique et Physique, UCLouvain, Chemin du Cyclotron, 1348 Louvain-la-Neuve, Belgium
6ZSI–Zentrum für Soziale Innovation, Linke Wienzeile 246, 1150 Wien, Austria
7Instituto de Tecnologías en Detección y Astropartículas (CNEA, CONICET, UNSAM), Universidad Tecnológica Nacional, Facultad Regional Mendoza,
Mendoza, Argentina
8Department of Physics, National and Kapodistrian University of Athens, 157 84 Athens, Greece
9The Lisbon Council for Economic Competitiveness and Social Renewal, IPC-Résidence Palace, 155 rue de la loi, 1040 Brussels, Belgium
10 Trust-IT Srl, Via Francesco Redi 10, Apt. 11–12, 4th floor, Pisa 56124, Italy
11 School of Physical Sciences, The Open University, Walton Hall, Milton Keynes MK7 6AA, UK
12 Univ Lyon, Univ Claude Bernard Lyon 1, CNRS/IN2P3, IP2I Lyon, UMR 5822, 69622 Villeurbanne, France
13 Instituto de Tecnologías en Detección y Astropartículas (CNEA, CONICET, UNSAM), Universidad de Mendoza, Mendoza, Argentina
14 Institute for Accelerating Systems and Applications, University Campus, 10024 Athens, Greece
15 Centre de Physique des Particules de Marseille (CPPM), Marseille, France
16 Laboratoire Astroparticule et Cosmologie, 75013 Paris, France
17 Université Paris-Saclay, CEA, IRFU, 91191 Gif-sur-Yvette, France
Received: 23 January 2024 / Accepted: 24 May 2024
© The Author(s) 2024
Abstract REINFORCE (Research Infrastructures FOR Citizens in Europe) is a Research & Innovation Project, supported by the
European Union’s Horizon 2020 SwafS, ‘Science with and for Society’ work programme (GA872859). The project, which ran from
December, 2019, to November, 2022, engaged the public in a variety of innovative ways. Four citizen-science demonstrator projects
were developed on the world-leading Zooniverse platform, each focussing on a different area of frontier physics: gravitational waves;
neutrino astronomy; particle physics; and muography. A range of art and science events were launched and undertaken. A data-
sonification tool—sonoUno—was developed in order to improve the accessibility of the data used in the four demonstrator projects.
A course on critical thinking and a history of the Second Scientific Revolution was provided on YouTube and in podcast form,
while a senior-citizen-science course was designed, co-developed and implemented. These initiatives were supported by a detailed
engagement plan, a dedicated communications and dissemination strategy, and a constantly evolving assessment and evaluation
approach. The experiences garnered during the project, in conjunction with consultations with project participants, volunteers and
stakeholders, were built into the form of a policy roadmap explaining how to integrate citizen science into research infrastructures
in Europe. The roadmap identifies a series of policy objectives and related policy gaps, associated challenges and lays out a series
of recommendations. This article describes the results of the REINFORCE project and draws together the experiences of each of
the involved twelve partner organisations.
This article is dedicated to the memory of Stavros Katsanevas. Stavros was a brilliant scientist, professor and thinker. He was also the driving force behind
the REINFORCE project and was the Project Coordinator from its inception until the 27th of November, 2022, when he passed away; just three days before
the official conclusion of the project. Stavros was a friend to all in the collaboration and his open and inclusive nature can be felt, diffused throughout the
project as a whole.
ae-mail: gary.hemming@ego-gw.it (corresponding author)
be-mail: christine.kourkoumelis@cern.ch (corresponding author)
ce-mail: massimiliano.razzano@unipi.it (corresponding author)
0123456789().: V,-vol 123
868 Page 2 of 38 Eur. Phys. J. Plus (2024) 139:868
1 Introduction
Large research infrastructures working in the field of frontier physics continue to open new observational windows on to the universe
and to explore the structure of matter in ever greater detail. As these discoveries become more and more sophisticated, so the expertise
required in order to understand them also increases. As a consequence, the public are often only able to access and interpret the
results of research through the prism of outreach activities, and are thus not able to contribute directly to the development of new
scientific knowledge. This distance contributes to a gap between science on one hand and society on the other; a gap that can be
problematic in the context of public-funded scientific research.
REINFORCE has aimed to co-design, along with citizen scientists themselves, an approach that breaks down the barriers to
access that contribute to the widening of this gap and to build communities of practice around four demonstrator projects on the
Zooniverse platform. Volunteers in these communities have contributed directly to the production of scientific knowledge and have
worked closely with different research teams to classify and understand data subjects in both quantitative and qualitative ways.
The REINFORCE project pursued a set of key aims: engage people to contribute to online frontier science; create a community
of citizens that actively participates in scientific endeavours; introduce responsible research and innovation into the frontier citizen
science landscape; assess the impact of frontier citizen science in science and society; create a policy roadmap for the implementation
of citizen-science projects in large research infrastructures (LRI); and explore the potential of frontier citizen science for inclusion
and diversity.
To reach these goals, a series of initiatives were developed and implemented, ranging from Zooniverse demonstrator projects,
to art and science initiatives, via courses on critical thinking and science for senior citizens, to the development of dedicated data-
sonification tools and courses. All of these initiatives were encompassed within clearly-defined engagement and dissemination
strategies and a thorough and wide-ranging evaluative approach.
The REINFORCE Collaboration is formed of twelve different partners — Centre national de la recherche scientifique (CNRS),
CONICET, Ellinogermaniki Agogi, European Gravitational Observatory (EGO), Institute of Accelerating Systems and Applications
(IASA), Open University, Oxford University, The Lisbon Council for Economic Competitiveness and Social Renewal, Trust-IT
Services, Université catholique de Louvain and Università di Pisa — from seven different countries across two continents —
Austria, Belgium, France, Greece, Italy, the United Kingdom and Argentina. The European Gravitational Observatory took on the
role of Project Coordinator.
The project was funded within the EU Science with and for Society (SwafS) work programme of the Research & Innovation
Project thematic area. Dedicated to the integration of society in science and innovation issues, policies and activities, it promotes the
integration of the interests and values of citizens in these areas with the aim of increasing the quality, relevance, social acceptability
and sustainability of research. REINFORCE also focussed on supporting two specific United Nations Sustainable Development
Goals (4. Quality Education and 5. Gender Equality).1
By developing the work of the project through participatory design, involving all members of the project, from partners to
volunteers, to stakeholders in other fields, the aim was to build a committed, diverse community of practice. As the project evolved,
the experiences garnered across its life cycle, fed into the development of a policy roadmap, designed to signpost issues and provide
recommendations for best practice in terms of the integration of citizen science in LRI across Europe.
This article looks first at the demonstrator projects on Zooniverse themselves. For each of them, a description of the project is
provided, as well as an examination of the successes; what worked well and why. Interaction with volunteers and the demonstrator
project community are also covered, as well as the impacts of the engagement strategy and activity implemented for each of them.
The report on each of the projects concludes with a look at the lessons learned, at the areas that might be approached differently
and the outcomes and future prospects for the long-term exploitation of the project and its resources. Following the demonstrator
projects, the article explores the effectiveness of approaches to increase accessibility to scientific research within the project and the
ways in which the effectiveness of these were understood and how they ultimately fed into the development of the policy roadmap,
which stands as a keystone resource at the end of the project.
2 Demonstrator projects on Zooniverse
At the core of REINFORCE were four demonstrator projects developed on the Zooniverse online platform for people-powered
research.2On Zooniverse, volunteers contribute to research projects and participate in the study and classification of data. The ethos
behind the platform is that anyone can be a researcher; that working together can accelerate research and enable research that
otherwise would not be possible, as well as potentially leading to new discoveries.
1https://sdgs.un.org/goals.
2https://zooniverse.org/.
123
Eur. Phys. J. Plus (2024) 139:868 Page 3 of 38 868
Each of the four REINFORCE demonstrator projects centred on a different area of frontier physics. GWitchHunters3introduces
volunteers to the data produced by the Virgo gravitational-wave detector to characterise its noise sources and improve its sensitivity.4
New Particle Search at CERN5allows volunteers to work through different stages classifying displaced vertices; identifying the
signatures of known particles; searching for Higgs boson decays; and looking for long-lived particle decays, in data provided by the
ATLAS detector at the Large Hadron Collider.6In Deep Sea Explorers,7volunteers classify bioluminescence signals that constitute
noise in the KM3NeT neutrino detectors8and identify bioacoustic signals recorded by hydrophones located at the detector sites,
in order to provide information useful for the study of the surrounding marine environment. In Cosmic Muon Images,9volunteers
identify particle tracks as they passed through the different stages of a muon detector.
Memoranda of understanding between the REINFORCE collaboration and each of the individual collaborations providing data
for the demonstrator projects — Virgo, KM3Net and ATLAS — were agreed in the early stages of the project, which were all
initially developed only in English. Following the launch of each of them, however, it was agreed to also implement alternative
language versions, in order to render them more accessible and to better facilitate their dissemination across target groups and to
contribute to the engagement targets of the project. At time of writing, GWitchHunters is available in both English and Italian, with
the Spanish version due to go online shortly, while New Particle Search at CERN is available in English, Greek and Spanish.
2.1 GWitchHunters
The GWitchHunters project [1] focuses on supporting research in the field of gravitational waves through the contribution of citizen
scientists. Citizens are asked to contribute to the study and characterisation of the data recorded by the Advanced Virgo interferometer
[2]. Virgo is the main gravitational-wave detector in Europe and, together with Advanced LIGO [3] and KAGRA [4], forms part of
an international network of experiments for gravitational-wave detection. These detectors are extremely sensitive machines, and, as
such, need to be shielded as much as possible from potential sources of noise, which can produce spurious signals in the detector data
that can be much larger than those expected from waves produced by astrophysical sources. It is therefore of paramount importance
to properly characterise noise artefacts and to mitigate and possibly remove them. In particular, rapid transient noise events, known as
glitches, are particularly detrimental for the detectors, since they can affect the duty cycle and can mask or mimic real astrophysical
signals.
The characterisation of glitches is not trivial, however, as they appear arbitrarily in time and can exhibit very different types of
behaviour. When plotted on a time-frequency map representing the time evolution of their spectral energy content (spectrogram),
glitches can show diverse morphologies, and it is possible to group them into families.
This approach is based on the premise that, at the origin of the various glitch families, there are also distinct noise sources;
identifying all the times when some of these sources manifest is the first step in further pursuing their identification for the
subsequent implementation of mitigation strategies. Given the high rate of these glitches, and their large number over entire data-
taking periods, studying them one by one can be very time-consuming. Machine learning (ML) is therefore of great help in their
rapid identification, as it has the potential to automatically classify glitches on the basis of their individual spectrogram. As has been
shown in different works, e.g. Zevin et al. [5], Razzano and Cuoco [6], convolutional neural networks are very effective at the task
of glitch classification and can form the basis for an automatic classification pipeline. Additionally, considering the complexity of
their morphology, Artificial Intelligence algorithms are proven to provide precious support in these classification tasks by noticing
details and similarities over very large datasets that the human eye would struggle to identify.
GWitchHunters volunteers provide classification information that can be used to train automatic classification pipelines. The
project has aimed to expand the contribution of citizen scientists to tasks beyond a simple classification, in order to provide more
complex and diverse data and to allow them to dig deeper in the investigation of the causes of the noise in the detector. In particular,
volunteers choose from a variety of workflows with different degrees of difficulty. The first workflow comprises a simple classification
of the shapes that appear in the spectrograms of the strain channel, the main channel in which gravitational-wave signals are detected,
choosing from a limited number of glitch families (Fig. 1). In the second workflow, volunteers identify the glitches drawing one or
more rectangles around the spectrogram regions exhibiting some excess of energy, and then perform their classification as in the
previous workflow. In the third and more complex workflow, volunteers are asked to compare the shapes of the glitches appearing in
strain channel with those of the so-called auxiliary channels, i.e. data coming from sensors that constantly monitor the detector parts
and their physical environment (Fig. 2). The project has been officially launched on the Zooniverse platform the 16th of November,
2021, and also provides a set of mobile workflows, i.e. tasks that can easily be performed on mobile devices.
3https://www.zooniverse.org/projects/reinforce/gwitchhunters.
4https://virgo-gw.eu/.
5https://www.zooniverse.org/projects/reinforce/new-particle-search-at-cern/.
6https://atlas.cern/Discover/Detector.
7https://www.zooniverse.org/projects/reinforce/deep-sea-explorers.
8https://www.km3net.org/.
9https://www.zooniverse.org/projects/reinforce/cosmic-muon-images.
123
868 Page 4 of 38 Eur. Phys. J. Plus (2024) 139:868
Fig. 1 Identifying a glitch family in GWitchHunters
Fig. 2 Looking for correlations between Virgo auxiliary channels and a glitch in the strain channel. The glitch morphology is superimposed on to each of
the auxiliary channel spectrograms
During the lifetime of REINFORCE, the GWitchHunters project obtained considerable success, both in terms of classifications
and feedback from volunteers. To date, more than 4,600 individual volunteers have participated in the project, undertaking more
than 700,000 classifications of 41,000 data subjects. These numbers are in broad alignment with the expectations for the project
123
Eur. Phys. J. Plus (2024) 139:868 Page 5 of 38 868
that were built on the basis of another Zooniverse project available on Zooniverse: Gravity Spy.10 Volunteers have proven to be very
enthusiastic; often going beyond the more basic tasks. In particular, they were also asked to look for glitches that were not in the
initial set of available glitch families. This aspect is very important in terms of discovering unknown noise sources, an aspect that
is difficult to solve using ML only. Another aspect that has been particularly successful, has been the availability of translations
in languages other than English. In order to test this aspect, a complete translation of the project into Italian was prepared, and
a dedicated press event was held, in order to properly present the project to the volunteers in Italy. The results have been very
interesting, showing that this approach can be very useful in promoting access to the project beyond only people who speak English
to at least some degree.
The GWitchHunters classification results, using data from the O3 data-taking run, have been input to a ML algorithm built on
a convolutional neural network (CNN) [1]. The performance of the ML has been compared to that of citizens, and the results have
shown that this approach, based on citizen science, is promising, not just in terms of preparing datasets for the training of ML
algorithms, but also in terms of contributing actively to the research in general. New datasets are in preparation, also taking into
consideration the upcoming observational campaign — O4 — and it is expected that the contribution of volunteers will become more
and more important. There are also other plans to further optimise the way in which the data subjects are presented to volunteers,
as well to improve the levels adaptation for mobile devices. It is also hoped that introduction of the Spanish-language version of the
project will help to broaden the participant base further.
2.1.1 Sonification of gravitational-wave data
The spectrogram images are a valuable tool for representing gravitational-wave detector data, particularly for the general public,
without signal analysis expertise. To make this kind of data representation even more inclusive, the GWitchHunters project has
developed a dedicated sonification algorithm to convert the images into sounds. At the base of this algorithm there is the association
of frequencies with musical notes, and the signal energy with the notes intensities, as we shall describe with the following example.
Let’s consider the first detection event of a gravitational-wave signal from the coalescence of a pair of black holes, GW150914 [7].
Its spectrogram is shown in the left-hand side part of Fig. 3. The vertical axis of the spectrograms, representing the frequency, has
been divided into intervals determined by the frequencies of the notes of the C-major scale of occidental music, as shown in the
image on the right-hand side part of the figure. This choice, although arbitrary, aligns with the white piano keys and helps convey
the meaning of different frequencies.
The energy in each band corresponds to the intensity of the note, and its evolution can be represented as a sequence of notes,
resulting in the conversion of the spectrogram into a musical score and, in turn, into a melody. No need for musical knowledge is
required to enjoy the latter and to let people able to distinguish different spectrogram morphologies as well as different melodies.
Additional data transformations can be implemented. For example, the gravitational-wave signal from the coalescence of a binary
black hole system is characterised by a duration of a few tenths of seconds and frequencies up to a few hundred Hz. These times are
too fast to appreciate the details of the signal evolution but, once converted to a musical score, the execution can be slowed down at
will.
Similarly, the frequencies of this signal are a bit too low to be felt as enjoyable by the human ear. However, they can easily be
shifted to a higher pitch by transposing the musical score by one octave (that is, doubling the frequencies) to match the human ear
sensitivity range better.
The multisensorial representation of gravitational-wave data, in the form of images and sounds, fosters inclusiveness and increases
the reach of the project by making the data enjoyable to a more vast public. Most importantly, it facilitates the engagement of
individuals with visual impairments in both gravitational-wave research and the exploration of the cosmos.
2.2 New Particle Search at CERN
The New Particle Search at CERN11 demonstrator project engages citizens in the state-of-the-art particle research performed at the
Large Hadron Collider (LHC) of CERN, in the quest for the understanding of the ultimate structure of matter. The demonstrator is
based on data collected by the ATLAS experiment12 which are produced by high-energy proton-proton collisions at the LHC.
Volunteers perform a visual inspection of data samples consisting of events, namely the registered products of proton-proton
collisions. In this way, they contribute to the search for yet undiscovered hypothetical particles predicted by theories Beyond the
Standard Model (BSM) [8]. Their search could lead to a discovery, which would be a direct proof of new physics and would highlight
a path for future research.
To enable the volunteers in their work, a three-stage architecture was adopted in the demonstrator project. The first two stages are
based on selected samples of simulated data and are used to train volunteers, but also to allow for a quantitative assessment of their
performance and a comparison with specially developed automated algorithms. The third stage of the demonstrator is a discovery
10 https://www.zooniverse.org/projects/zooniverse/gravity-spy.
11 https://www.zooniverse.org/projects/reinforce/new-particle-search-at-cern/.
12 https://atlas.cern/.
123
868 Page 6 of 38 Eur. Phys. J. Plus (2024) 139:868
Fig. 3 Alternative spectrogram representations of the GW150914 gravitational-wave signal as observed by the LIGO Hanford detector. aUsual time–fre-
quency representation of the energy of the signal, as in the original article [7]. bAlternative representation where the energy corresponding to the different
frequency bands is represented by the series of curve. The higher the curve, the greater the intensity of the corresponding musical note
Fig. 4 The first stage of New Particle Search at CERN on Zooniverse
stage, employing datasets of real events from the ATLAS Open Data Set Release [9]. The third stage provides two research paths:
(a) study of Higgs boson decays to two photons, one of which could be converted to an electron-positron pair by interaction with
detector material; and (b) search for yet undiscovered neutral long-lived particles, predicted by certain theories of the BSM. The
volunteer research involves the identification of specific signatures, which are produced from the decay of these new long-lived
particles. These decay products could originate from displaced vertices (DVs), namely vertices — formed by two or more tracks —
that are displaced with respect to the main collision point of the two protons.
The demonstrator project provides visual analysis tools that allow the citizens not only to classify static images — in order
to recognise the DVs — but also to interact with the event displays, select specific tracks and calculate kinematical quantities
characteristic of the sought-after particles.
In Stage 1, which is hosted entirely on Zooniverse, volunteers are trained to recognise DVs in a high-purity sample of simulated
data, corresponding to the various scenarios of new particles with displaced vertices. The volunteers only inspect stationary images
of the traces that charged particles leave in the inner part of the ATLAS detector. They look for tracks that intersect at a point other
than the main interaction point and need to inspect both views of the inner detector in order to be able to properly recognise track
intersection. In the two different projections of the inner detector, which are depicted in Fig. 4, the tracks are given different colours,
so that a user may identify the same track in both views. The volunteers are then asked to spot the DVs in both views, and the answer
is internally assessed by Zooniverse, based on the truth information which is also provided to the platform.
The quality of the volunteers’ work is assessed and compared against a specially developed automated algorithm, which detects
the presence of displaced vertices in the events. The algorithm extrapolates the tracks and looks for their intersection, using only
123
Eur. Phys. J. Plus (2024) 139:868 Page 7 of 38 868
Fig. 5 The second stage of new particle search at CERN on HYPATIA
detector information that is also available to the volunteers. The overall goal was to compare the DV-identification efficiency of
the volunteers to that achieved by the algorithm. The efficiency is defined as the fraction of long-lived particle decays that were
successfully identified by the volunteers. The results, from 180,000 classifications provided by the users, were analysed in multiple
ways. The most interesting results were obtained by considering the user consensus as an identification criterion. The identification
efficiency of the user consensus is found to be about 93% on average, depending on the projection of the ATLAS detector provided in
the image (89% in the transverse detector view, 96% in the longitudinal detector view). This efficiency is very close to the respective
identification efficiency obtained by the dedicated automated algorithm, which was 94%.
For Stages 2 and 3, the users are directed from Zooniverse to the HYPATIA [10] event display. Instead of simply examining static
images and locating DV they interact with the event display in order to perform in-depth analysis of the events. In the context of the
demonstrator project, significant additions were made to the HYPATIA platform, in order to incorporate the functionality necessary
to enable the volunteers to perform their analysis in each stage of the project. In addition, a new event format was developed to
include additional information, such as the display of DV.
In Stage 2 volunteers are asked to identify certain particle types that are useful for the next discovery stage. They look for
signatures of muons, electrons, photons and converted photons. The HYPATIA particle information table, shown in Fig. 5,displays
momentum, charge and direction information for each track or cluster of the event. The volunteer can identify tracks and clusters,
using the button that corresponds to the type of track or cluster to which they have determined it belongs (electron, muon, photon
or converted photon). When the volunteer clicks on the Next button to display another event, the selections made are stored in a
back-end database for later processing. Since the sample consists of simulated events, the particle generation identification is already
known and thus can be compared to the user classification, in order to determine the validity of the choice made. The volunteers
made a total of 37,000 classifications in this way, with 80.4% of them being correct.
The volunteer output was also compared to an ML algorithm, which was developed based on Boosted Decision Tree classifiers,
and which uses exactly the same information that is available to the volunteers, in order that this comparison be fair and impartial.
The comparison shows that the identification ability of the volunteer cannot be better than a dedicated ML algorithm, but, in the
cases of the electron and photon, the user identification efficiency is not very far from it.
In the Higgs boson stage, the volunteer can indicate that a cluster belongs to either a photon or a converted photon that could
originate from Higgs boson decays. HYPATIA automatically calculates the invariant mass when the volunteer selects a pair of
photons. In addition, the volunteer can rate the event (from one to five stars) based on the instructions given on the platform. In this
study, users selected 94% of all photon-pair masses in the 106–160 GeV mass range. Distinguishing the Higgs boson signal, which
lies on top of the background, requires a much larger dataset than that available to the volunteers and a sophisticated statistical
analysis of the data. The volunteers rated 1156 events as being worthy of five stars, with most events receiving low star-ratings as
they did not contain unusual Higgs candidate decays. Of those 1156 five-star-rated events, only a few contain extra leptons, but
the invariant mass of the photons (or converted photons) is outside the mass range of the Higgs boson mass, therefore those events
cannot be attributed to complex Higgs boson production and decay mechanisms.
123
868 Page 8 of 38 Eur. Phys. J. Plus (2024) 139:868
In the Long-Lived-Particle (LLP) Hunting part of the demonstrator project, volunteers search for long-lived particles predicted
by some BSM models, through the identification of secondary vertices of particle decays. Following the vertex identification, they
were asked to look for muons originating from the displaced vertex and to examine certain kinematic variables, which could be
characteristic of a candidate long-lived particle. Volunteers were also asked to rate each event from one to five stars, according to
how similar they believed it to be to one of the sought-after decays. A total of 81,894 classifications were performed on a sample
consisting of 2440 events. Events which satisfy all given requirements have a much higher probability of originating from a new
long-lived particle. Since these particles are as yet undiscovered, the relevant candidates are extremely rare. The volunteers only
viewed 27 of them (volunteers viewed each one multiple times) and correctly marked with five stars 23 of those events. Two of the
five-star events were good candidates for new BSM long-lived particles (hypothetical super symmetric particles). The events were
scrutinised by our team and further information (which was not available to the volunteers) was inspected. It turns out that, most
probably, both events are due to the expected background. A number of volunteers did isolate 24 events with more than one muon
in the DV and posted their findings in the project Talk forum, where they were discussed by the wider community. After further
investigation by the research team, it was revealed that these events were either due to the interactions of a known particle with the
detector or faults in the reconstruction of particle tracks.
2.2.1 Interaction with users, engagement and impacts
From the launch of New Particle Search at CERN as an official Zooniverse project on the 19th of October, 2021, until the end of the
REINFORCE project, on the 30th of November, 2022, the project research team received and replied to 2,852 Talk messages (seven
messages per day on average) posted by volunteers on the project forums available on Zooniverse.13 Volunteers communicated
technical questions regarding the stages of the project (especially during its first few months), questions on physics related (or not)
to the tasks provided to them, their results and observations, as well as interesting suggestions and features they would like to see
added. The members of the scientific research group were delighted to respond to volunteer posts on a daily basis, but also motivated
discussion between volunteers, while assigning the role of expert to the most talented among them. Furthermore, in collaboration
with the Ellinogermaniki Agogi (EA) team, various activities during the course of the project were prepared, which served to ensure
the motivation of volunteers remained high. These included:
•Awarding hard-copy certificates to interested volunteers who had completed over 200 classifications with over 50% efficiency,
acknowledging their citizen-science contribution;
•Meeting with volunteers in person at the REINFORCE Summer School, 2022, in Marathon, Greece.14 During this summer school,
volunteers were presented with the ongoing research and the latest achievements in the field of high-energy physics, and had the
opportunity to work together with our scientific team;
•Four challenges (Winter Challenge 2022,15 Easter Challenge 2022,16 Challenge for Greek teachers,17 EPS competition18) during
which the performance of each volunteer was closely monitored. At the end of each challenge, the most successful volunteers
were awarded prizes, with the results announced on the Zooniverse forums;
•Online meetings with volunteers, in which their questions and ideas were discussed, as well as virtual visits to LHC experiments
(ATLAS and ALICE). These activities maintained the engagement of citizens in New Particle Search at CERN and volunteers
ended up contributing an impressive 179,887 classifications (441 classifications per day on average) to Stage 1 of the project alone.
According to the data gathered by the Zooniverse platform, volunteers spent about five minutes per classification, on average.
In conjunction with the work undertaken by the REINFORCE engagement team, the New Particle Search at CERN team also
developed a set of feedback metrics and mechanisms, which supported volunteers during their early encounters with the projects.
During the final months of the REINFORCE project, all of the New Particle Search at CERN materials were translated into Greek,
while the REINFORCE partner CONICET also provided a Spanish translation. All three language versions are now available on
the project’s Zooniverse page. Furthermore, data on the particle identification from the second stage was provided to the CONICET
team in order to develop a way to represent the different particle types with distinct sounds. Sound files from a small number of
events were generated as part of this effort and tested during different dissemination events. It was determined that the second stage
of the demonstrator project is the only part of it that can be sonified, as the other two are too visually complex and don’t lend
themselves to auditory representation.
In addition to motivating volunteers, the scientific community was also motivated towards citizen science through presentations
of the project at six international scientific conferences, with contributions to the respective conference proceedings.
13 https://www.zooniverse.org/projects/reinforce/new-particle-search-at-cern/talk.
14 https://reinforce.ea.gr/international-training-course/.
15 https://reinforce.ea.gr/winter-challenge/.
16 https://reinforce.ea.gr/easter-challenge/.
17 https://reinforce.ea.gr/therinos-diagonismos/.
18 https://reinforce.ea.gr/eps-citizen-science-competition/.
123
Eur. Phys. J. Plus (2024) 139:868 Page 9 of 38 868
2.2.2 Outcomes and the future
The final goal of the discovery stages is to have thousands of citizen scientists rate each event with 1–5 stars, according to how
similar they believe them to be to one of the new long-lived particle decays. In this way, the volunteers are directly involved in
potential discoveries, without requiring any prior knowledge of high-level physics or computing skills. Furthermore, they contribute
to a frontier research science topic, performed at the largest particle physics facility in the world.
Finally, a large number of dedicated volunteers left very informative and useful comments in the Talk forum during the classi-
fications of the two discovery stages. Overall, New Particle Search at CERN should be considered a success, as it managed to get
a large number of volunteers involved in complicated analysis of high-energy-physics signatures collected by the state-of-the art
ATLAS detector.
The REINFORCE project ended at the end of 2022, but the demonstrator project remains open and available on the Zooniverse
platform and volunteers continue to classify events. When a new ATLAS Open Data release is available, together with other resources,
new, fresh data will be added.
2.3 Deep Sea Explorers
The Deep Sea Explorers19 demonstrator project takes place in the context of the KM3NeT experiment,20 a neutrino telescope
currently being deployed in the Mediterranean Sea. While the main focus of the KM3NeT Collaboration is on the search for
neutrinos, Deep Sea Explorers volunteers were asked to study events caused by sea mammals and benthic fauna.
KM3NeT is equipped with light sensors to detect Cherenkov light produced by a neutrino interaction in the sea, as well as
hydrophones that are used to calibrate the instrument as it moves with the sea currents. In addition to being able to detect elementary
particles crossing the detector, KM3NeT is also sensitive to its environment. Volunteers were therefore asked to classify the various
signatures of light and acoustic noise produced in this environment. The aim of these classifications was to help to better understand,
on one hand, KM3NeT data and the response of the instrument, and, on the other, the presence of life in the deep sea in which the
detector is deployed.
In the bioluminescence workflow, volunteers were asked to classify data subjects and determine the number of peaks visible within
them. In the bioacoustics workflow, they were asked to study different visual and audio representations of data subjects recorded
by the hydrophones around the detectors and to determine whether they signalled the presence of sperm whales, short-finned pilot
whales, were just pure noise or were likely something else.
As the KM3NeT Collaboration is an experiment that is still in the deployment phase, it was decided to split the acquisition of the
classifications performed by citizen scientists into two separate phases. In addition to giving the scientist team more time to acquire
additional raw data to be classified by the participants, this also created a motivating feature for the citizens. The first phase of the
demonstrator ended in February, 2022, after the launch of the project earlier that year. The first classifications were analysed and
the results, presented in the next section, were presented to the Deep Sea Explorers participants before the start of Phase 2.
A much larger set of events, a factor 10 and 20 for the bioluminescence and bioacoustics workflows, respectively, was created
for Phase 2 and uploaded onto the Zooniverse platform a month later. Both workflows, despite not being fully completed, made it
possible to acquire sufficient data to carry out a comparative study.
2.3.1 Successes: what worked and why
Deep Sea Explorers proved to be a real success. It has been possible to demonstrate that the help of citizen scientists improves the
output of a classifier when labelled datasets are available, i.e. in the case of the bioacoustics workflow. In the case of both workflows,
sufficient trained classifiers were obtained to now be used on the data collected in KM3NeT over its entire lifetime.
After the first phase of the project, it became possible to clearly identify some events as belonging to one of the four categories
made available to volunteers on Zooniverse. As an example, the two events shown in Fig. 6have been classified as being one peak
events with a 90% and 96% certainty (i.e., this is how often they were classified in this category by the participants). On the other
hand, some events have not been classified unanimously by the participants, such as that displayed in Fig. 7.
The goal of the project was to obtain a set of very well classified data subjects. Despite some events already fulfilling the
requirements to reach this objective, more events were needed for Phase 2. In this second phase, the aim was to provide new subjects
so that the number of events that are classified with a high probability in only one category could be increased. Phase 2 therefore,
had many more events than Phase 1, in order to significantly increase the data available for the final statistics.
At the end of Phase 2, the events of the bioacoustics workflow were used to train a convolutional neural network (CNN), previously
developed using data from hydrophones located at the surface. While the events used in the demonstrator, which were originally
classified with less than 70% accuracy by the already existing neural network, were used to train the new CNN and led to a precision
in the classification of sperm whale and short finned pilot whale of 90% and 100%, respectively.
19 https://www.zooniverse.org/projects/reinforce/deep-sea-explorers.
20 https://km3net.org/.
123
868 Page 10 of 38 Eur. Phys. J. Plus (2024) 139:868
Fig. 6 Bioluminescence events
from Phase 1 that were identified
as being one peak events by the
citizen scientists
We concluded that the data collected from the volunteer classifications provided through the demonstrator project can help us to
enhance the accuracy of classification of KM3NeT events in comparison to the original neural network.
2.3.2 Areas that might be approached differently
The involvement of citizen scientists in active research projects taking place within the KM3NeT collaboration was a real success
with concrete scientific results as proof. However, the personpower required for such a project was misevaluated by the Deep Sea
Explorer team. As the volunteers were discovering new research fields, moving between neutrino physics and astronomy to marine
biology, detailed explanations and continuous support from the science team was needed. The Talk forum, which forms part of every
Zooniverse project, allows for an easy and direct interaction between the science team and the citizen scientists, but, as with any
forum, an active and constant engagement in the discussion takes time.
123
Eur. Phys. J. Plus (2024) 139:868 Page 11 of 38 868
Fig. 7 Example of an event that was classified as a short-finned pilot whale (28%), sperm whale (35%), or pure noise (24%)
2.3.3 Lessons learned
Deep Sea Explorers results demonstrated to the science team, as well as to a considerable fraction of the particle physics community,
the substantial gains that citizen science can bring to event classification. As other projects are added to Zooniverse, an important
characteristic of them might be to keep in mind the time and investment required to properly nurture, grow and support a project of
this kind. The Deep Sea Explorers science team will be developing more projects of this nature and will aim to maintain a support
team that is at least twice the size of the team used for this project.
2.3.4 Interaction with the demonstrator project community
Besides successfully achieving the classification goal, the Deep Sea Explorer volunteer community caused a considered reflection
within the research team on how scientific wording can be perceived. The original name of the demonstrator project was due to
be Deep Sea Hunters, as researchers in the KM3NeT Collaboration often called themselves Neutrino Hunters. During beta testing,
however, the Zooniverse community pointed out that the word hunter could have a negative connotation when associated to marine
fauna and could discourage people to take part in the project. The demonstrator was duly renamed as a result of this advice and
became the Deep Sea Explorers project that ultimately went to official launch.
2.3.5 The future: next steps; short-, mid- and long-term exploitation of the project and its resources
This project has led some members to ask for the help of citizen scientists in relation also to other projects, such as, for example, in
the classification of sub-threshold events recorded by the IceCube Neutrino Observatory, another neutrino telescope located at the
South Pole. While no bioluminescence activity is expected in the ice, the detector itself creates patterns in the data that are not well
modelled and for which the help of citizen scientists in their classification would be useful. This would constitute a new project on
the Zooniverse platform that will be created using the know-how acquired during the development of Deep Sea Explorers.
123
868 Page 12 of 38 Eur. Phys. J. Plus (2024) 139:868
The science team also created fruitful ground for interdisciplinary projects and collaboration with bio-informaticians and biol-
ogists. This collaboration will continue after the end of REINFORCE and throughout the lifetime of KM3NeT. Several grant
applications to hire PhD students and postdocs in this framework have been submitted. The help of citizen scientists to disentangle
neutrinos from the signal sent by living species at the bottom of the sea will be one of the methodologies used in these projects.
2.4 Cosmic Muon Images
The Cosmic Muon Images21 demonstrator project is based on muon tomography [11], an imaging technique to study the internal
density distribution of massive objects. Muon tomography uses muons produced in cosmic ray atmospheric showers as a probe for
inferring the density anisotropies of geological, archaeological and civil engineering objects like volcanoes, pyramids and tumuli.
The principle behind this technique is similar to that behind a medical X-ray; a particle detector is placed downstream of an
object with respect to the incoming particle flux and takes measurements for an extended period of time. The angular profile of the
reconstructed muon tracks is then compared to the theoretical expectation, taking into account the external geometric characteristics
of the object as well as some basic assumptions on its composition. Combining all this information provides an image of the internal
density distribution for the object under study.
The method was first used by Alvarez to search for voids inside the pyramid of Chephren at the Giza plateau, back in the 1960s
[12]. It was proposed for volcanology in the mid-90 s [13] and reached its modern form in the mid-2000s [14]. Since then, the domain
of study has rapidly extended into new fields, such as: the prediction of natural hazards, which includes monitoring for landslides,
lahars and other similar mass displacements; the absorption/retention and distribution of rain water in underground repositories
and aquifers; the investigation of pyramids for unknown voids, as well as other sites of archaeological interest, such as tumuli; the
monitoring of industrial facilities like blast furnaces or nuclear waste disposal containers; and, last but not least, homeland security
applications focused on the detection of nuclear contraband within transportation containers [15].
Since muon tomography brings together particle physics, cosmic ray physics and a diverse conglomerate of scientific fields it
serves as a concrete example of how particle physics can leave the constraints of the laboratory, become mobile and address issues
that affect our everyday lives in more immediate and palpable ways.
The Cosmic Muon Images project focuses on the analysis of data from an archaeological muon tomography expedition that took
place during the summer of 2018 in Greece at the ancient tumulus of Apollonia in Khalkidhiki, which hosts an ancient Macedonian
tomb [16]. In the context of REINFORCE, all data acquired were shaped into a Structured Query Language (SQL) database, which
acted as a repository for plot creation and which were then uploaded to the Zooniverse platform, in order to allow citizen scientists
to inspect and analyse [17].
The goal of the analysis was to identify track-like topologies that had been missed by the algorithm used by the research team.
The extreme horizontality and the closeness of the detector to the studied object made the signal-to-noise ratio for this experiment
a factor that could potentially be ameliorated through the input of the volunteers.
The track-selection algorithm generally used for volcanology data is based on a series of empirical selection criteria that mostly
favour low-multiplicity events, which triggered a low number of detector channels, i.e., fewer than 100 out of the 2,304 possible
combinations. This provides a strict sample of muon tracks for the tomography and rejects events with complicated topologies as
background, even though a track (or more) might be hidden among these busy topologies.
These events were separated into two samples and were then provided to the volunteers in the form of detector-representation
plots, with the possible particle strikes depicted on the detector surfaces. The task at hand was the identification of tracks crossing
the detector and extra particle strikes on the three detector planes.
The first sample comprised simpler events, for which our algorithm could provide a candidate track, while the second contained
events for which no reconstructed tracks were retrieved. The first sample was classified by the volunteers within the Introductory
workflow, which also served as a stepping stone to becoming familiar with the event categorisation procedure. The second, more
complicated sample, was incorporated in the FreesStyle workflow [18].
Both workflows ask volunteers to draw a pair of lines, representing a track, passing through the detector channels that showed
a charge and mark up extra particle strikes, by pairing detector channels that did not participate in the formation of the line. The
difference being that for the Introductory workflow, a pair of lines is already present (representing a particle track) and the volunteers
need to decide if these are valid, if they need to be redrawn properly or if they are just an artifact of our algorithm and the event
should be categorised as background. The logic for the FreeStyle workflow is the same, with the topologies being more complicated
and the volunteers having to point out if it is clearly a background event or if there are tracks that might be identified. For both
workflows there is an extra step after the basic task of identifying the tracks, which is the highlighting of extra particle strikes.
2.4.1 Successes: what has worked and why
Towards the end of the project the volunteers had analysed 9,099 events for the Introductory workflow, having performed 48,577
classifications. Of these, 5,765 events were identified as signal and 3,344 events as pure background. This was an interesting finding,
21 https://www.zooniverse.org/projects/reinforce/cosmic-muon-images.
123
Eur. Phys. J. Plus (2024) 139:868 Page 13 of 38 868
as the algorithm used by the research team would not be expected to reach such a high percentage (30%) of false positives. This is
the first useful result provided by the inspection of the events, and it needs to be cross-checked with appropriate simulations to see
if and why it holds true.
Trying out different ML algorithms from the scikit-learn22 library, the Gradient Boosted Trees algorithm was identified as optimal
for signal-to-noise discrimination. The unblinding of the data showed that this method could potentially provide a better muon sample
for muon tomography imaging, with 10% less noise than the initial sample. The impact of this selection on the muon tomography
result remains to be studied [19].
2.4.2 Improvements; what has not worked and why
The FreeStyle workflow did not yield a similar result. The first cause for this was the complexity of the events combined with the
small number of classified data that did not provide the ML algorithm with enough clear-cut information on which to perform. The
overall participation of the volunteers for this workflow was comparable to the Introductory workflow, with 46,866 classifications
of a dataset of 8,529 subjects, so the failure to train likely relates to the task itself and its origin is more objective than subjective
[19].
The second issue originated from the research team side and the design of the workflow. The fact that there were many degrees of
freedom through which the volunteers could navigate the classifications led to diverse realisations as to what might be background
or not. Some completionists would try to draw all possible tracks they could see, no matter the validity of the physical interpretation,
while others would follow a lazier approach and reject events as background that would otherwise prove to have a track after careful
examination. The complicated topology of a subject would only make things worse, leading to a weak consensus. The heuristic
approach that the research team aspired to was eventually not suitable in the framework of the demonstrator project.
2.4.3 Lessons learned
A key takeaway from the Cosmic Muon Images project is that citizen science projects can provide help even for tasks that at a first
glance might seem simple or mundane. That being said, workflows need to be designed with care and attention and an extended period
for tests and preliminary analyses of the classifications, in order to catch early on any potential misunderstandings and confusion
that might arise when non-experts try to grasp and implement concepts that experts perceive as given and trivial. In retrospect, a
gradual involvement of audiences with decreasing levels of scientific literacy would be beneficial. Maybe going from university
students to other audiences that can function under guidance, such as amateur astronomers, geologists etc., would be beneficial for
the formation of the project, before it reaches its final form and becomes available to the general public.
This proposed strategy springs from the fact that a workflow needs to remain unchanged throughout its lifetime, so that all data
are analysed by the volunteers in a uniform way. The Zooniverse staff, being experienced with this aspect, have similar procedures
in terms of beta testing by volunteers, but the focus is more the optimisation of the tools rather than the scientific end product and
its usefulness. This gradual opening to audiences of different scientific expertise should help with the latter aspect.
Concerning the FreeStyle workflow dataset, it is possible in retrospect to say that it would have been more successful if an
unsupervised ML algorithm had initially been run over it, in order to have caught some basic patterns. It would then have been
possible to identify which of them made physical sense, in terms of tracks and extra particles. In this manner, it would have been
easier to guide and better constrain volunteer classifications. With this method we could have better limited the data sample to
only those events that could have a physical interpretation, rather than leaving this task to the volunteers. This strategy could have
improved the consensus and would have decreased the number of classifications needed per subject.
The first step to remedy these drawbacks would be to re-design the FreeStyle workflow, providing better guidance and a more
constrained path for the classification procedure. It would also be useful to remove events with extreme multiplicities, as, following
discussion with the volunteers, it is clear that these can be a factor of confusion.
2.4.4 Interaction with the demonstrator project community
The interaction with the volunteers has proven to be very useful. Since the start of the project, it has been possible to identify
two kinds of participant: those interested in the actual categorisation process; and those wanting to discuss the instructions, guides
and the other demonstrator materials. Most commonly, people have asked if a specific categorisation that they have done is right.
Answering these questions led gradually to the establishment of a database of examples with proper categorisations. The answers
from the research team clarified why something would work and how certain choices would have made more sense than others,
taking into account the detector operation and the processes involved in the production of the events.
Some volunteers were more inclined to address appearance and form issues relating to the demonstrator project. The research
team was provided with a full list of spelling and grammatical mistakes that had been missed during the writing and correcting of
the project materials, while others offered ideas on how the event depiction of the workflows could have a better, more intuitive
22 https://scikit-learn.org/.
123
868 Page 14 of 38 Eur. Phys. J. Plus (2024) 139:868
representation. This feedback was incorporated into the field guide section of the project, as implementing changes directly on the
workflows after their launch is discouraged.
Interaction with the volunteers also showed that many of them choose to go directly to the workflows and start performing
classifications right away. They would come back to the research team for help only when they realised that their intuition alone,
without going through our explanations, led to inconsistencies. As the project progressed and more answers for issues accumulated,
a gradual decrease of this kind of question was observed.
At the start of the project, the research team aspired to create a small team of volunteers willing to delve deeper into muon
tomography, but it quickly became clear that even the most dedicated individuals only wanted to help through the Zooniverse
platform and nothing more. Another aspect that we found, was that uploading data in small bunches of 5,000 subjects per upload
created some frustration among the volunteers. Most of them wanted the entire dataset to have been uploaded in one go, so that
they would be able to have a clear time horizon for the completion of the project, rather than watch the progress bar reach the 100%
mark, signifying that the dataset had been completely classified, only for it to go down again. It is difficult to say to what extent this
affected the churn of the project, but it is clear that the practice was not received well and should not be repeated. Overall, feedback
proved very useful and the communication with the volunteers serves now as a guide, with which to shape future developments of
the project.
2.4.5 The future
The demonstrator has reached almost 170,000 classifications over a total of 13,626 subjects for the Introductory workflow and 12,867
subjects for the FreeStyle workflow, adding, respectively, 4,257 and 4,338 classified subjects to the numbers mentioned previously.
It is now time to automate the work of volunteers for the Introductory workflow by means of a suitable neural network architecture.
In the case that the current number of subjects is not enough for this task, the research team will need to explain the new course of
its work and enrich the workflow with more data.
The FreeStyle workflow is, at present, suspended and the data analysis should focus more on the different tendencies of the
volunteers towards the classification procedure. It will be necessary to identify those that are pointed in the right direction and to
steer the citizen scientists towards them. The research team will continue to adopt a heuristic approach, but this time there will be
a focus on narrowing down the outcomes to a degree that would make them useful, as is the case with the Introductory workflow.
This should eventually result in a new workflow, with different datasets, tutorials etc.
Finally, an entirely new workflow is being considered. The idea being that this new workflow would provide volunteers with
tomography plots for analysis and would focus on pattern recognition for the fast deciphering of objects and formations. Based on
the experience gained so far, it is clear to the research team that this would be a much more complex task than the tasks provided so
far. It represents an opportunity for the muon tomography community to provide data, offer perspectives and expertise, but also to
involve many expert participants to support and evolve the project. The muon tomography community has limited experience with
citizen science and a discussion on how to integrate the approach should prove beneficial for the domain as a whole.
The demonstrator project is now also an organic component of the outreach activities carried out by the research team and
functions as a useful tool in terms of introducing audiences to the experimental prerequisites of muon tomography, while, at the
same time, benefiting data analysis techniques through the categorisation of more data subjects.
3 Sonification activities
The use of sonification in astronomy has existed for years. Researchers have long highlighted the need for multi-modal approaches
in teaching and learning environments, presenting principles for inclusive material, based on user-centred and universal design.
Recently [20], following an international sound workshop, held in August, 2021, created a repository of existing software, which
collects the results of 98 projects that have been developed since 1962, many of them now discontinued, with a lack of documentation
or without evidence of applications in science. Almost 80% of these sonification projects were carried out between 2011 and 2021.
In most cases, the sonification mapping of the dataset was defined by its creator and shared as a final product or even with some
musicalisation, not clearly devoted to the study of the data, the identification of features or even to research.
Taking into account the increasing examples of sonification in astrophysics, sonoUno is one of the sonification software that
translates data from two or more column tables into sound. It is an open-source application, based on a modular design, which
allows users to open different datasets, and explore them through visual and auditory display, the last permitting them to adjust
visual and sound settings to enhance their perception. The project has been user centred from the beginning, and has been designed
following different steps during which the user has always guided improvements, from the analysis of accessibility using the ISO
9241-171:2008 standard [21,22], to focus group sessions, with people with and without visual disabilities testing the different
sonoUno versions [23].
The software emerged with an early first release, at the early stages of the REINFORCE project, and, because of this, it was
possible to add multiple new functionalities and developments, including the sonification of diverse sets of data from the demonstrator
projects.
123
Eur. Phys. J. Plus (2024) 139:868 Page 15 of 38 868
3.1 The team working on sonification
The sonification team required specialists and, to this end, a group with people acting on specific modules of the software, working
in a cooperative way, was created. This allowed to improve both the desktop version of the software and the web interface, using
new resources. The team is multidisciplinary, including astronomers, engineers, software designers, educators, disability specialists,
neurologists, and sociologists, both disabled and not.
3.2 User-centred design
From the beginning, sonoUno was designed with the focus on the final user, first with a theoretical framework based on user-testing
analysis of other platforms [22] and the ISO 9241-171:2008 standard; then with a focus group using the sonoUno prototype [23].
One of the conclusions of the focus group sessions was that sonoUno allows people with different sensory styles to explore and
analyse scientific data presented in tabular format, through a synchronised visual and sound representation.
Following the first release, each update of the software was tested with different users. They tested the software with their own
data sets and sent feedback back to the development team by email or through the completion of a form. In general, the users are
contacted once a year, and even when not all participants answer, feedback from at least five people is obtained. This methodology
of user testing gives visibility to sonoUno even, in some cases driving new use cases, such as: Sensing the Dynamic Universe,23
Taller de sonorización dictado por el IES José de Frugoni Pérez en España24,25 and The Sounds of BEARS.26
3.3 sonoUno development
Two versions of the software have been developed: a browser-based version, accessible via the web, and a desktop version.
3.3.1 Desktop-based version
The first approach to sonification was developed for a desktop computer; it used the graphic user interface design obtained from the
previous software and user-centred analysis. Then, with the first prototype based on MIDI sonification, a focus group session was
conducted to test the software’s usability and efficiency. From that analysis, it becomes apparent that user-centred software allows
visually impaired people to explore the data, not only as a user, but also as part of the development and improvement community
[23]. Some bugs were also detected and fixed in future updates; one of the notable modifications was the change of the sound library
to one that allowed for more resolution in the sound.
The current version of sonoUno now contains sonification and visualisation of one-dimensional data sets; math functions; sound-
setting configuration and plot-setting functionalities; a bash script that uses data provided by the user and stores the sonification and
plot files; and three additional scripts to sonify specific data sets related to the demonstrator projects.
3.3.2 Web-based version
The web-based version of the software was developed in response to user recommendations and in order to ensure wide and global
access to the software
The browser version was developed using HTML, CSS and JavaScript and uses the ARIA protocol to ensure communication
with screen readers. For the sound synthesis itself, the tone.js27 library was selected. The result was a versatile application for use
in the sonification of any data set using a tabular format and that can be used via a web browser on a computer, tablet, or mobile
device.
3.4 Sonifying the demonstrator-project data
Different approaches were adopted in order to sonify the datasets of the different demonstrator projects. These are detailed below.
•For GWitchHunters and Deep Sea Explorers data, image sonification was used. The openCV28 library for image manipulation
was used in conjunction with the sonoUno sound library. This consisted of converting a spectrogram image (Fig. 8,29 Figs. 9,
10) to grayscale and translating to sound the normalised total grey value of each column of data. For the sound parameters, the
23 https://lweb.cfa.harvard.edu/sdu/index.html.
24 https://astronomiayeducacion.org/taller-de-introduccion-a-la-sonificacion/.
25 https://astronomiayeducacion.org/taller-2-de-sonificacion-descubriendo-el-universo/.
26 https://stephenserjeant.github.io/sounds-of-bears/.
27 https://tonejs.github.io/.
28 https://opencv.org/.
29 https://www.youtube.com/watch?v=pkiGdZu5gEo.
123
868 Page 16 of 38 Eur. Phys. J. Plus (2024) 139:868
Fig. 8 ABlip type glitch, with
neither grid nor axis edges. The
blue vertical bar indicates the
sonification position
Fig. 9 Example of a Deep Sea Explorers bioacoustics workflow data subject. A click (millisecond-range soundwave) from a sperm whale
brightness value corresponded to the highest tone, and the darkest value corresponded to the lowest tone (silence). The other
parameters, such as instrument and frequency limits of sound, are fixed at the beginning of the code, but it is intended to add a
way to indicate them via command line in the future;
•For New Particle Search at CERN, a different approach was necessary in order to be able to classify different particles using only
sound. Upon detecting activity in any of the detector layers (calorimeter or inner detector) a scanning process was initiated within
a narrow cone covering the active region. This scanning commenced at the centre of the detector and extended radially outward in
three-dimensional space. Exploiting the distinctive interactions of different particles in each detector layer, various sound patterns
and audio volumes were utilised to provide comprehensive information about the activity within the specific detector region.
123
Eur. Phys. J. Plus (2024) 139:868 Page 17 of 38 868
Fig. 10 A data subject from the
Deep Sea Explorers Bioacoustics
workflow, sound-tracked with the
code for sonoUno image sound
enhancement
Each particle is visually represented in two views (an example is shown in (Fig. 11): an electron with a track in the inner detector
that points to a cluster in the calorimeter; a converted photon with two very close tracks in the inner detector that point to a
cluster in the calorimeter; a muon with a long track that traverses all of the detector layers and which could, although this is not
necessarily the case, be pointing to a cluster; a photon with a cluster in the calorimeter, but with no track in the inner detector; or
an unknown particle with only a track in the inner detector or another representation that is not covered by any of those described
above.
The audio representation of the above cases progressed simultaneously in both views and was based on the existence (a continuous
sound of two seconds) or absence (a silence of two seconds) of the track in the inner detector, the existence (a characteristic sound
of one second) or absence (a silence of one second) of the cluster in the calorimeter (the sound volume representing the energy),
and the existence of a track beyond the inner detector (in this case, the initial continuous sound length changed from two seconds
to four seconds)30;
•In the case of Cosmic Muon Images, the representation with sound was based on the possibility of correlating the deposit of
energy through the three layers of the detector (Fig. 1231). The sonification was based on the one-dimensional plot, with the top
and bottom representations containing 32 channels and the middle one only 16 channels. In this case, 16 piano notes are used,
one for each channel in the middle plot, and for the other plots, one note for two close channels. In the sonification process, the
note heard corresponds to a deposit of energy in the plot; when more than one channel presents a deposit of energy, a combination
of notes is provided.
3.5 Tactile models
Within REINFORCE, the team working on the sonification aspects of the project, also produced a toolbox of tactile models; physical
representations of data subjects used in each of the demonstrator projects. These proved useful resources in workshops held during
the project.
3.6 Challenges: difficulties and issues faced; how they were overcome
One of the most significant challenges of the work of the sonification team in REINFORCE was understanding how best sonify
the individual data subjects, keeping in mind the different data formats and ways in which the visual representations varied. This
required extensive collaboration with the individual demonstrator project research teams and numerous iterations in order to be able
to arrive at a point that was satisfactory for all involved.
It proved necessary to develop the software several times during this, including introducing specific piano notes to describe the
particle tracks, and tick marks, to indicate the beginning and end of a sonification.
3.7 What worked particularly well
Working with the different demonstrator project research teams proved to be a huge benefit to the development of sonoUno overall.
The exchange with the different teams and different ways of investigating and exploring the data, made it possible to produce a
versatile sonification library applicable to multiple scenarios. In this sense, the work done in REINFORCE particularly emphasises
the importance of trans-disciplinary collaborations.
3.7.1 Course on sonification
Previous studies have shown that auditory performance improves when combined with visual stimuli and vice versa [24]. Taking
this into consideration, the sonification team used two sensory pathways, sound and vision, with the intention of performing a test
30 https://www.youtube.com/watch?v=XMaYIJkJIHg.
31 https://www.youtube.com/watch?v=EYhcdyO2w2I.
123
868 Page 18 of 38 Eur. Phys. J. Plus (2024) 139:868
Fig. 11 A comparison of an event displayed in HYPATIA with its representation in sonoUno
of multi-sensory training and with the aim of achieving the familiarisation of participants with this type of technique in signal
detection. In order to schedule training that made it possible to presenting visual and auditory signals, allowing for the identification
of patterns and the saving of this user interaction, the PsychoPy software was used [25]. This software was designed for the creation
of experiments in behavioural sciences (such as psychology, neuroscience, and linguistics, among others) and allows precise spatial
control and synchronisation with different stimuli.
Over two sessions a complete introduction to multi-modal perception (visual, auditory and tactile) as well as specific training in
sonification, was performed. The course was designed so that the complexity of the actions required on the part of the participants
grew over the two sessions. Each session was made up of three blocks, one for each type of data: (1) three types of glitches; (2)
two LHC events; and (3) four muography events (two with the presence of a muon and two without). The first part of the training
session was devoted to understanding the data and knowing how to classify it, presenting examples of each type. In the second part
of the activity, different events were presented for each type of, randomly selected, data.
123
Eur. Phys. J. Plus (2024) 139:868 Page 19 of 38 868
Fig. 12 Views of the same event representing the existence of a muon: 3D view (left), 2D view(right), note the channels mentioned in the text
Following the activity, participants were asked to complete a survey about the web version of sonoUno and the training course.
Responses in direct relation to the web interface were overwhelmingly positive, with 77% of participants feeling that the software
could be used to improve their own work, research, professional and educational activities, while responses in relation to the
complexity of the sonifications of the demonstrator project data proved what were expecting; i.e., that the GWitchHunters glitch
sonifications and those relating to the Cosmic Muon Images project were relatively easy to interpret, while those relating to the New
Particle Search at CERN demonstrator project were less straightforward, owing to their complexity and depth.
3.8 Plans for the future
In the framework of the REINFORCE project, progress was made towards the achievement of two objectives. The software was
made more versatile in terms of the handling of sound effects, going beyond a single time series; while work also progressed in the
provision of sound as a service. This novel idea of embedding the service in a web browser was investigated using a Web Server
Gateway Interface (WSGI) application that acts as a sonoUno server using WebAssembly32 and would maximise code reuse between
the actual sonoUno server, used by researchers for computationally intensive or batch jobs, local browsers for the computers of
citizen scientists and users of Zooniverse. A prototype, using Flask,33 looks very promising, and is being explored.
4 Citizen-engagement strategy
In REINFORCE, citizen science acts as the vehicle that aims to bridge the gap between large research infrastructures in physics and
society, providing the framework and tools for effective and sustainable interaction between them. On one hand, citizens are trained
in frontier science, they are in constant connection with researchers through dedicated communities of practice, they provide their
feedback, they voice their concerns, and they actively contribute to the exploration of the boundaries of knowledge. On the other
hand, researchers can receive help and support to refine their instruments and advance their research. This interplay leads to several
questions, such as: how do we design a successful citizen science project that balances social inclusion and scientific efficiency
[26]? What motivates citizens overall in an online citizen science project and how can we sustain this motivation over time [27,28]?
Who are the potential citizen scientists [29] and are they actually able to develop new knowledge at the frontiers of physics [30].
32 https://webassembly.org/.
33 https://flask.palletsprojects.com/en/3.0.x/.
123
868 Page 20 of 38 Eur. Phys. J. Plus (2024) 139:868
To address these questions, a citizen-engagement strategy was designed and implemented throughout the duration of the project
implementation [31], with prime focus on balancing inclusion and scientific efficiency. Large-scale events and inclusive design
are very important in terms of raising awareness of large numbers of citizens, offering everyone the opportunity to contribute to
a citizen-science project [32]. According to Spiers et al. [29], a typical Zooniverse project has a classification curve that displays
a peak of activity after launch, when then rapidly declines, a fact indicating that the majority of users contribute for a short time
before moving to another project that captures their attention. Meanwhile, the advanced requirements in terms of content knowledge
for the REINFORCE demonstrator projects and the large number of images for classification, combined with the retirement limits
set, as well as the need for careful classification of every image in order to achieve a scientifically sound result, posed a series of
requirements for the recruitment, continuous engagement and training of participants, which was estimated to be better facilitated
by forming a core group of citizens working on the demonstrator projects.
The strategy formulated was based on: an in-depth review of literature of existing citizen-science activities and public-engagement
frameworks, addressing citizens’ documented needs, motivations, characteristics, and interest in participating in citizen-science
projects in general; a dedicated citizen survey to support the collection of data, together with a series of vision-building workshops
and focus groups with small groups of citizens to identify key motivations of specific target groups (teachers, elderly citizens); a
task analysis of the four REINFORCE demonstrator projects to provide a careful consideration of the kinds of projects offered, the
scientific goals to be achieved, how to coordinate contributions, and arrange tasks within each demonstrator project.
To this end, the engagement team aimed to recruit volunteer citizens to participate in the project demonstrators, and also sought
to minimise the effects of losing momentum following the initial peak in participation, and to even increase and maintain a higher
participation over a longer period of time. More specifically, the strategy described the progressive approach for engaging citizens
in the demonstrator projects, while, at the same time, describing the tasks and the expected contributions from citizens and the
roles of the scientists in the process. A five-step approach is laid out, which involves enrichment of the demonstrator projects with
dedicated educational resources and training materials and encompasses activities aiming to inform citizens, to involve them, to
facilitate collaboration, to receive consultation and to empower them to become ambassadors of the projects in their communities.
Figure 13 details the engagement activities, citizens’ expected contributions and the researcher’s role in each step of the engagement
strategy. The engagement strategy was translated into a series of engagement activities that were implemented throughout the duration
of the project, both before the launch of the demonstrators and throughout their piloting period.
5 Participatory-engagement activities
The REINFORCE engagement activities focused both on broad-reaching activities that increased the project’s visibility and supported
the continuous influx of new interested participants; and on dedicated activities that engaged citizens to enhance their contribution
to the building of a core community of practice around a project that provides meaningful scientific output. It was estimated that
reaching out to at least 100,000 citizens would be an achievable and realistic performance indicator in terms of achieving the project
science and inclusion objectives. The ambitious goal of reaching out to this number of citizens was hindered by the emergence of the
Covid-19 pandemic and the consequent long periods of quarantine in Europe. This situation posed a great challenge, since traditional
forms of impactful engagement activities (such as public visits to large research infrastructures) could not be implemented for most
of the lifetime of the project. To this end, the project engagement activities were shifted to online and hybrid formats (for example
moving from face-to-face visits to virtual visits), while new types of activities were tested for the first time with great success.
Different audiences require different engagement methodologies. To this end, the REINFORCE consortium worked with varying
target groups, including: teachers, students, multi-modally-impaired citizens and senior citizens. Two particular cases are expanded
upon in detail here: using art as a vehicle to engage students in complex topics that go beyond the standard curriculum (Sect. 6.1);
and science for senior citizens (Sect. 7). Throughout the duration of REINFORCE, a total of 96 events, online and face-to-face, were
organised, all following the project’s engagement framework as described in Fig. 13. 47% of the events addressed the general public
and specific citizen groups; 25% were focused on reaching out to academics and researchers; 17% to teachers through dedicated
workshops; and 11% to students through dedicated implementation activities. Aggregating the citizens reached through events, as
well as through the online awareness activities for the project (such as Zooniverse newsletters), more than 500,000 citizens were
informed, while 30,000 citizens participated in events aiming to inform them and provide training. 21,400 citizens collaborated with
the project team and contributed classifications. The project team received consultation from 1,400 citizens, while it is estimated
that more than 1,000 citizens were empowered to become project ambassadors.
5.1 Impact of engagement activities and community building
To assess the impact of different types of events in terms of citizen participation and contribution, citizen engagement was system-
atically monitored throughout the project implementation using the Zooniverse portal and Google analytics. Quantitative measures
were applied to track the quantity of citizen contributions and type of engagement over time. This made it possible to build an
understanding of the dynamics of engagement in each of the REINFORCE demonstrator projects. The overall results are presented
in Table 1.
123
Eur. Phys. J. Plus (2024) 139:868 Page 21 of 38 868
Fig. 13 Details of the REINFORCE citizen-engagement strategy
123
868 Page 22 of 38 Eur. Phys. J. Plus (2024) 139:868
Fig. 14 Engagement activities in REINFORCE. Top left: Informing citizens through face-to-face visits at the Virgo detector; Top middle/right: Involving
citizens through hands-on activities; Bottom left: Inviting citizens to collaborate in dedicated challenges; Bottom middle: Receiving consultancy from citizens
through dedicated meetings; Bottom right: Empowering citizens to become REINFORCE ambassadors through the intensive International Training Course
Tab l e 1 Metrics and results for
citizen collaboration with the
REINFORCE demonstrator
projects
Metric Value
Total number of citizens collaborating with REINFORCE 21,400
Total number of classifications 1,125,431
Total live-time of REINFORCE demonstrator projects (months) 41.6
Total person-time dedicated by citizens (months) 82.9
Number of discussions in Zooniverse community 5206
According to analysis, citizen scientists devoted double the effort (calculated based on the time allocated by the user logs) invested
by the research team. This is evident from the time dedicated by volunteers compared to the total live-time of the demonstrator
projects. This is a unique outcome of the work performed in the framework of the project, as it demonstrates that the contribution
of citizen scientists could really be quite significant. Figure 15 displays citizen classifications over time during the piloting period
of the project, which began on the 19th of October, 2021, and ended on the 25th of October, 2022.
By organising similar events for the different demonstrator projects, the project team had the chance to acquire useful data in terms
of the user-friendliness of the approaches and the interfaces used and the complexity of the tasks assigned to citizens. It was observed
that the period with the most engagement activities corresponds to a higher classification rate, as well as a considerably higher number
of participants. The engagement of the citizens over time and the overall demonstrator-project appeal were periodically assessed,
allowing the engagement team to providing the individual demonstrator project research teams with triggers as to when to act and
ensure their project remained interesting and motivating for the volunteers.
Awareness campaigns and dissemination activities, such as the events and newsletters flanking the official launches of the
demonstrator projects on Zooniverse, appear to have had a high impact in terms of daily classifications, with a low number of
classifications per participant and a high drop-out rate. According to the data, online events with massive participation, such as
dedicated challenges and competitions, provided 41% of the total classifications of the demonstrators, with the majority of these
provided by a core team of dedicated participants. As examples, the REINFORCE Winter Challenge34 acted as a social trigger that
engaged 720 citizens to perform a total of 80,000 classifications, while the EPS Citizen Science Competition35 engaged 374 citizens,
who registered and performed a total of 201,000 classifications. The sustained participation of citizens in these events displays a
high dependence on these challenges, as well as the related prizes.
It is possible to argue that the engagement activities managed to balance inclusion and scientific efficiency by reaching out to
a large audience through dedicated large-scale events, while forming a core community of participants who contributed the bulk
of the project classifications. This community-building was made possible by the commitment and accessibility of the researchers
operating the demonstrator projects. The Zooniverse Talk forum was the place where day-to-day interactions between researchers
and volunteers took place. For the research teams, this included answering questions and discussing the work of the demonstrator
project, as well as more general topics, announcing events, recognising citizen participation, empowering citizens and announcing
project results. Beyond forum interactions, community building was further facilitated through meetings (both face-to-face and
34 https://reinforce.ea.gr/winter-challenge/.
35 https://reinforce.ea.gr/eps-citizen-science-competition/.
123
Eur. Phys. J. Plus (2024) 139:868 Page 23 of 38 868
Fig. 15 Daily Classifications versus time overall (black) and per demonstrator project (blue: GWitchHunters, orange: Deep Sea Explorers, red: New Particle
Search at CERN, green: Cosmic Muon Images) during the piloting period. The peaks that correlate with the organisation of specific events have been labelled
accordingly
Tab l e 2 Overview of the
engagement metrics for the
REINFORCE demonstrator
projects. Data correspond to the
period starting on the 19th of
October, 2021, and ending on the
25th of October, 2022
Metric GWitchHunters Deep sea explorers New particle search Cosmic muon imgs
Live time (months) 11.4 8.6 12.1 9.5
Total number of participants 8223 4221 7983 3951
Total number of
classifications
645,592 109,409 260,637 109,703
Median classification time
(sec)
7.0 12.6 26.0 19.0
Median classifications per
participant
20.0 6.0 5.0 7.0
Median time per participant
performing classifications
(mins)
11.8 6.3 13.7 8.3
Gini coefficient 0.84 0.84 0.89 0.86
online) which helped citizens and researchers to also put a face to the person on the other side of their monitor; the person they had
been collaborating with for a long time.
5.2 Highlights of individual demonstrator-project characteristics
The effectiveness of the engagement activities heavily depended upon the respective characteristics of each demonstrator project.
Tabl e 2provides an overview of individual demonstrator-project metrics for the piloting period.
The differences between the four demonstrator projects stem from the convolution of different contributing factors, including the
nature and difficulty of the tasks (from pattern recognition of noise features to track reconstruction), the respective characteristics of
the interfaces, the live-time of each demonstrator, the community support, the knowledge background required, the popularity of the
respective science topic, the support from other dissemination channels, etc. As an example, the easy-to-use game-based interface
of the GWitchHunters demonstrator, as well as its launch in a mobile-friendly version, have contributed significantly to the success
of the demonstrator project. A major contributing factor was also the support of a large research infrastructure, which increased its
dissemination activities in relation to the project a lot. With a median classification time of seven seconds, the effort per classification
can also be argued to be lower than that of the other demonstrators. These facts contribute to the appealing nature of the topic of
this demonstrator. According to Spiers et al. [29], “All Zooniverse projects display unequal volunteer classification contribution”.
This characteristic is also present in the four REINFORCE demonstrator projects. In order to identify the inequality in terms of
citizen contributions, the Gini coefficient for each project was calculated. The Gini coefficients presented in Table 2[0.84−0.89]
123
868 Page 24 of 38 Eur. Phys. J. Plus (2024) 139:868
indicate an unequal distribution of classifications among the participants of each demonstrator project, with a maximum variation
of the order of 5% between demonstrators. Based on the results provided by Spiers et al. [29], the Gini coefficients reported for the
REINFORCE demonstrators are within the range of coefficients reported for the bulk of Zooniverse astronomy-oriented projects.
A further comparison of REINFORCE with other existing relevant Zooniverse projects is currently a research in progress.
As an epilogue, for one to reverse or delay the rapid fall of interest in the demonstrator and all online citizen-science projects
in general, the engagement model tested in REINFORCE has great added value, a fact that is shown in the shapes of classification
distributions over time and effectively in the classification output of the project. which ultimately resulted in more data for the
demonstrator project research teams. Furthermore, organisation of such engagement activities goes hand in hand with community-
building activities, which require the commitment of researchers and citizens and affects the overall sustainability of citizen-science
projects over time.
6 Art & Science
As Root-Bernstein et al. [33] remind us, more than fifty years ago, the Nobel Prize winner van’t Hoff [34] proposed a correlation
between creative activities outside of science and scientific imagination. Other studies have shown that a positive association exists
between creative activities and success in science [35]. More generally, adopting art practice in elementary schools, has proven to
enhance young children’s learning in science [36]. Indeed, by using art in science, children are stimulated to explore scientific facts
and concepts in a multi-dimensional way, referring not only to inquiry and analysis, but also to imagination, emotions, and creativity
[37,38].
6.1 International Youth Art & Science competition
Starting from these assumptions, the REINFORCE project launched an International Youth Art & Science competition under the
theme Humans as observer and listener of the cosmos. The focus of the contest was on concepts of fundamental physics, such as
space, time and the nature and structure of the Universe. These same concepts underlay a cross-reflection between the artists and
the scientists involved in the project.
The aim of the contest was to stimulate the participation of young people in the re-questioning of the fundamental concepts of
physics that are mentioned above, by asking them to translate into artistic manifestations their understanding of these notions. To
this end, entries in the REINFORCE International Youth Art & Science contest were grouped into two different age categories, from
four-to-twelve years old and from twelve to eighteen years old. The individual calls for submission were designed accordingly, in
order to capture the attention of potential participants and stimulate their interests in the underlying science. More precisely, for
young children the project sought artwork representing humans observing space; listening to the sound of stars; discovering the
nature of space and time; and the structure of the Universe. On the other hand, participants in the 12–18 range were asked to represent
humanity observing space and more generally nature, through a multitude of senses, from vision to sound and touch; discovering
the embedding of REINFORCE research in the Universe, including the environment; wondering about the nature of space, time
and matter. Around twenty artworks were submitted by young artists aged five to seventeen. These were contributed from three
different continents (Europe, Asia and America) and six different countries (Italy, Greece, Bahrain, Ecuador, Argentina and North
Macedonia), with a prevalence of female artists (60%). All participants were awarded a certificate of participation and invited to
attend a remote or in-person tour of the Virgo interferometer, located at the European Gravitational Observatory.
Although the number of entries was not high (mainly because the contest was launched during the summer recess period, when it
becomes much more difficult to engage schools, teachers and students), the contest afforded the children the opportunity to explore
and learn about fundamental physics, the Virgo experiment and the project itself. For the youngest artists, the contest was also the
first time they were confronted with new concepts, such as the sound of the stars and the nature of space and time. The REINFORCE
project, on the other hand, built on this initiative to involve young citizens in science and organised a visit to Virgo, as well as a
Kid’s Lab on Messengers from Space,36 during the European Researcher’s Night events of 2022.
6.2 Art & Science events organised within REINFORCE
The REINFORCE project has shown very clearly how the tools of citizen science can involve anyone in fundamental physics
research, something that is so distant from people’s daily lives: from exploring the fundamental constituents of matter with particle
accelerators to listening to the most violent cosmic phenomena through gravitational waves, to the study of neutrinos and cosmic
rays. This is research that almost always generates applications that bring us back to the context in which our lives take place,
through the development of innovative technologies or monitoring the natural or human-made environment of experiments.
However, there is another aspect that establishes an important link between fundamental physics research and society: its profound
influence on thought, art and culture in general. The new visions of the world and the cosmos, often revolutionary and counter-intuitive,
imagined and then verified with experiments by physicists, answer ancestral questions and humanity’s irrepressible aspiration to
36 https://www.reinforceeu.eu/platform-for-artistic-intervention/kids-lab-messengers-space-during-european-researchers-night-2022.
123
Eur. Phys. J. Plus (2024) 139:868 Page 25 of 38 868
investigate the origin and reflect on its place in the cosmos. Because of this, they inspire new artistic and philosophical visions,
generate stories that enter the common imagination at all levels, from cinema to comics, from science fiction to interactive and
artistic installations.
Research infrastructures operating at the frontier of our knowledge can be among the leading actors in this process, developing
collaborations with visual artists and musicians, stimulating the production of installations for the general public and organising
public events, theatrical performances, conference-shows, in which scientists and artists can dialogue directly with the public, using
different languages. These experiences have a dual significance: on the one hand, they make knowledge and ideas that are often
not yet widely disseminated, accessible to the general public; on the other, they are a true laboratory for the development of new
languages and syntheses between science and imagination, knowledge and art.
In this sense, the dimension of game and entertainment also plays a crucial role in communicating effectively and bringing the
public closer to subjects with which they are not familiar. Recently, it has been emphasised in several contexts how an intertwining
of education and basic research, with innovative languages, is increasingly urgent, not least to counter the negative and persistent
effects of the pandemic on the education of the youngest [39].
Finally, these actions related to public communication and cultural imagery are complementary, supporting and reinforcing the
motivation of citizens directly engaged in citizen-science initiatives. There is a virtuous circle between public communication and
citizen-science actions, which evidently reinforce each other.
In this context and with this inspiring vision, EGO, in collaboration with partners, developed, within the framework of the
REINFORCE project, a series of initiatives and events aimed at the general public and, in some cases, particularly at the youngest.
The events mainly explored topics related to gravitational and multi-messenger astronomy, i.e. listening to the cosmos via
gravitational waves. The theme of cosmic sounds was especially explored and developed, both as a metaphor and narrative suggestion,
and as an actual object of musical research, also through the sonification and elaboration of scientific data, in collaboration with
musicians and sound designers.
Several public events were organised in this direction, also taking advantage of the intensive research on the sonification of data
undertaken within the REINFORCE project. Below is a brief description of a selection of some of the art and science events and
initiatives organised within REINFORCE.
•Black holes and gravitational waves in the cosmic concert - Concert/Installation - Online and in Rome (31st of December,
2020) Black holes and gravitational waves in a cosmic concert by Tomàs Saraceno, organised as part of the Festa d i R o ma,the
New Year’s Eve celebration in Rome. The event involved a public concert and installation by Tomas Saraceno, along with a
discussion involving the artist, the curators and scientists.37
•E quindi uscimmo a riveder le stelle - EGO, Cascina (Italy) (1st of October, 2021) A special evening under the stars of Virgo to
discover Dante’s cosmos and how we listen to and narrate the Universe today, in the unique setting of the European Gravitational
Observatory. Accompanying Lina Bolzoni, historian of Italian literature at the Scuola Normale Superiore, Stavros Katsanevas,
physicist and director of the European Gravitational Observatory, and Riccardo Pratesi, professor of mathematics and scholar of
the Divine Comedy, were the canti cantati of the ensemble A Ricuccata and the irreverent humour of David Riondino.38
•Lo Spazio e i Sensi - Pisa (Italy) (24th of September, 2021) Event held at La Nunziatina, Pisa. A night of science and music on
the many ways we can explore and listen to the cosmos, as well as the world around us, in the name of diversity and inclusion. The
guests were two exceptional protagonists of scientific research: Marica Branchesi, internationally renowned for her contribution
to the birth of so-called multi-messenger astronomy, and Wanda Diaz-Merced, a blind astronomer of Puerto Rican origin, one
of the global leaders in research into the sonification of astronomical signals. Stavros Katsanevas spoke with them, while the
conversation between space, sounds and gravitational waves was accompanied by music from the Italian-Greek singer Marina
Mulopulos.39
•Il suono dell’Universo, Lajatico (Italy) (19th of July, 2022) The Sound of the Universe - Dialogue under the stars between
gravitational waves and cosmic sounds. During the event, EGO director Stavros Katsanevas and Wanda Diaz Merced, world
leader in the sonification of astronomical data, talked about data sonification, what it means to listen to the cosmos and how
this is done, accompanied by the music of sound designer Massimo Magrini. Magrini’s contribution was developed within the
framework of REINFORCE.40
•Il suono dell’Universo, Genova (Italy) (30th of October, 2022 Within the context of the Genoa Festival of Science, The Sound
of the Universe took place: a dialogue on what it means to listen to the cosmos with Wanda Diaz-Merced and Stavros Katsanevas,
and the musical contribution of sound designer Massimo Magrini.41
•Athens Science Festival, Athens (Greece) (22nd and 23rd of October, 2022 Over 600 children, students and people of all
ages discovered Virgo and the science of gravitational waves through many interactive activities organized by Ellinogermaniki
Agogi. A room dedicated to gravitational waves was available, including an art and science installation developed by the Italian
37 https://www.ego-gw.it/blog/2020/12/30/black-holes-and-gravitational-waves-in-the-cosmic-concert-by-tomas-saraceno/.
38 https://www.ego-gw.it/blog/2021/09/17/e-quindi-uscimmo-a-riveder-le-stelle/.
39 https://www.ego-gw.it/blog/2021/09/24/lo-spazio-e-i-sensi-science-and-music-beyond-the-senses/.
40 https://www.ego-gw.it/blog/2022/07/15/gravitational-waves-and-cosmic-sounds-during-the-summer-around-pisa/.
41 https://www.ego-gw.it/blog/2022/10/29/the-sound-of-the-universe-at-genoa-science-festival/.
123
868 Page 26 of 38 Eur. Phys. J. Plus (2024) 139:868
Institute of Nuclear Physics (INFN) and a model interferometer. Visitors were able to find many work-stations at which they could
experiment with the GWitchHunters Zooniverse demonstrator project.42
7 Senior-citizen science
Another initiative implemented in REINFORCE designed and developed with the intention of increasing accessibility to the work
of the demonstrator project research teams was the Senior Citizen Science course. The course was co-designed with members of
the Università della libera età (University of the free age) group, based in Cascina, in the vicinity of the European Gravitational
Observatory (EGO). Sessions in the course were provided by members of the REINFORCE Collaboration based at the University
of Pisa and at EGO.
Running a course for senior citizens in the middle of a global pandemic was inevitably more complicated than it might otherwise
have been and required a certain logistic elasticity in terms of the calendar, which had to be shuffled and re-shuffled, as well as in
terms of participation numbers, with the number of people available to participate from one session to the next being relatively fluid.
Despite this, the implementation ultimately proved a success and even resulted in a subsequent edition for the 2022/23 academic
year, i.e. well beyond the lifetime of the REINFORCE project.
7.1 The course syllabus
The first edition of the course took place during the 2021/22 academic year and was formed of nine sessions, plus a concluding visit
to the Virgo interferometer, situated at EGO. The course took the following form:
•Classical particle physics was given by Giancarlo Cella, Researcher at the Italian National Institute for Nuclear Physics (INFN),
and Professor of Astroparticle Physics at Pisa University, on the 6th of October, 2021, and covered the concept of space and time
in Galileo, Descartes and Newton; particle trajectory, mass, speed, acceleration; and the Laws of Newton and the movement of
the planets;
•Particles & waves in the XX century was delivered by Massimiliano Razzano, Professor of Experimental Physics at the University
of Pisa, on the 8th of October, 2021, and covered: the scale and structure of the Universe; sub-nuclear structures; Newton, Maxwell,
Faraday, Hertz; electromagnetic waves; Einstein and Relativity; Planck, Schrödinger and the quantum revolution; Rutherford;
linear and circular accelerators and the Standard Model;
•Waves: concept and detection took place on the 10th of November, 2021, and was given by Stefano Rinaldi, PhD at the University
of Pisa, and covered: the concepts of wave in Huygens, amplitude, frequency, interference; electromagnetic theory and waves,
aether; their detection with the Michelson-Morley interferometer;
•The cosmology of the (in)visible universe took place on the 3rd of December, 2021, and was delivered by Stavros Katsanevas,
REINFORCE Coordinator and Professor of Physics (Exceptional Class) at the Université de Paris. The session ranged across a
broad spectrum of themes, covering: Newton and Maxwell; Einstein’s Theory of Gravity; quantum physics — Bohr, Schrödinger,
Dirac, Heisenberg, Feynman; and the Standard Model of subatomic particles; astroparticle physics; cosmic rays; Hubble; gravita-
tional lensing; redshift; neutrinos; quarks, gluons, protons, neutrons; Higgs theory and supersymmetry; ‘shadow’ particles; matter
and antimatter; nucleosynthesis; and gravitational waves.
•The fifth session, Citizen Science: from theory to practice, was delivered by Francesco Di Renzo, postdoctoral researcher at the
University of Pisa, on the 9th of March, 2022, and focussed on the work of the GWitchHunters demonstrator project;
•The General relativity session was delivered by Valerio Boschi, Researcher at the INFN, on the 12th of January, 2022, and
covered: spacetime, gravitational waves and their detection, LIGO and Virgo; sources of noise, analyses of waves and glitches;
•On the 29th of April, 2022, Gary Hemming, REINFORCE Technical Manager, of EGO, held a session under the heading Brain-
storming and resolution of technical and theoretical problems, with the aim of evaluating directly with course participants,
the ways in which the course had worked well up to that point, as well as to try to establish how and where it could be modified
to ensure it reached its full potential;
•The sonification of gravitational waves session was delivered by Francesco Di Renzo on the 6th of May, 2022, and covered:
signals and what they are; a particular example of a gravitational-wave signal: GW150914; signals as vehicles of information;
the characteristics of a sound; pentagrams and time-frequency representations; listening to gravitational-wave signals; handling
noise; and the voice of a glitch;
•The final formal session in the course, Art & Science was again delivered by Stavros Katasanevas and focussed on: space-time-
matter; the senses and beyond: the two infinities; the notion of cosmos, order and violence, the singularities; from multi-messenger
to multi-sensorial; and from Uranus to Gaia.
The course concluded with a final visit of the participants to the Virgo detector at EGO on the 18th of May, 2022 (Fig. 16). The tour
included a view of the interferometer from the top of the North Arm Technical Building; a view of the Main Experimental Hall from
the atrium of the Central Building of the interferometer; the Virgo Control Room; the Hall of the Main Building, which is home
42 https://www.ego-gw.it/blog/2022/10/27/ego-at-athens-science-festival/.
123
Eur. Phys. J. Plus (2024) 139:868 Page 27 of 38 868
Fig. 16 Senior Citizen Science
course participants visiting the
Virgo gravitational-wave
interferometer, based at the
European Gravitational
Observatory
to a miniature model of a Michelson-Morley laser interferometer, an original prototype of the Superattenuator suspension system,
used to reduce seismic isolation at the level of sensitive optical components in Virgo and a mock-up of one of the mirrors used in
the detector; entering into the North Arm tunnel of the detector in order to see the interferometer vacuum tube at close quarters.
7.2 What needed to be worked on?
An analysis of the evaluation session that was held with course participants, combined with feedback from session providers, brought
to light some interesting outcomes. Participants were genuinely interested in the subject matter approached, although finding the
right way to pitch it to them required some calibration. It became clear that a classical lecture-based environment was not the best
approach in terms of ensuring the active participation of the group, which favoured a more relaxed, informal back-and-forth. This
way of delivering sessions was subsequently adopted, with significant success.
It was also clear from the evaluation session that some members of the group were concerned that the level of the course had
aimed too high. Again, this feedback proved useful in recalibrating the sessions slightly, in order to try and ensure that they remained
as accessible as possible to as wider number of participants as possible.
7.3 What worked well?
Certainly, the more informal approach that was adopted as the course evolved proved a real success, building enthusiasm and
motivation within the group and resulting in an enjoyable visit to EGO, followed by a request to continue the course into the
subsequent academic year. One fascinating discovery from the feedback sessions, however, related to the session contents and their
degree of accessibility. The The Cosmology of the Visible Universe session, delivered by Stavros Katsanevas, was the only session
of the entire course that was delivered in English. Despite the fact that all of the group were mother-tongue Italian speakers, with a
self-confessed limited knowledge of English, the feedback from the group showed that this session was the one that they had found
most interesting and inspiring. Accompanied by a useful presentation, containing many images for reference, the group were clearly
inspired by the session and felt that they had been able to put together the pieces, picking up enough information from Stavros,
together with the slides he presented, to build a picture that was enthusiastically received.
Above all, the course proved the importance of creating an environment in which participants feel comfortable, can ask questions
and not have to worry about getting things wrong.
8 Critical and Scientific Thinking course
The REINFORCE Course on Critical and Scientific Thinking took place over the course of five different days between the 24th of
October and the 2nd of November, 2022. On the first day, Nobel Prize