Evaluation of the Psychometric Properties of PainChek® in UK Aged Care Residents with advanced dementia

Abstract

Background
The aim of this study was to further validate PainChek®, an electronic pain assessment instrument, with a population living with dementia in a UK care home.

Method
This study utilised a correlational design to evaluate the psychometric properties of PainChek® when compared to the Abbey Pain Scale (APS). Blinded paired pain assessments were completed at rest and immediately post-movement by a researcher and a nurse. A total of 22 participants with a diagnosis of moderate-to-severe dementia and a painful condition were recruited using opportunity sampling.

Results
Overall, 302 paired assessments were collected for 22 participants. Out of these 179 were conducted during rest and 123 were immediately post-movement. The results demonstrated a positive significant correlation between overall PainChek® pain scores and overall APS pain scores ( r = 0.818, N = 302, p < .001, one-tailed), satisfactory internal consistency (α = 0.810), moderate single measure intraclass correlation (ICC = 0.680) and substantial inter-rater agreement (κ = 0.719).

Conclusions
PainChek® has demonstrated to be a valid and reliable instrument to assess the presence and severity of pain in people with moderate-to-severe dementia living in aged care.

Full text

R E S E A R C H A R T I C L E Open Access
Evaluation of the Psychometric Properties
of PainChek® in UK Aged Care Residents
with advanced dementia
Ivana Babicova
1*
, Ainslea Cross
2
, Dawn Forman
1
, Jeffery Hughes
3
and Kreshnik Hoti
3,4
Abstract
Background: The aim of this study was to further validate PainChek®, an electronic pain assessment instrument,
with a population living with dementia in a UK care home.
Method: This study utilised a correlational design to evaluate the psychometric properties of PainChek® when
compared to the Abbey Pain Scale (APS). Blinded paired pain assessments were completed at rest and immediately
post-movement by a researcher and a nurse. A total of 22 participants with a diagnosis of moderate-to-severe
dementia and a painful condition were recruited using opportunity sampling.
Results: Overall, 302 paired assessments were collected for 22 participants. Out of these 179 were conducted
during rest and 123 were immediately post-movement. The results demonstrated a positive significant correlation
between overall PainChek® pain scores and overall APS pain scores (r= 0.818, N= 302, p< .001, one-tailed),
satisfactory internal consistency (α= 0.810), moderate single measure intraclass correlation (ICC = 0.680) and
substantial inter-rater agreement (κ= 0.719).
Conclusions: PainChek® has demonstrated to be a valid and reliable instrument to assess the presence and severity
of pain in people with moderate-to-severe dementia living in aged care.
Keywords: PainChek®, dementia, pain, validation, observational pain assessment
Background
Pain in frail older adults with dementia is a major con-
cern [1], with literature consistently reporting poor
treatment and management in terms of inappropriate
administration of analgesics and incorrect recognition of
presence and severity of pain [2,3]. Up to 80 % of indi-
viduals with dementia living in care homes experience
acute or chronic pain [4], yet it is still poorly recognised
and treated. This reiterates the need and importance to
develop an effective means to recognise and evaluate
pain in this population [5]. Less accurate and valid self-
report of pain in individuals with dementia has led
researchers to develop instruments which help to recog-
nise presence and severity of pain. Appropriate treat-
ment and management of pain is a fundamental human
right [6] and therefore not providing it is unethical. Per-
sistent and untreated pain in dementia has been linked
with an increased level of cognitive deterioration [7],
which may lead to premature death.
In addition, up to 90 % of people with dementia ex-
press Behavioural and Psychological Symptoms of De-
mentia (BPSD), which include symptoms such as
hallucinations, delusions, anxiety or depression and their
presence has been associated with increase in psychiatric
referral, incorrect use of antipsychotic medication and
increased healthcare costs [8]. BPSD are commonly
associated with cognitive decline in dementia, with
© The Author(s). 2021 Open Access This article is licensed under a Creative Commons Attribution 4.0 International License,
which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give
appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if
changes were made. The images or other third party material in this article are included in the article's Creative Commons
licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons
licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain
permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/.
The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the
data made available in this article, unless otherwise stated in a credit line to the data.
* Correspondence: i.babicova@derby.ac.uk
1
College of Health, Psychology & Social Care, University of Derby, Derby, UK
Full list of author information is available at the end of the article
Babicova et al. BMC Geriatrics (2021) 21:337
https://doi.org/10.1186/s12877-021-02280-0
Content courtesy of Springer Nature, terms of use apply. Rights reserved.

symptoms usually being present from early stages of de-
mentia and gradually worsening over time which is there-
fore negatively impacting life and the progress of the
condition [9,10]. Pain has a negative impact on BPSD,
which in turn has a negative impact on aspects of daily
living in people with dementia [11]. In such cases, effect-
ive pain management is needed to reduce BPSD such as
depressive symptoms, anxiety, stress or agitation.
It should also be noted, that BPSD have been reported
to be a major source of distress for caregivers and family
members [12]. Informal family carers perceive BPSD as
challenging as the symptoms are often associated with
declining relationship between the person with dementia
and the family members, suggesting an inevitable deteri-
oration of the person’s condition [13].
Current observational pain assessment instruments are
mostly paper-based and rely on observation and correct
identification of presence and severity of pain indicators
by the assessor. While an observational pain assessment
is not the same as self-report of pain, observational pain
assessment instruments can be useful to assess and
monitor changes in severity of pain over time [14]. How-
ever, many of the instruments have methodological and
practical shortcomings [15].
Several factors which could influence appropriate pain
assessment and treatment have been identified [16]
which included racial and ethnic disparities. While there
are some underpinning physiological mechanisms in-
volved, pain is largely a subjective experience, and as
such, observer judgement is a factor which could hinder
pain assessment due to observer’s previous knowledge,
experience and bias of pain [17] suggesting that some
observational pain assessment instruments are subject-
ive, have limited evidence of accuracy, lack consistency,
may be subject to bias and are underutilised in practice.
To increase the accuracy of pain assessment, these
shortcomings and biases need to be addressed.
PainChek® has been developed as a potential solution; it
is a point-of-care application (App) which uses artificial
intelligence and smart automation, to limit some of the
factors which could hinder objective pain assessment
[18]. PainChek® minimises human error and bias
through the use of automated facial recognition and ana-
lysis to detect in real-time facial micro-expressions
called action units (AUs), derived from the Facial Action
Coding System [FACS] [20], which are indicative of pain.
After detecting pain related facial AUs, the user then
goes through a series of digital checklist and enters the
presence of other non-facial pain cues covering the fol-
lowing domains: The Voice, The Movement, The Behav-
iour, The Activity, and The Body. To minimise inter-
rater variability, definitions for each item are provided
within the App and a binary scoring system is used (i.e.
is this behaviour present: Yes = 1 and No = 0). The App
collates the data from the six domains to automatically
compute a total pain score. The overall pain score is
then used to assign the severity of pain, where a score of
0–6 indicates no pain, 7–11 indicates mild pain, 12–15
indicates moderate pain and 16–42 indicates severe pain
based on calibration against the Abbey Pain Scale (APS).
More details on how the PainChek® tool works has been
published elsewhere [19]. PainChek® has previously
undergone psychometric [18,20] and clinimetric [21]
evaluations in Australia, and has demonstrated signifi-
cant potential for use in the assessment of pain in people
with moderate-to-severe dementia when compared to
the APS. In 2018 the Royal College of Physicians, British
Pain Society and British Geriatric Society provided
guidelines for observational pain assessment in older
people with dementia. While the guidelines did not pro-
vide a recommendation for a single observational pain
assessment tool, it included a practical suggestion for
the APS, among other tools within the UK [22].
In addition, the APS has been selected as a suitable
clinical reference for this study for several reasons.
Firstly, while there is no known single ‘gold standard’for
an observational pain assessment tool, recent UK guide-
lines recommend either the APS or Pain Assessment in
Advanced Dementia Scale (PAINAD, 23). Additionally,
the APS has been recognised as one of the better avail-
able observational pain assessment tools in terms of psy-
chometric properties [5,23–29]. The APS also includes
at least one item from each of the six domains recom-
mended by the American Geriatric Society [AGS] (see
Table 1). Secondly, the APS is commonly used in the
UK and other English speaking countries. Lastly, previ-
ous studies in Australia demonstrated the validity and
reliability of PainChek® when compared to the APS [19,
20,22,30].
Taking into consideration the previous concerns
around observational pain assessment instruments and
the need for a highly valid means of assessing pain in
people with advanced dementia, the rationale for this re-
search was to further investigate the validity and reliabil-
ity of PainChek® in a UK care home setting with a
British cohort, using an Apple iOS operating system
which had not previously been clinically validated. As
such, the aim of this study was to further validate the
psychometric properties of the PainChek® in UK a care
home setting, using (Apple iOS version 2.14.1 (236) of
the App), with a British cohort of individuals living with
moderate-to-severe dementia.
Methods
Recruitment
Firstly, a nurse was recruited from the care home to as-
sess pain using the APS during the data collection
period. The nurse was selected on the basis of previous
Babicova et al. BMC Geriatrics (2021) 21:337 Page 2 of 8
Content courtesy of Springer Nature, terms of use apply. Rights reserved.

experience, skill, training, level of familiarity with resi-
dents and knowledge of dementia. The recruited nurse
had been working in the care home for six years, was
highly familiar with the structure, dynamics and resi-
dents within the care home and had completed in-house
pain in dementia training. Furthermore, the nurse was
highly familiar with the residents’typical and atypical be-
haviours and was trained and experienced in using the
APS.
Secondly, participants were recruited from a UK care
home using opportunity sampling (see Table 2for cri-
teria). Due to the nature of dementia, the participants
lacked capacity to give informed consent and were un-
able to comprehend the procedures of the study. Hence,
informed consent was obtained through their personal
consultee, legal guardian or Power of Attorney, which is
in line with Mental Capacity Act guidelines [31]. The in-
dividual who provided informed consent on behalf of
the participant was most often a close relative who came
to visit the participant on a regular basis. Ethical ap-
proval was granted by the University of Derby College
Research Ethics Committee, as well as the NHS Re-
search Ethics Committee.
Individuals diagnosed with Parkinson’s Disease were
excluded, as their facial features could have been com-
promised due to the nature and progress of the condi-
tion [32]. In addition, individuals with facial palsy, facial
deformities or who were partially or fully unable to ex-
hibit facial features were excluded. The inclusion and ex-
clusion criteria were based on a previous study [30].
PainChek®
PainChek® is a Class 1 medical device with a regulatory
clearance in Australia (Therapeutic Goods Administra-
tion), Europe (CE Mark) Canada (Health Canada) and
Singapore (Health Sciences Authority) for the assess-
ment of pain in individuals who are unable to verbalise
their pain, such as those living with moderate to severe
dementia. It is an observational pain assessment tool
which consists of 42 items spread across six domains;
namely The Face (n= 9), The Voice (n= 9), The Move-
ment (n= 7), The Behaviour (n= 7), The Activity (n=4)
and The Body (n=6) [30]. Items included in PainChek®
cover all six pain domains recommended by the AGS
[21] for accurate and reliable observational pain assess-
ment in people with cognitive impairment (See Table 1).
However, the items in The Face domain have been based
around the FACS, because as reported by Beach et al.
[33], behavioural pain scales that have objective facial
measures have better psychometric properties, than
those containing vague facial descriptors.
PainChek® is simple to use. Users are provided with on
device PainChek® User Guide and there is an online train-
ing module available at www.painchek.com. Users also
undergo one and a half hours of training delivered either
face-to-face or online which covers pain in dementia, what
is PainChek® and how to complete a PainChek® assess-
ment conducted by a PainChek® clinical consultant.
Table 1 Comparison of pain domains and items of American
Geriatrics Society and PainChek®
Pain Behaviour Domains and Items
AGS[19] PainChek® [26]
Facial expression
Slight frown; sad, frightened face
Grimacing, wrinkled forehead,
closed or tightened eyes
Any distorted expression
Rapid blinking
The Face (9 items based on FACS
allowing automated analysis)
Brow lowering (AU 4)
Cheek raising (AU 6)
Tightening of eyelids (AU7)
Wrinkling of nose (AU 9)
Raising upper lip (AU 10)
Lulling of corner lip (AU 12)
Horizontal mouth stretch (AU 20)
Parting lips (AU 25)
Closing eyes (AU 43)
Vocalisation/Verbalisation
Sighing, moaning, groaning
Grunting, chanting, calling out
Noisy breathing
Asking for help
Verbally abusive
The Voice (9 items)
Noisy sounds e.g. ouch, ah
Requesting help repeatedly
Groaning
Moaning
Crying
Screaming
Loud talk
Howling
Sighing.
Body movements
Rigid, tense body posture, guarding
Fidgeting
Increased pacing, rocking
Restricted movement
Gait or mobility changes
The Movement (7 items)
Altered or random leg/arm
movement
Restlessness
Freezing
Guarding/touching of body parts
Moving away
Abnormal sitting/standing/walking
Pacing/wandering
Changes in interpersonal
interactions
Aggressive, combative, resisting
care Decreased social interactions
Socially inappropriate, disruptive
Withdrawn
The Behaviour (7 items)
Introvert (unsocial) or altered
behaviour
Verbally offensive
Aggressive
Fear or extreme dislike of touch,
people
Inappropriate behaviour
Confused
Distressed behaviours.
Changes in activity patterns or
routines
Refusing food, appetite change
Increase in rest periods
Sleep, rest pattern changes
Sudden cessation of common
routines Increased wandering
The Activity (4 items)
Resisting care
Prolonged resting
Altered sleep
Altered routine
Mental status change
Crying or tears
Increased confusion
Irritability or distress
The Body (6 items)
Profuse sweating
Pale/flushed (red faced)
Feverish/cold
Rapid breathing
Painful injuries
Painful medical conditions
Babicova et al. BMC Geriatrics (2021) 21:337 Page 3 of 8
Content courtesy of Springer Nature, terms of use apply. Rights reserved.

For this study, PainChek® (Apple iOS version 2.14.1
(236) of the App) was installed on Apple iPhone 6s (iOS
version 12.2). This was the first clinical evaluation of the
iOS version of PainChek® application undertaken in a
different geographical location.
Procedure
Prior to the recruitment process, the researcher spent
four months visiting and observing processes and rou-
tines in the care home, this helped with familiarisation
and integration of the researcher into the care home.
Once consent was given, baseline MMSE scores
were collected. Paired pain assessments were collected
by the two assessors independently, but at the same
time for each participant at rest and immediately
post-movement. The two assessors were blinded to
each other’s assessment results. Paired ratings refer to
a total of four pain assessments taken during one ses-
sion; two assessments (one from APS one from
PainChek®) were obtained during a restful state (e.g.
participant sitting or lying down), followed by two as-
sessments (one from APS one from PainChek®) ob-
tained immediately after movement (e.g. participant
was asked to stand up and sit down, or after a trans-
fer from bed to a wheelchair). The paired assessments
were collected from participants over a period of 16
weeks. Once all paired pain assessments were com-
pleted, final MMSE assessments for each participant
were collected and debrief letters were sent out to
legal guardians or those with Power of Attorney.
Data Analysis
All statistics were analysed using IBM SPSS-26. The
demographics of study participants and raw data were
analysed using standard descriptive statistics, including
mean, median, mode, range, standard deviation and fre-
quencies or percentage for pain diagnoses and types of
dementia.
Several validity and reliability measures were investi-
gated. Firstly, concurrent validity was assessed by investi-
gating Pearson’s correlation coefficients between the
overall scores collected by APS and PainChek® as well as
separately for the rest and post movement pain scores.
This measure assesses the strength of a linear association
between two variables. As the two instruments had dif-
ferent scoring systems the correlation is not an indica-
tion of a total agreement, but a strong and significant
correlation would indicate that there’s a consistent in-
crease or decrease of pain score in both instruments.
Correlation coefficients were interpreted as follows:
values 0-0.3 indicate negligible correlation, 0.3–0.5 low
positive correlation, 0.5–0.7 moderately positive correl-
ation, 0.7–0.9 high positive correlation and 0.9-1 very
high positive correlation [34].
Secondly, interrater reliability was assessed. Cohen’s
kappa [35] statistic was used to assess total agreement
for the following categorical data: no pain, mild pain,
moderate pain and severe pain. The Kappa scores were
interpreted as follows: values ≤0 indicate no agreement,
0.01–0.20 indicate none to slight agreement, 0.21–0.41
as fair, 0.41–0.60 as moderate, 0.61–0.80 as substantial
and 0.81–1.00 as almost perfect agreement [32].
Next, intraclass correlation coefficient (ICC) measured
the reliability of ratings. The ICC ranges from 0 to 1,
where a value closer to 1 indicates a higher similarity be-
tween values from the ratings of the same group.
Values ≤0.5 are indicative of poor reliability, 0.50–0.74
indicate moderate reliability, 0.75–0.90 indicate good re-
liability and > 0.90 indicate excellent reliability [36].
Lastly, internal consistency was measured using Cron-
bach’s alpha (α), which examined whether the two in-
struments were measuring the same constructs.
Cronbach’s alpha was computed based on overall scores
of the two instruments, as well as rest and post-
movement scores. Hulin et al. [37] suggest that αof
0.60–0.70 generally indicates an acceptable level of reli-
ability and α≥0.80 indicates a very good level of
reliability.
In addition, discriminant validity was also investigated.
This was achieved by looking at comparisons between
pain scores at rest compared to post-movement for
PainChek® and APS. A significant result (p ≤.05) indi-
cates that the observational pain instruments are activity
dependent.
Table 2 Inclusion and exclusion criteria for this study
Inclusion criteria Exclusion criteria
Diagnosis of moderate or severe dementia (measured by baseline
MMSE score ≤20)
Individuals diagnosed with Parkinson’s Disease
Documented history of chronic pain condition (e.g. arthritis) OR
residents which are often treated for pain due to painful complaints
Individuals partially or fully unable to exhibit facial features (such as some
stroke survivors or those with facial deformities)
Residents of 65 years of age or older Individuals with a significant mental health condition such as severe
depression, anxiety disorders or schizophrenia which could result in
unnecessary distress
Individuals who have been advised not to take part by their GP, staff or family
member
Babicova et al. BMC Geriatrics (2021) 21:337 Page 4 of 8
Content courtesy of Springer Nature, terms of use apply. Rights reserved.

Results
Demographic characteristics of the participants
Overall, 302 paired assessments were collected from 22
participants. Of these, 179 assessments were conducted
during rest and 123 were conducted immediately post-
movement. The participants had a variety of dementia
diagnoses, pain conditions and demographic characteris-
tics as shown in Table 3. Given the demographic preva-
lence of dementia, it is unsurprising that the majority of
the participants were female (77 %). The first MMSE as-
sessment was obtained at baseline (Week 1), and final
MMSE assessment was obtained at the end of data col-
lection (Week 16). At the time of enrolment three resi-
dents were classified as having moderate dementia and
19 severe dementia based on their MMSE scores. During
the course of the study, two residents died, both of
whom had severe dementia (MMSE scores of 2 and 4,
respectively).
Correlation between the APS and PainChek® for overall
pain scores
The overall scores (i.e. pain assessments taken both dur-
ing rest and immediately post-movement) were analysed.
The overall mean for PainChek® pain scores were higher
(M = 6.73, SD = 2.66) than the overall mean for APS pain
scores (M = 3.57, SD = 1.42).
Pearson’s correlation coefficient revealed a significant
correlation between overall PainChek® pain scores and
overall APS pain scores (r= 0.82, N= 302, p< .001, one-
tailed). In addition, the following reliability measures
were tested; internal consistency (α= 0.81), single meas-
ure ICC (0.68; 95 % CI: 0.62 −0.74) and inter-rater
agreement (κ= 0.72).
Correlation between the APS and PainChek® pain scores
post-movement
In line with the direction of previous findings, the mean
was higher for PainChek® post-movement scores (M =
7.91, SD = 2.58) compared to APS (M = 4.12, SD = 1.60).
Concurrent validity was tested using Pearson’s correl-
ation coefficient, which demonstrated a significant rela-
tionship for post-movement scores between PainChek®
and APS (r= 0.81, N= 123, p< .001, one-tailed). Internal
consistency (α= 0.84), intraclass correlation for single
measures (0.73; 95 % CI: 0.63 −0.80) and an interrater
agreement (κ= 0.84) were also tested.
Correlation between the APS and PainChek® pain scores
at rest
The means for resting assessment pain scores were
higher for PainChek® (M = 5.91, SD = 2.40) than APS
(M = 3.18, SD = 1.14). Pearson’s correlation coefficient
for the rest condition also demonstrated a significant
correlation between APS and PainChek® pain assessment
scores at rest (r= 0.79, N= 179, p< .001, one-tailed).
Interrater agreement (α= 0.76), single measures intra-
class correlation (0.62; 95 % CI: 0.52 −0.70), and agree-
ment (κ= 0.64) were also tested.
In addition, in terms of discriminant validity, the dif-
ference in scores at rest and post-movement for
PainChek® (p< .001) and APS (p< .001) were both
significant.
Discussion
This psychometric evaluation study of PainChek® pro-
vides further evidence for the suitability of PainChek® as
a pain assessment instrument for individuals living with
moderate-to-severe dementia. In addition to existing lit-
erature, this study provides evidence specifically related
to the UK setting, in which the PainChek® has not previ-
ously been tested. Our findings provide further evidence
that PainChek® has strong concurrent validity [34], high
interrater agreement [35], good to very good internal
consistency [38] and a moderate level of intraclass cor-
relation [36]. Such findings support those from previous
validation studies [18,20,30]. It is important to note
that PainChek® consistently demonstrated these proper-
ties both when used to assess pain at rest as well as
post-movement, findings which are consistent with pre-
vious validation studies [18,20,30]. This was also
Table 3 Clinical and demographic characteristics of participants
Characteristics
Mean age (SD), years
Median age (range), years
84.7 (5.6)
85.5 (74–95)
Sex, N (%)
Male
Female
5 (23)
17 (77)
Ethnicity, N (%)
White British
Black British
21 (95.5)
1 (4.5)
Baseline mean MMSE score, (SD), N=22
Median MMSE (range)
5.78 (5.3)
4.5 (0–17)
End of study mean MMSE score, (SD), N=20
Median MMSE (range)
3.6 (4.5)
2(0–14)
Mean length of residency (SD), months at baseline
Median lengths of residency (range)
25.8 (25.5)
14.5 (3–83)
Diagnosis of dementia, N (%)
Alzheimer’s Disease
Mixed Dementia
Vascular Dementia
Other/Unspecified Dementia
Korsakoff’s Dementia
10 (45.5)
6 (27.3)
3 (13.6)
2 (9.1)
1 (4.5)
Diagnosis of pain conditions, N (%)
Arthritis
Osteoporosis
Osteoarthritis
Other musculoskeletal pain*
5 (22.7)
6 (27.3)
2 (9.1)
11 (50)
* Other musculoskeletal pain included lower back pain, tendonitis, stress
fractures and bed sores
Babicova et al. BMC Geriatrics (2021) 21:337 Page 5 of 8
Content courtesy of Springer Nature, terms of use apply. Rights reserved.

evidenced by the statistically significant elevation of pain
scores in the post-movement condition compared to the
rest condition for both tools. Further, pain scores imme-
diately post movement were significantly higher than
those undertaken at rest. This finding is also consistent
with previous studies and provides evidence of the tool’s
discriminant validity [18,27]. Please note the care of res-
idents and any decisions regarding their pain when they
scored highly during observational pain assessment man-
agement was dependent on the result of the APS and
the communication of the nurse with the rest of the
team, not the researcher.
In comparison to the previous PainChek® studies [18,
30], overall in this study, the PainChek® has demon-
strated a slightly lower concurrent validity. This could
be partially explained by more closely examining the rest
and the post-movement conditions. This study, like
many others, has further demonstrated that pain behav-
iours are best elicited by movement [39,40], hence the
consistently lower scores in the rest condition could be
explained by the difficulty of obtaining an accurate pain
score during a resting condition.
In terms of internal consistency, the results from the
present study demonstrated slightly lower agreement at
rest, but higher agreement in the post-movement condi-
tion when compared to Atee et al. [27]. Other observa-
tional assessment tools which have demonstrated similar
interrater agreement are the NOPPAIN [41], MOBID
[42] and previous PainChek® studies [30].
Overall interrater agreement scores were lower during
the rest condition than the post-movement condition.
This is consistent with previous literature indicating that
pain is easier to detect once elicited by movement [39,
40]. In addition, the pain assessment tool PAINE [43]
had a similar interrater agreement scores to those found
within this study.
The comparison of the baseline and end of study
MMSE scores demonstrated a deterioration in cognition
of all participants over the 16 weeks period of the study.
This is in line with expected deterioration of cognition
as a result of the progression of dementia [44]. This
finding further evidences the reliability of PainChek®, as
it continued to consistently identify the presence and se-
verity of pain over the 16-week data collection period,
despite the progression of dementia and the associated
deterioration of cognition. There are clinical implica-
tions for this finding, suggesting if PainChek® is used on
regular basis in UK care homes, it is able to continuously
provide an accurate pain assessment for individuals over
time, regardless of deterioration of symptoms or func-
tional status, therefore overcoming the potential human
error and bias which may occur with healthcare profes-
sionals who have daily contact with the individuals.
Overall, this could facilitate pain management through
improved pain assessment and better use of pain related
pharmacological and non-pharmacological interventions.
In terms of concurrent validity, previous researchers
[5] specified adequate observational pain assessment in-
struments should demonstrate at least a score of 0.4–
0.6, which PainChek® has exceeded. Observational pain
assessment instruments, such as the MOBID-2 [45]or
the Pain Assessment Checklist for Seniors with Limited
Ability to Communicate (PACSLAC) [46] and many
others as outlined in a systematic review [25] have also
demonstrated concurrent validity values which meet the
specified adequate range as outlined by Herr et al. [5].
However, research demonstrated inconsistencies be-
tween observational pain assessment instruments, spe-
cifically in the facial expression domain [46,47], due
to their subjectivity. It has also been suggested that
better accuracy and precision would benefit observa-
tional pain assessment instruments and that this
could be achieved by further refining and developing
the facial expression domain [47]. PainChek® has an
automated facial expression decoding function, there-
fore mitigating the chance of non-agreement between
raters by utilising facial recognition technology, which
does not rely on manual input of pain presence and
severity. This gives PainChek® an advantage over
other observational pain assessment instruments
which require the assessor to score the facial expres-
sion manually, as the face domain is often problem-
atic, difficult to score and remains to be one of the
most poorly scored domains carrying the highest level
of subjectivity, in comparison to other pain domains
in observational pain assessment instruments [48].
PainChek® is considered to be a screening tool, which
acts as a guide for users as part of future decision-
making process relating to pain management and treat-
ment of residents with dementia. The instrument in-
forms the user about the intensity of present pain, which
adds to its clinical utility in that the choice of analgesics
is often based on the intensity of the pain detected. The
cut-off points for different pain intensities have previ-
ously established against the APS [21]. However, to date
no studies have been reported that evaluated the impact
of PainChek® use on clinical outcomes, although this is
planned as part of a major implementation trial in aged
care currently underway in Australia.
In terms of study limitations, the study aimed to be
pragmatic in terms of being as realistic and practical as
possible, to help replicate everyday care home dynamics.
For example, participants were not taken into a separate
quiet room during pain assessments, as this would have
been unrealistic and unsustainable for this particular
care home. When residents show behaviour, which could
be associated with pain, the initial assessments are con-
ducted in the communal areas of the care home. Because
Babicova et al. BMC Geriatrics (2021) 21:337 Page 6 of 8
Content courtesy of Springer Nature, terms of use apply. Rights reserved.

of this, the administration of PainChek® and APS were also
completed in the communal areas, to replicate a realistic
approach to pain assessment in care homes. Some gener-
alisability limitations are present as it is a single site study
involving one care home. Although participants with a
variety of dementia diagnoses, levels of severity and a
range pain diagnoses and conditions were recruited, all as-
sessment were completed by the same two raters. A larger
study involving multiple trained nurses across multiple
care homes should be conducted to increase the overall
generalisability and ecological validity of the findings.
In addition, multiple attempts have been made in the
design of PainChek® to minimise potential human error.
This includes use of automated facial analysis, binary
scoring, in app definitions of each item, forcing users to
complete each domain and automated computing of
pain scores and assignment of pain intensities. Addition-
ally, users are also provided with structured training to
ensure their confidence and competence in completing
PainChek® assessments.
Strengths of this study include the observation visits
prior to data collection, which took place one to three
times a week over a period of four months. This enabled
the researcher to become familiar with the environment,
procedures, dynamics, staff and residents, but also
helped to understand how PainChek® could be imple-
mented within the care home’s daily processes and pro-
tocols in the future. The observational period took place
from September 2018 until January 2019, during this
time the researcher was often involved in small activities
with the residents, such as preparing tea and coffee,
helping to feed residents during mealtime and talking
with residents who were verbal. This enabled the re-
searcher to collect highly ecological and organic data, as
the participants reacted to the researcher with the same
familiarity as they would have reacted to a staff member.
Conclusions
In conclusion, this study provides evidence of PainChek®’s
psychometric properties when used within the UK home
care setting, therefore supporting its previously reported
validity and reliability from studies conducted in Austra-
lian residential aged facilities. This instrument has the
capacity to empower caregivers to accurately assess, treat
and manage pain in care homes. As such, PainChek® has
the potential to support healthcare providers to improve
quality of life in the population with dementia.
Abbreviations
APS: Abbey Pain Scale; GP: General Practitioner; ICC: Intraclass Correlation
Coefficient; iOS: iPhone Operating System; MMSE: Mini-Mental State
Examination; NHS: National Health Service; PACSLAC: Pain Assessment
Checklist for Seniors with Limited Ability to Communicate
Acknowledgements
The authors would like to thank the residents and staff of the care home
which allowed for this research to take place. The authors would also like to
thank Professor David Sheffield and Mustafa Atee for their guidance and
advice throughout this study.
Authors’contributions
All authors discussed the research aims, objectives and questions and
participated across the stages of this study, including study design and
protocol development (IB, AC, DF, JH and KH), recruitment and data
collection (IB), data analysis and interpretation (IB, JH and KH) write up of
manuscript (IB, AC, DF, JH and KH) and critical review of manuscript (AC, DF,
JF and KH). All authors have read and approved the final manuscript.
Funding
The authors received no financial support for this research.
Availability of data and materials
The PainChek® datasets used and analysed as part of this study are not
publicly available.
Declarations
Ethics approval and consent to participate
Ethical approval was granted by the University of Derby College Research
Ethics Committee, as well as the NHS Research Ethics Committee. Due to
nature of moderate to severe stages of dementia, the recruited participants
lacked the capacity to consent, therefore a legal guardian or a representative
provided a written consent on behalf of the participant.
Consent for publication
N/A.
Competing interest
JF is one of the co-founders of PainChek® and is a shareholder in PainChek
Ltd. He is also employed under contract through Curtin University as the
Chief Scientific Officer of PainChek Ltd. KH is one of the originators and
shareholder at PainChek Ltd, which is marketing the PainChek™instrument
(previously known as ePAT). KH consults for PainChek Ltd while serving as an
Associate Professor at University of Prishtina.
Author details
1
College of Health, Psychology & Social Care, University of Derby, Derby, UK.
2
University of Derby Online Learning, University of Derby, Derby, UK.
3
Curtin
Medical School, Curtin University, Perth, Australia.
4
Division of Pharmacy,
Faculty of Medicine, University of Prishtina, Prishtina, Kosovo.
Received: 22 August 2020 Accepted: 11 May 2021
References
1. Ferrell BA, Ferrell BR, Osterweil D. Pain in the nursing home. J Am Geriatr
Soc. 1990 Apr 1;38(4):409–14.
2. Won AB, Lapane KL, Vallow S, Schein J, Morris JN, Lipsitz LA. Persistent
nonmalignant pain and analgesic prescribing Patterns in elderly nursing
home residents. J Am Geriatr Soc. 2004 Jun;52(6)(1):867–74.
3. Reynolds KS, Hanson LC, DeVellis RF, Henderson M, Steinhauser KE.
Disparities in pain management between cognitively intact and
cognitively impaired nursing home residents. J Pain Symptom Manage.
2008;35(4):388–96.
4. Corbett A, Husebo B, Malcangio M, Staniland A, Cohen-Mansfield J, Aarsland
D, et al. Assessment and treatment of pain in people with dementia. Nat
Rev Neurol. 2012;8(5):264–74.
5. Herr K, Bjoro K, Decker S. Tools for assessment of pain in nonverbal older
adults with dementia: A state-of-the-science review. J Pain Symptom
Manage. 2006;31(2):170–92.
6. Somerville MA. Death talk: the case against euthanasia and physician-
assisted suicide. McGill-Queen’s Press-MQUP; 2001.
7. Whitlock EL, Diaz-Ramirez L, Glymour M, Boscardin J, Covinsky KE, Smith AK.
Chronic pain predicts accelerated memory decline and dementia. Innov
Aging. 2017 Jul 1;1(suppl_1):467–467.
Babicova et al. BMC Geriatrics (2021) 21:337 Page 7 of 8
Content courtesy of Springer Nature, terms of use apply. Rights reserved.

8. Cerejeira J, Lagarto L, Mukaetova-Ladinska EB. Behavioral and psychological
symptoms of dementia. Front Neurol. 2012;3(73):1–21.
9. David R, Mulin E, Mallea P, Robert PH, David R, Mulin E, et al. Measurement
of neuropsychiatric symptoms in clinical trials targeting Alzheimer’s disease
and related disorders. Pharmaceuticals. 2010 Jul 26;3(8):2387–97.
10. Fernández M, Gobartt AL, Balañá M. Behavioural symptoms in patients with
Alzheimer’s disease and their association with cognitive impairment. BMC
Neurol. 2010;10(1):87.
11. Cipher DJ, Clifford PA. Dementia, pain, depression, behavioral disturbances,
and ADLs: toward a comprehensive conceptualization of quality of life in
long-term care. Int J Geriatr PSYCHIATRY Int J Geriatr Psychiatry. 2004;19:
741–8.
12. Craig D, Mirakhur A, Hart DJ, McIlroy SP, Passmore AP. A cross-sectional
study of neuropsychiatric symptoms in 435 patients with Alzheimer’s
disease. Am J Geriatr Psychiatry. 2005 Jun;13(6)(1):460–8.
13. Feast A, Orrell M, Charlesworth G, Melunsky N, Poland F, Moniz-Cook E.
Behavioural and psychological symptoms in dementia and the challenges
for family carers: systematic review. Br J Psychiatry. 2016;208:429–34.
14. Herr K. Pain assessment strategies in older patients. J Pain. 2011 Mar;12(3):3–13.
15. Zwakhalen S, Hamers J, Abu-Saad HH, Berger MP. Pain in elderly people
with severe dementia: a systematic review of behavioural pain assessment
tools. BMC Geriatr. 2006;6(1):3.
16. Sengupta M, Bercovitz A, Harris-Kojetin LD. Prevalence and management of
pain, by race and dementia among nursing home residents: United States,
2004. NCHS Data Brief. 2010 Mar;(30):1–8.
17. Goubert L, Craig KD, Vervoort T. Facing others in pain: the effects of
empathy. Pain. 2005;118:285–8.
18. Atee M, Hoti K, Parsons R, Hughes J. Validation of the Electronic Pain
Assessment Tool (ePAT) in residents with moderate to severe dementia: a
pilot study [Internet]. ePAT. 2017 [cited 2017 Oct 25]. p. 1. Available from:
http://www.epattechnologies.com/wp-content/uploads/2017/04/3104-Valida
tion-A4-Flyer.pdf.
19. Atee M, Hoti K, Hughes JD. A Technical Note on the PainChek™System: A
Web Portal and Mobile Medical Device for Assessing Pain in People With
Dementia. Front Aging Neurosci. 2018 Jun 12;10(JUN):117.
20. Atee M, Hoti K, Parsons R, Hughes JD. Pain assessment in dementia:
evaluation of a point-of-care technological solution. J Alzheimer’s Dis. 2017;
60(1):137–50.
21. Hoti K, Atee M, Hughes JD. Clinimetric properties of the electronic pain
assessment tool (ePAT) for aged-care residents with moderate to severe
dementia. J Pain Res. 2018;11:1037–44.
22. Schofield P. The Assessment of Pain in Older People: UK National
Guidelines. Age Ageing. 2018;47:i1–22.
23. Zwakhalen SM, Hamers JP, Huijer Abu-Saad H, Berger MP. Pain in elderly
people with severe dementia: A systematic review of behavioural pain
assessment tools. BMC Geriatr. 2006;6(3):1–5.
24. Park J, Castellanos-Brown K, Belcher J. A review of observational pain scales
in nonverbal elderly with cognitive impairments. Res Soc Work Pract. 2010;
20(6):651–64.
25. Lichtner V, Dowding D, Esterhuizen P, Closs SJ, Long AF, Corbett A, et al.
Pain assessment for people with dementia: a systematic review of
systematic reviews of pain assessment tools. BMC Geriatr. 2014;14(1):138–57.
26. Herr K, Coyne PJ, Ely E, Elinas G, Manworren RCB. Pain Assessment in
the Patient Unable to Self-Report: Clinical Practice Recommendations in
Support of the ASPMN 2019 Position Statement. Pain Manag Nurs.
2019;20(5):404–17.
27. Kim Y-S, Park J-M, Moon Y-S, Han S-H. Assessment of pain in the elderly: A
literature review. Natl Med J India. 2017 Jul 1;30(4):203.
28. Geriatric Pain. Abbey Pain Scale (APS). 2019.
29. Parkman S, Mastel-Smith B, McGuire A, Duke G. Insights to Identifying and
Managing Pain in Persons With Dementia in Long-Term Care: A Mixed
Methods Study Comparing the Abbey Pain Scale and Pain Assessment in
Advanced Dementia Scale. J Gerontol Nurs. 2021 Feb 1;47(2):21–30.
30. Atee M, Hoti K, Hughes JD. Psychometric evaluation of the electronic pain
assessment tool: an innovative instrument for individuals with moderate-to-
severe dementia. Dement Geriatr Cogn Disord. 2018;6845:256–67.
31. Mental Capacity Act. 2005 [Internet]. 2005 [cited 2017 Oct 26]. Available from:
http://www.legislation.gov.uk/ukpga/2005/9/pdfs/ukpga_20050009_en.pdf.
32. Parkinson’s Foundation. Facial Masking [Internet]. 2019 [cited 2019 Aug 28].
Available from: https://www.parkinson.org/Understanding-Parkinsons/
Symptoms/Facial-Masking.
33. Beach PA, Huck JT, Miranda MM, Foley KT, Bozoki AC. Effects of Alzheimer
Disease on the Facial Expression of Pain. Clin J Pain. 2015;32:478–87.
34. Hinkle D, Wiersma W, Jurs S. Applied statistics for the behavioral sciences.
5th ed. Boston: Houghton Mifflin; 2003.
35. Cohen J. A coefficient of agreement for nominal scales. Educ Psychol Meas.
1960 Apr 2;20(1):37–46.
36. Koo TK, Li MY. A Guideline of selecting and reporting intraclass correlation
coefficients for reliability research. J Chiropr Med. 2016;15(2):155–63.
37. Hulin C, Netemeyer R, Cudeck R. Can a Reliability Coefficient Be Too High?
Methodol Stat Concerns Exp Behav Res. 2001;10(1):55–8.
38. Bland JM, Altman DG. Cronbach’s alpha. BMJ. 1997;314:572.
39. Lukas A, Barber JB, Johnson P, Gibson S. Observer-rated pain assessment
instruments improve both the detection of pain and the evaluation of pain
intensity in people with dementia. Eur J Pain. 2013;17(10):1558–68.
40. Feldt KS. The Checklist of Nonverbal Pain Indicators (CNPI). Pain Manag
Nurs. 2000;1(1):13–21.
41. Horgas AL, Nichols AL, Schapson CA, Vietes K. Assessing pain in persons
with dementia: relationships among the non-communicative patient’s pain
assessment instrument, self-report, and behavioral observations. Pain Manag
Nurs. 2007;8(2):77–85.
42. Herr K, Sefcik JS, Neradilek MB, Hilgeman MM, Nash P, Ersek M.
Psychometric Evaluation of the MOBID Dementia Pain Scale in U.S. Nursing
Homes. Pain Manag Nurs. 2019;20(3):253–60.
43. Cohen-Mansfield J. Pain Assessment in Noncommunicative Elderly Persons-
PAINE. Clin J Pain. 2006 Jul;22(6):569–75.
44. Flicker C, Ferris SH, Reisberg B. Mild cognitive impairment in the elderly:
predictors of dementia. Neurology. 1991;41(7):1006–9.
45. Husebo BS, Strand LI, Moe-Nilssen R, Husebo SB, Snow AL, Ljunggren AE.
Mobilization-Observation-Behavior-Intensity- Dementia Pain Scale (MOBID):
Development and validation of a nurse-administered pain assessment tool
for use in dementia. J Pain Symptom Manage. 2007;34(1):67–80.
46. Fuchs-Lacelle S, Hadjistavropoulos T. Development and preliminary
validation of the pain assessment checklist for seniors with limited ability to
communicate (PACSLAC). Pain Manag Nurs. 2004 Mar;5(1):37–49.
47. Sheu E, Versloot J, Nader R, Kerr D, Craig KD. Pain in the elderly validity of
facial expression components of observational measures. Clin J Pain. 2011;
27(7):593–601.
48. Monroe TB, Parish A, Mion LC. Decision factors nurses use to assess pain in
nursing home residents with dementia. Arch Psychiatr Nurs. 2015 Oct;29(5):
316–20.
Publisher’sNote
Springer Nature remains neutral with regard to jurisdictional claims in
published maps and institutional affiliations.
Babicova et al. BMC Geriatrics (2021) 21:337 Page 8 of 8
Content courtesy of Springer Nature, terms of use apply. Rights reserved.

1.
2.
3.
4.
5.
6.
Terms and Conditions
Springer Nature journal content, brought to you courtesy of Springer Nature Customer Service Center GmbH (“Springer Nature”).
Springer Nature supports a reasonable amount of sharing of research papers by authors, subscribers and authorised users (“Users”), for small-
scale personal, non-commercial use provided that all copyright, trade and service marks and other proprietary notices are maintained. By
accessing, sharing, receiving or otherwise using the Springer Nature journal content you agree to these terms of use (“Terms”). For these
purposes, Springer Nature considers academic use (by researchers and students) to be non-commercial.
These Terms are supplementary and will apply in addition to any applicable website terms and conditions, a relevant site licence or a personal
subscription. These Terms will prevail over any conflict or ambiguity with regards to the relevant terms, a site licence or a personal subscription
(to the extent of the conflict or ambiguity only). For Creative Commons-licensed articles, the terms of the Creative Commons license used will
apply.
We collect and use personal data to provide access to the Springer Nature journal content. We may also use these personal data internally within
ResearchGate and Springer Nature and as agreed share it, in an anonymised way, for purposes of tracking, analysis and reporting. We will not
otherwise disclose your personal data outside the ResearchGate or the Springer Nature group of companies unless we have your permission as
detailed in the Privacy Policy.
While Users may use the Springer Nature journal content for small scale, personal non-commercial use, it is important to note that Users may
not:
use such content for the purpose of providing other users with access on a regular or large scale basis or as a means to circumvent access
control;
use such content where to do so would be considered a criminal or statutory offence in any jurisdiction, or gives rise to civil liability, or is
otherwise unlawful;
falsely or misleadingly imply or suggest endorsement, approval , sponsorship, or association unless explicitly agreed to by Springer Nature in
writing;
use bots or other automated methods to access the content or redirect messages
override any security feature or exclusionary protocol; or
share the content in order to create substitute for Springer Nature products or services or a systematic database of Springer Nature journal
content.
In line with the restriction against commercial use, Springer Nature does not permit the creation of a product or service that creates revenue,
royalties, rent or income from our content or its inclusion as part of a paid for service or for other commercial gain. Springer Nature journal
content cannot be used for inter-library loans and librarians may not upload Springer Nature journal content on a large scale into their, or any
other, institutional repository.
These terms of use are reviewed regularly and may be amended at any time. Springer Nature is not obligated to publish any information or
content on this website and may remove it or features or functionality at our sole discretion, at any time with or without notice. Springer Nature
may revoke this licence to you at any time and remove access to any copies of the Springer Nature journal content which have been saved.
To the fullest extent permitted by law, Springer Nature makes no warranties, representations or guarantees to Users, either express or implied
with respect to the Springer nature journal content and all parties disclaim and waive any implied warranties or warranties imposed by law,
including merchantability or fitness for any particular purpose.
Please note that these rights do not automatically extend to content, data or other material published by Springer Nature that may be licensed
from third parties.
If you would like to use or distribute our Springer Nature journal content to a wider audience or on a regular basis or in any other manner not
expressly permitted by these Terms, please contact Springer Nature at
onlineservice@springernature.com