Medical University, Tianjin, China;
†Shanghai Jiao Tong University
School of Nursing, Shanghai, China;
‡Department of Cardiovascular
Surgery, Shanghai Children’s
Medical Center, Shanghai, China;
and§Department of Pediatric
Surgery, Erasmus Medical Center–
Address correspondence to Lily Hsu,
Shanghai Jiao Tong University
Received February 26, 2010;
? 2009 by the American Society for
Pain Management Nursing
Validation of the
Scale and the FLACC
in Chinese Children after
--- Jinbing Bai, RN, MSN,* Lily Hsu, RN, MSN,†
Yan Tang, RN, BSN,‡and Monique van Dijk, RN, PhD§
status of critically ill children. However, the utility of the face, legs,
activity, cry, consolability (FLACC) scale and the COMFORT Behavior
(COMFORT-B) scale have not been extensively explored for children
after cardiac surgery in China. A repeated-observation study was con-
ducted to evaluate the concurrent validity and the sensitivity and spec-
ificity of the COMFORT-B and FLACC scales for pain assessment after
B scale simultaneously at 18 fixed time periods for 3 days. Correlations
among pain assessments were computed to calculate concurrent valid-
ity. Receiver operating characteristic curve analyses identified the
FLACC and COMFORT-B scores that could best discriminate painand no
with FLACC and COMFORT-B pain scores as dependent variables and
disease-related treatment characteristics as predictor variables for con-
ducting the analysis. A total of 170 children (98 boys and 72 girls) were
included. Significantly positive correlations, ranging from 0.31 to 0.86,
were found among the COMFORT-B, FLACC, and VASobs pain assess-
ment instruments. The COMFORT-B and the FLACC scores for children
assessed to be in pain (i.e., VASobs $4), were significantly higher than
scores for children notinpain(VASobs <4). COMFORT-Bshowed good
sensitivity (86%) and specificity (83%) with a cutoff point of 13. FLACC
showed excellent sensitivity (98%) and good specificity (88%) for a cut-
off point of2.TheCOMFORT-B andFLACC scalesseemtobeuseful tools
in pain assessment for Chinese children at postcardiac surgery stages,
cutoff point for pain than originally set for these two instruments.
? 2009 by the American Society for Pain Management Nursing
Pain Management Nursing, Vol -, No - (--), 2010: pp 1-9
Pain is recognized as a subjective and multidimensional
experience and is a concern after surgical and invasive
medical procedures in the critical care setting (Locscr
& Cousins, 1990; Walker & Wagner, 2003). Although
self-report is considered to be the gold standard for
pain assessment (Guo & Wang, 2008; von Baeyer &
Spagrud, 2007), this is not feasible in preverbal
patients. Furthermore, it may not be feasible in
critically ill patients for a number of reasons such as
sedation, mechanical ventilation,
consciousness (Bennett, 2001; Shannon & Bucknall,
Many children experience moderate to severe
pain after cardiac surgery (Hammer, Ramamoorthy,
Cao, Williams, Boltz, Kamra, et al., 2005; Lin, Ma,
Liu, & Li, 2008; Ren, 2006). Repeated or long-term
exposure to pain has negative consequences for
children, including pain sensitivity alteration, neuro-
anatomic abnormalities, and emotional, behavioral,
and learning disabilities (American Academy of
Pediatrics, Committee on Fetus and Newborn and
Section on Surgery, & Canadian Pediatrics Society
and Fetus and Newborn Committee, 2006; Ruda,
Ling, Hohmann, Peng, & Tachibana, 2000). One
would do well, therefore, to identify the pain
status for children who can not report their pain
verbally. In this regard, Anand and Craig proposed
that pain-related behaviors are valuable proxies for
self-report (American Academy of Pediatrics &
American Pain Society, 2001; Anand & Craig,
1996). The use of validated assessment tools may
be helpful to identify such behavior in critically ill
children after major surgery.
Several pain assessment instruments have been
validated for postoperative pain, such as the Face,
Legs, Arms, Cry, Consolability (FLACC) scale devel-
oped by Merkel et al. (1997) and the COMFORT scale
developed by Ambuel et al. (1992). Both the FLACC
and COMFORT scales were recognized as well estab-
lished pain instruments after major surgery in critically
ill children by the Society of Pediatric Psychology’s
Evidence-based Assessment Task Force and the Pediat-
ric Initiative on Methods, Measurement, and Pain
Assessment in Clinical Trials (Cohen, La Greca,
Blount, Kazak, Holmbeck, & Lemanek, 2008; Cohen,
Lemanek, Blount, Dahlquist, Lim, Palermo, McKenna,
et al., 2008; von Baeyer & Spagrud, 2007).
The COMFORT Behavior (COMFORT-B) scale was
derived from the original COMFORT scale by exclud-
ing the two physiologic parameters (heart rate and
mean arterial blood pressure) (van Dijk, Peters, van
Deventer, & Tibboel, 2005). Both the FLACC and
COMFORT-B scales have been widely applied with ex-
cellent reliability and validity for critically ill children
(Manworren & Hynan, 2003; van Dijk, de Boer, Koot,
However, the psychometric properties of these
postoperative pain assessments in Chinese children
have not been evaluated.
Few well established Western pain assessment in-
struments have been translated into Chinese (Zhu, Fu,
Li, Zheng, & Zhang, 2008). Culture can significantly
impact children’s pain behavioral responses, as evi-
denced by the fact that Chinese infants showed differ-
ent behavioral responses than non-Chinese Canadian
infants (Rosmus, Johnston, Chan-Yip, & Yang, 2000).
Also children’s pain behavioral expressions and mani-
festations might vary across countries. Therefore,
validation of a reliable and valid behavioral pain assess-
ment instrument based on Chinese children’s charac-
teristics is imperative for adequate pain management.
The present study explored the utility of the FLACC
and COMFORT-B scales in Chinese children after car-
The specific purposes of this study were: 1) to
identify the concurrent validity of the FLACC and
COMFORT-B scales for pain assessment in Chinese chil-
dren after cardiac surgery; 2) to evaluate the sensitivity,
specificity, and the optimal FLACC and COMFORT-B
scale cutoff scores; and 3) to explore factors that pre-
dict COMFORT-B and FLACC scores.
PATIENTS AND METHODS
A repeated-measures design was conducted in a 47-bed
cardiac intensive care unit (CICU) of a tertiary pediat-
ric care facility in Shanghai with >3,000 cardiac sur-
gery cases annually. To be eligible for the study,
participants had to meet the following criteria: 1) chil-
dren aged 0-7 years with congenital heart disease; 2)
admission to CICU for $2 days after cardiac surgery;
3) no intellectual disability such as Down syndrome;
and 4) not requiring extracorporeal membrane oxy-
genation (ECMO). The hospital’s Research Review
Board approved this study, and informed consent was
obtained from the parents.
Based on the power analysis, the calculated effect
size based on the absolute difference between the pain
scores on the operation day and pain scores on postop-
eration day 1 and 2 demonstrated a moderate effect
(h2¼ 0.06). With power set at 0.80, and the signifi-
cance level at .05, the required sample size calculated
by the GPower calculator was 159 (Erdfelder, Faul, &
Buchner, 1996; Polit & Beck, 2004, pp. 495-502). A
total of 174 patients were enrolled into this study
between September 2008 and January 2009. Two
patients died from disease complications, and two
Bai et al.
received ECMO therapy during the course of the data
collection, leaving a final group of 170 participants in
the data analysis.
Analgesics and Sedatives Treatment
In this CICU, morphine and sufentanil at dosages be-
gin at 5-10 mg/kg/h and 0.5 mg/kg/h, respectively,
are the most frequently used analgesics for children
on continuous ventilatory support after major surgery.
A nonopioid alternative is diclofenac sodium. Sedative
prescription depends on the child’s medical condi-
tion; some children receive combinations of sedative
agents to prevent possible tolerance. Diazepam is
rarely prescribed when children show unstable phys-
iologic parameters or when they are scheduled for
withdrawal of mechanical ventilation. The common
order for sedation is to start with phenobarbital and
continue with phenergan, diazepam, or chloral hy-
drate if necessary. This sedation strategy is thought
to benefit children both on and off ventilatory sup-
port. Figure 1 presents a schedule of the analgesics
and sedatives treatment.
Children who are so agitated that hemodynamic
stability is affected but hard to control by analgesics
and sedatives receive small boluses of vecuronium to
facilitate synchronization with mechanical ventilation.
In a South American cohort, vecuronium was found to
have an effect time of 30-40 minutes (de Almeida,
Filho, & Troster, 2000); a shorter effect time was found
in Chinese children, i.e., 12-16 minutes (Liu, Wang, &
ObservationalVisual Analog Scale (VASobs). The
VASobs is a horizontal continuous 10 cm line with the
anchors ‘‘no pain’’ (0) at the left side and ‘‘worst pain
possible’’ (10) at the right side. Observers place
a mark on the line that indicates the severity of pain ac-
cording to their perception. With a ruler, they read the
number of cm from the left side of the VAS line
that represents the score. It is frequently used as an
observational instrument but may also be used for
self-report. Scores below the cutoff point of 4 are con-
sidered to reflect mild pain to no pain; higher scores
reflect moderate to severe pain requiring analgesic
treatment (Buchholz, Karl, Pomietto, & Lynn, 1998;
van Dijk, Koot, Saad, Tibboel, & Passchier, 2002; Zhu
et al., 2008).
Face, Legs, Activity, Cry, Consolability (FLACC)
scale. The FLACC scale was developed to assess post-
operative pain in 0-7-year-old children (Merkel, Lewis,
Shayevitz, & Malviya, 1997). It incorporates five cate-
gories of behavior, each scored on a 0–2-point scale
so that the total score ranges from 0 to 10. Chen
(2003) has defined total scores of 0-3 as mild or no
pain, 4-7 as moderate, and 8-10 as severe pain.
FLACC has been shown to have good interrater
reliability and validity in pain assessment for young
children outside of China (Merkel et al., 1997; Voepel-
Lewis, Merkel, Tait, Trzcinka, & Malviya, 2002; Willis,
Merkel, Voepel-Lewis, & Malviya, 2003).
The FLACC scale has been translated into Chinese.
The primary investigator received permission to use
FIGURE 1. - Flowchart of the analgesics and sedatives administration.
Pain in Chinese Children after Cardiac Surgery
the original version and the Chinese version from the
original author. Interrater reliability of the Chinese ver-
sion was tested as follows. The primary investigator
and a nursing faculty member with expertise in pediat-
ric intensive care conducted paired observations in
four children who met the sampling criteria and then
independently assigned FLACC scores. The calculated
intraclass correlation coefficient (ICC) for these obser-
vations was 0.84.
COMFORT Behavior (COMFORT-B) scale. The
COMFORT-B scale was derived from the original COM-
FORT scale developed by Ambuel et al. (1992). The
two physiologic dimensions (heart rate and blood pres-
sure) were deleted and an item for nonventilated
patients added. Thus, it contains six behavioral items:
alertness, calmness, respiratory response (in ventilated
patients) or crying (in nonventilated patients), muscle
tone, physical movement, and facial tension. Each item
is scored on a Likert scale from 1 to 5. The total score
thus varies from 6 to 30. Scores of $17 indicate the
need for a pain reducing intervention. This instrument
has been validated for postoperative pain as well as for
sedation management in the pediatric intensive care
unit (Ista, van Dijk, Tibboel, & deHoog, 2005; van
Dijk et al., 2005).
The COMFORT-B scale was originally written in
English, and no Chinese version was available; there-
fore, according to standard procedures (including
and harmonization) the investigator together with sev-
eral experts prepared a translation into Chinese (Wild,
Grove, Martin, Eremenco, McElroy, Verjee-Lorenz,
et al., 2005). Then, to test content validity, three pedi-
atric critical care experts (the head nurse of the CICU,
one nursing faculty member with expertise in pediat-
ric intensive care, and one nursing director with
expertise in oncology pain management) rated each
item on a 3-point Likert scale ranging from 1 (suitable)
to 3 (unsuitable). Scores 2 and 3 reflected that the
item needed some correction. One expert suggested
correcting language expression of the items muscle
tone and facial tension. The content validity index
of COMFORT-BChinesewas excellent at 0.92. The inter-
rater reliability of COMFORT-BChinesewas tested simul-
taneously with FLACC as described above. The ICC of
four paired observations for COMFORT-BChinese was
excellent at 0.98.
The proposed standard of pain assessment requires
routine evaluations of vital signs every 3-6 hours
(American Academy of Pediatrics & American Pain
Society, 2001; van Dijk, Bouwmeester, Duivenvoorden,
Koot, Tibboel, Passchier, et al., 2002). In the present
study, the primary investigator assessed children’s
pain levels using VASobs, COMFORT-BChinese, and
FLACC simultaneously in random order at 2-hour inter-
vals. The set times on the dayofoperation were 3 p.m.,
5 p.m., 7 p.m., and 9 p.m.. The set times on the first
and second postoperation days were 9 a.m., 11 a.m.,
1 p.m., 3 p.m., 5 p.m., 7 p.m., and 9 p.m.. These
time points were selected to avoid major confounding
factors such as hunger, administration of medication,
and sleep. After receiving informed consent from the
parents, the investigator prepared the scoring ques-
tionnaires and took the best position from which the
face and entire body of the child could be observed;
then the child was observed for a full 2 minutes con-
firmed by an alarm clock with ending by gently touch-
ing the arms or legs to determine muscletone (van Dijk
et al., 2005).
The data were analyzed with SPSS, version 15.0 (SPSS,
Chicago, IL, USA). Descriptive statistics consisted of
mean and SD or median and interquartile range (IQR)
for data deviating from normality. The concurrent val-
idity of FLACC and COMFORT-BChinesewas calculated
by Pearson product moment correlation for continu-
ously distributed data with 95% confidence intervals
(CIs). The median of COMFORT-BChineseand FLACC
scores for those assessed not to be in pain (VASobs
<4) and those assessed to be in pain (VASobs $4)
were compared with the Mann-Whitney U test.
The expert opinion of the researcher represented
by VASobs was used as the gold standard against which
cutoff scores for COMFORT-BChineseand FLACC were
evaluated. Cutoff scores are useful for clinical practice,
because they can indicate the level of comfort or pain
that the child experiences and can guide analgesic
treatment. When VASobs pain scores were $4, a value
of 1 was assigned. Values <4 were assigned 0. Different
cutoff scores for COMFORT-BChineseand FLACC were
tested against this classification. Cutoff scores were de-
rived from comparing the combinations of specificity
and sensitivity for different cutoff scores of COM-
FORT-BChineseand FLACC as presented in a receiver op-
erating characteristic (ROC) curve (Jekel, Elmore, &
Katz, 1996). The ROC curve plots the sensitivity on
the y-axis and 1 ? specificity on the x-axis for each pos-
sible cutoff and joins the points.
Multiple linear regression analyses were con-
ducted with the mean COMFORT-BChineseand FLACC
scores as dependent variables. Independent variables
included analgesics given (yes/no), muscle relaxants
given (yes/no), sedative agents given (yes/no), dura-
tion of mechanical ventilation, nature of surgical pro-
cedure(elective vs. palliative), the number of
Bai et al.
congenital heart disease lesions with gender and age as
covariables. The nominal variables were dichotomized
into dummy variables in the analytic equation. Multi-
collinearity was tested by determining the variance in-
A total of 2,815 triple observations (VASobs, COM-
FORT-BChinese, and FLACC) were obtained in 170 chil-
dren: 98 male (57.6%) and 72 female (42.4%). Further
demographics, diagnoses, and surgical procedures
are presented in Table 1 and Table 2.
A hundred thirty-five children were observed 18
times, i.e., on 3 consecutive days; the other 35 patients
were observed 11 times, i.e., on the operation day and
first postoperative day. Even with the same medical
conditions as the 18-times observed patients, the pa-
tients who were observed 11 times recovered and
were discharged from the CICU more quickly than ex-
pected. Analgesic management consisted of continu-
ous opioids for 55.9% (20% morphine and 35.9%
dosage of 12.5 mg by bolus rectal route. Thus, 35.9% of
children did not receive any analgesics after surgery
based on the decisions from the clinical staff. Sedation
management consisted predominantly of intermittent
bolus sedatives. Some children also received small bo-
luses of a muscle relaxant if there were too agitated, ac-
cording to physicians’ orders. In the present study, 237
of 2,815 observations concerned children on muscle
relaxants: 12 observations were done within 30 min-
utes of muscle relaxant usage, 102 within 30-60 min-
utes, and 123 after 1 hour. Table 3 summarizes the
analgesics, sedatives, and muscle relaxants used.
Concurrent validity of FLACC and COMFORT-
(38.8%) and comprised 3.7% of all 2,815 VASobs-
assessed scores. These scores were associated with
a median COMFORT-BChinesescore of 16 (IQR 7-25),
which was significantly higher than the median of 10
(IQR 6-24) associated with VASobs scores <4 by
Mann-Whitney U test (p < .0001). The median FLACC
scores discriminating children in pain (VASobs $4)
and not in pain (VASobs <4) were 5 (IQR 0-9) and 1
(IQR 0-8), respectively, which also indicated significant
difference by Mann-Whitney U test (p < .0001).
(r ¼.51; p ¼ .0001). VASobs had a low correlation with
lation with FLACC (r ¼ .86; p ¼ .0001). No statistically
significant correlations were found between three
pain assessments and pain-related physiologic parame-
ters (heart rate, arterial blood pressure) (p > .05). The
correlations with 95% CI are listed in Table 4.
Sensitivity, Specificity, and ROC Curves of the
FLACC and COMFORT-BChinese
The FLACC and COMFORT-BChinesecutoff values that
best discriminate between pain and no pain (based
Demographic Data of the Participants (n ¼ 170)
Age in months, median (IQR)
Gender (female/male), n
Weight in kg, median (IQR)
LOS in ICU in days, median (IQR)
Length of mechanical ventilation
in hours, median (IQR)
IQR ¼ interquartile range; LOS ¼ length of stay.
Diagnosis ofCongenital HeartDisease (CHD) and
Surgical Procedures (n ¼ 170)
Diagnosis, n (%)*
Ventricular septal defect
Atrial septal defect
Patent ductus arteriosus
Tetralogy of fallot
Transposition of the great arteries
Double-outlet right ventricle
Complete atrioventricular canal
Total anomalous pulmonary venous
Patent foramen ovale
Coarctation of the aorta
Right/left ventricular outflow tract
No. of lesions of CHD, n (%)
Surgical procedures, n (%)
*Some patients had a combination of diagnoses.
Pain in Chinese Children after Cardiac Surgery
on VASobs $4), were determined through ROC curve
analysis. The optimal cutoff score is determined visu-
ally by assessing which score combines maximal sensi-
tivity with optimal specificity. Figure 2 presents the
ROC curve showing sensitivity and specificity results
for cutoff values close to the upper left-hand corner.
Area under the curve (AUC) for FLACC was 0.98 (p ¼
.0001; 95% CI 0.954-0.998). Sensitivity and specificity
for FLACC cutoff scores of $2 were 0.98 and 0.88,
respectively. AUC for COMFORT-BChinese was 0.93
(p ¼ .0001; 95% CI 0.905-0.955). Sensitivity and spec-
ificity for COMFORT-BChinese cutoff scores of $13
were 0.86 and 0.83, respectively.
Factors that predict the FLACC and COMFORT-
Multiple linear regression analysis with mean COM-
FORT-BChinesescores as outcome variable revealed sig-
nificantly higher pain scores for children who were
on mechanical ventilation for a shorter period of time
(p < .001). Lower pain scores were assigned to chil-
dren who received muscle relaxants (p ¼ .025) and an-
algesics (p ¼ .008) after cardiac surgery. Also, the
younger the child, the higher the pain scores (p ¼
.028). The regression analysis with COMFORT-BChinese
as dependent variable explained 37% of the variance
(adjusted R2¼ 0.37). For the FLACC scale, 11% of
the variance (adjusted R2¼ 0.11) in the scores were
predicted by the children’s age (p ¼ .001) and by re-
ceiving a muscle relaxant (p ¼ .021). Younger children
and children who received a muscle relaxant had
higher FLACC scores than older children and children
who did not receive a muscle relaxant (Table 5).
This study describes the validity, sensitivity, and speci-
ficity of two widely used pain tools (FLACC and
COMFORT-B) for children admitted to a CICU in Shang-
hai, China, after cardiac surgery. This study also ex-
assessment outcomes. Both tools showed high sensitiv-
ity and specificity, for the prediction of pain (VASobs
$4), with a recommended optimal cutoff point of 2
for FLACC and 13 for COMFORT-BChinese.
Compared with earlier studies (Chen, 2003; van
Dijk et al., 2005), these relatively low cutoff values may
perhaps be attributed to our sedation management,
which differs from that in Western countries. These
cutoff points could be applied in pain management,
e.g., to design a treatment algorithm that will help
caregivers to decide whether treatment should be
adapted or not.
Differences relate to the number of sedatives pre-
scribed, mode of infusion (bolus in China vs. continu-
ous in Western countries), and use of neuromuscular
Analgesic, Sedatives, and Muscle Relaxant
Usage (N ¼ 170)
Medicationn (%) Dose, median (IQR)
34 (20.0) 13.05 mg/kg/h (5-21)
61 (35.9) 0.50 mg/kg/h (0.2-2.0)
25 (14.7) 12.5 mg (12.5-25)
5 mg/kg (5)
1 mg/kg (1)
0.1 mg/kg (0.1)
0.3 mg/kg (0.2-0.3)
2.0 mg/kg/h (2)
128 (75.3) 0.1 mg/kg (0.1)
IQR ¼ interquartile range.
*Some patients received a combination of drugs.
Correlations (and 95% Confidence Intervals [CIs]) between average scores on pain assessment
instruments (n ¼ 170)
Variabler 95% CIr 95% CIr95% CI
0.162 to 0.436
0.820 to 0.898
?0.078 to 0.221
?0.148 to 0.153
0.392 to 0.615
?0.063 to 0.235
0 to 0
?0.087 to 0.212
?0.098 to 0.202
BP ¼ arterial blood pressure; HR ¼ heart rate.
*p < .01.
Bai et al.
blockers. Lamas et al. (2009) reported that 41% of car-
diac surgery patients in a European setting received
neuromuscular blocking agents (same dosages as in
our study), in contrast to 75% in the present study. Fur-
thermore, sedation and pain assessment are not yet
common practice in China. The high specificities of
both pain tools reduce the risk of administering analge-
sics to patients who do not need these, thus avoiding
possible side effects of the medication.
FLACC, COMFORT-BChinese, and VASobs have sig-
nificant concurrent validity in critically ill children,
with correlation coefficients ranging from moderate
to high (Johansson & Kokinsky, 2009; van Dijk et al.,
2002). Nevertheless, the
COMFORT-BChinese and VASobs was low in the
present study. A possible explanation is that most of
the children received small boluses of sedative agents
and small doses of muscle relaxant to facilitate
analgesic and sedation. Both sedatives and muscle
relaxants could diminish children’s pain-related behav-
iors. Focusing on behavior alone might therefore result
in underestimation of the children’s pain status. Thus,
it would be more acceptable to combine behavioral
observation with a global pain assessment represent-
ing the expert (nurse) opinion, such as the VASobs
(van Dijk et al., 2005; Johansson & Kokinsky, 2009).
Pain assessment during the action of neuromuscular
blocking agents remains
because, e.g., bispectral index monitoring and heart
rate/blood pressure are not validated in children
under these circumstances (Playfor, Jenkins, Boyles,
Choonara, Davies, Haywood, et al., 2007).
COMFORT-BChinese and FLACC scores were not
correlated with heart rate and blood pressure, similarly
to earlier studies (Ista et al., 2005; Lamas et al., 2009;
van Dijk et al., 2005). This is not surprising in view
of the desired hemodynamic stability after cardiac
surgery. These parameters tend to be controlled by
various inotropic medications. Moreover, changes in
vital signs due to pain are short lasting in acute pain
and tend to return toward baseline during prolonged
pain (van Dijk, Peters, van Deventer, & Tibboel, 2005).
The predictors of the mean COMFORT-BChinese
and FLACC scores indicated that there needs to be
more attention paid to the younger children and those
with prolonged mechanical ventilation. The higher
pain scores for the younger children can be explained
as follows. In many health care facilities within main-
land China, pain management for infants by adopting
an evidence-based approach is to be encouraged for
health professionals, and Chinese culture encourages
people to bear their suffering, including pain. These
conditions may explain that some of the children did
not receive routine analgesics after cardiac surgery.
Nevertheless, we found that children who needed pro-
longed ventilatory support commonly received more
analgesics or sedatives.
Several limitations must be addressed. This study was
conducted in one pediatric tertiary center, limiting
generalization. The sample size, though, was large
enough to permit valid conclusions. An additional lim-
itation is related to the fact that environmental factors
(e.g., noise, light) were not controlled and that
unplanned invasive procedures (e.g., received oxygen-
ation, withdrewendotracheal tube, received ultrasonic
nebulization) were performed during data collection.
Furthermore, the fixed observation time points pro-
vided only snapshots of children’s pain level and man-
ifestation of pain. This study was unable to assess the
pain and sedative status before and after the additional
dose of analgesics or sedatives, which was a limitation
to determine the instruments’sensitivity to the change
of pain status.
The FLACC and COMFORT-BChinesescales were both
found to be useful tools for pain detection in this vul-
nerable population, similarly to previous studies of
children <7 years old with critical illnesses. Both
FIGURE 2. - Receiver operating characteristic curves and
cutoff points for COMFORT-BChinese and FLACC when
the level of pain was classified by VASobs for children after
cardiac surgery in CICU. The best cutoff point for COM-
FORT-BChinesewas 13 (sensitivity 86%, specificity 83%)
The area under the curve (AUC) was 93% (p < .01; 95%
CI 0.905-0.955). The best cutoff point for FLACC was 2
(sensitivity 98%, specificity 88%). The AUC was 98%
(p < .01; 95% CI 0.954-0.998).
Pain in Chinese Children after Cardiac Surgery
scales showed excellent sensitivity and specificity
based on VASobs to predict pain during the postcardiac
surgery period. As in an earlier study (Johansson &
Kokinsky, 2009), the results seem to indicate that
FLACC performed better than COMFORT-BChinesefor
pain. Conversely, the COMFORT-BChinese provided
more detailed information about sedation than did
the FLACC scale. Future studies need to further ex-
plore the utility of these pain assessments tools in
non-Western settings in combination with pain and se-
dation treatment algorithms.
The authors thank Sandra Merkel for generously sharing the
The authors also thank Ko Hagoort for editing the manuscript.
Ambuel, B., Hamlett, K. W., Marx, C. M., & Blumer, J. L.
(1992). Assessing distress in pediatric intensive care envi-
ronments: The COMFORT scale. Journal of Pediatric Psy-
chology, 17(1), 95–109.
American Academy of Pediatrics & American Pain Society
(2001). The assessment and management of acute pain in in-
American Academy of Pediatrics, Committee on Fetus and
Newborn and Section on Surgery, & Canadian Pediatrics
Society and Fetus and Newborn Committee (2006). Preven-
tion and management of pain in the neonate: An update.
Pediatrics, 118(5), 2231–2241.
Anand, K. J., & Craig, K. D. (1996). New perspectives on
the definition of pain. Pain, 67(1), 3–6.
Bennett, M. (2001). Pain assessment and management in pe-
diatric intensive care: Part 1. Pediatric Nursing, 13(5), 26–28.
Buchholz, M., Karl, H. W., Pomietto, M., & Lynn, A. M.
(1998). Pain scores in infants: A modified infant pain scale
versus visual analogue. Journal of Pain and Symptom
Management, 15(2), 117–124.
Chen, M. F. (2003). Pain assessment for infants and chil-
dren: FLACC scale. Foreign Medical Sciences: Nursing For-
eign Medical Science, 22(6), 289–290, (in Chinese).
Cohen, L. L., La Greca, A. M., Blount, R. L.,
Kazak, A. E., Holmbeck, G. N., & Lemanek, K. L. (2008).
Introduction to special issue: Evidence-based assessment
in pediatric psychology. Journal of Pediatric Psychology,
Cohen, L. L., Lemanek, K., Blount, R. L., Dahlquist, L. M.,
Lim, C. S., Palermo, T. M., McKenna, K. D., & Weiss, K. E.
(2008). Evidence-based assessment of pediatric pain. Jour-
nal of Pediatric Psychology, 33(9), 939–955.
Multiple Regression Analysis with COMFORT-BChineseand FLACC as Dependent Variables
Variable Unstandardized BStandardized b
p Value 95% CI for B
COMFORT-BChinese(n ¼ 170)
No. of CHD lesions‡
Duration of MV
Muscle relaxant usage†
FLACC (n ¼ 170)
No. of CHD lesions‡
Duration of MV
Muscle relaxant usage†
?0.024 to ?0.001
?0.565 to 0.100
?0.888 to ?0.137
?0.524 to 0.218
?0.356 to 0.436
?0.015 to ?0.008
?0.577 to 0.356
?0.889 to ?0.059
?0.011 to ?0.003
?0.099 to 0.135
?0.196 to 0.068
?0.040 to 0.221
?0.168 to 0.110
0.000 to 0.002
?0.166 to 0.162
0.027 to 0.319
CHD ¼ congenital heart disease; MV ¼ mechanical ventilator.
Adjusted R2¼ .37 for COMFORT-BChineseas dependent variable; adjusted R2¼ .11 for FLACC as dependent variable.
*p < .05.
†No ¼ 0; yes ¼ 1.
‡1 lesion of CHD ¼ 0; $2 lesions ¼ 1.
§Curative surgical procedure ¼ 0; palliative surgical procedure ¼ 1.
Bai et al.
de Almeida, J. F. L., Filho, W. J. K., & Troster, E. J. (2000).
Neuromuscular blockade in children. Revista do Hospital
das Clı ´nicas, 55(3), 105–110.
Erdfelder, E., Faul, F., & Buchner, A. (1996). GPower: A
general power analysis program. Behavior Research
Methods, Instruments, Computers, 28, 1–11.
Guo, Y. M., & Wang, C. (2008). Progress, assessment and
development tendency for postoperative analgesia (in
Chinese). Guangdong Medical Journal, 29(2), 338–340.
Hammer, G. B., Ramamoorthy, C., Cao, H., Williams, G. D.,
analgesia after spinal blockade in infants and children un-
dergoing cardiac surgery. Anesthesia & Analgesia, 100(5),
Ista, E., van Dijk, M., Tibboel, D., & de Hoog, M. (2005).
Assessment of sedation levels in pediatric intensive care
patients can be improved by using the COMFORT Behavior
scale. Pediatric Critical Care Medicine, 6(1), 58–63.
Jekel, J. F., Elmore, J. G., & Katz, D. L. (1996). Epidemiol-
ogy biostatistics and preventive medicine. Philadelphia, PA:
Johansson, M., & Kokinsky, E. (2009). The COMFORT
behavioral scale and the modified FLACC scale in pediatric
intensive care. Journal of the British Association of Critical
Care Nurses, 14(3), 122–130.
Lamas, A., Lopez-Herce, J., Sancho, L., Mencia, S.,
of the level of sedation in children after cardiac surgery.
Annals of Thoracic Surgery, 88, 144–150.
Lin, Y., Ma, J. X., Liu, R. M., & Li, H. F. (2008). Pain assess-
ment and nursing for critically ill children (in Chinese).
Journal of Nursing, 15, 61–62.
Liu, J. Z., Wang, C., & Hu, J. Q. (2002). The influence of
children’s nutrition status on vecuronium (in Chinese).
Chinese Journal of Anesthesiology, 22(7), 429–430.
Locscr, J. D., & Cousins, M. J. (1990). Contemporary pain
management. Medical Journal of Australia, 153(4), 208–
Manworren, R. C. B., & Hynan, L. S. (2003). Clinical vali-
dation of FLACC: Preverbal patient pain scale. Pediatric
Nursing, 29, 140–146.
Merkel, S. I., Lewis, T. V., Shayevitz, J. R., & Malviya, S.
(1997). The FLACC: A behavioral scale for scoring postoper-
ative in young children. Pediatric Nursing, 23(3), 293–297.
Playfor, S., Jenkins, I., Boyles, C., Choonara, I., Davies, G.,
Haywood, T., Hinson, G., Mayer, A., Morton, N., Ralph, T., &
Wolf, A. (2007). Consensus guidelines for sustained neuro-
muscular blockade in critically ill children. Pediatric Anes-
thesia, 17(9), 881–887.
Polit, D. E., & Beck, C. T. (2004). Nursing research: Prin-
ciples and methods, (7th ed.) Philadelphia, PA: Lippincott
Williams & Wilkins.
Ren, M. M. (2006). Comfort nursing for children post
open-heart surgery(in Chinese). Journal of Medical Forum,
Rosmus, C., Johnston, C. C., Chan-Yip, C., & Yang, F.
(2000). Pain response in Chinese and nonChinese Canadian
infants: Is there a difference? Social Science & Medicine, 51
Ruda, M. A., Ling, Q. D., Hohmann, A. G., Peng, Y. B., &
Tachibana, T. (2000). Altered nociceptive neuronal circuits
after neonatal peripheral inflammation. Science, 289(5479),
Shannon, K., & Bucknall, T. (2003). Pain assessment in
critical care: What have welearntfrom research. Intensive &
Critical Care Nursing, 19(3), 154–162.
van Dijk, M., Bouwmeester, N. J., Duivenvoorden, H. J.,
Koot, H. M., Tibboel, D., Passchier, J., & de Boer, J. B.
(2002). Efficacy of continuous versus intermittent mor-
phine administration after major surgery in 0–3-year-old
infants: A double-blind randomized controlled trial. Pain,
van Dijk, M., de Boer, J. B., Koot, H. M., Passchier, J.,
Tibboel, D., & Duivenvoorden, H. J. (2000). The reliability,
stability and validity of the COMFORT scale as a postopera-
tive pain instrument in 0 to 3-year-old infants. Pain, 84,
van Dijk, M., Koot, H. M., Saad, H. H., Tibboel, D., &
Passchier, J. (2002). Observational visual analog scale in
pediatric pain assessment: Useful tool or good riddance?
Clinical Journal of Pain, 18(5), 310–316.
van Dijk, M., Peters, W. B., van Deventer, P., & Tibboel, D.
(2005). The COMFORT Behavior scale: A tool for assessing
pain and sedation in infants. American Journal of Nursing,
Voepel-Lewis, T., Merkel, S., Tait, A. R., Trzcinka, A., &
Malviya, S. (2002). The reliability and validity of the face,
legs, activity, cry, consolability observational tool as a mea-
sure of pain in children with cognitive impairment. Anes-
thesia & Analgesia, 95, 1224–1229.
of observational (behavioral) measures of pain for children
and adolescents aged 3 to 18 years. Pain, 127, 140–150.
Walker, P. C., & Wagner, D. S. (2003). Treatment of pain in
pediatric patients. Journal of Pharmacy Practice, 16,
Wild, D., Grove, A., Martin, M., Eremenco, S., McElroy, S.,
Verjee-Lorenz, A., & Erikson, P. (2005). Principles of good
practice for the translation and cultural adaptation process
for the patient-reported outcomes (PRO) measures: Report
of the ISPOR Task Force for translation and cultural adapta-
tion. Value Health, 8(2), 94–104.
Willis, M. H., Merkel, S. I., Voepel-Lewis, T., & Malviya, S.
(2003). FLACC behavioral pain assessment scale: A compar-
ison with the child’s self-report. Pediatric Nursing, 29(3),
Zhu, J. H., Fu, W., Li, Z. L., Zheng, X. P., & Zhang, Y. F.
(2008). The status Quo and the influencing factors of the
postoperative pain assessment in children (in Chinese).
Journal of Nursing Science, 23(6), 71–74.
Pain in Chinese Children after Cardiac Surgery