External quality assessment for enterovirus 71 and coxsackievirus A16 detection by reverse transcription-PCR using armored RNA as a virus surrogate.
ABSTRACT Three armored RNAs (virus-like particles [VLPs]) containing target sequences from enterovirus 71 (EV71) and coxsackievirus A16 (CA16) and a pan-enterovirus (pan-EV) sequence were constructed and used in an external quality assessment (EQA) to determine the performance of laboratories in the detection of EV71 and CA16. The EQA panel, which consisted of 20 samples, including 14 positive samples with different concentrations of EV and either EV71 or CA16 armored RNAs, 2 samples with all 3 armored RNAs, and 4 negative-control samples (NaN(3)-preserved minimal essential medium [MEM] without VLPs), was distributed to 54 laboratories that perform molecular diagnosis of hand, foot, and mouth disease (HFMD) virus infections. A total of 41 data sets from 41 participants were returned; 5 (12.2%) were generated using conventional in-house reverse transcription-PCR (RT-PCR) assays, and 36 (87.8%) were generated using commercial real-time RT-PCR assays. Performance assessments of laboratories differed; 12 (29.3%) showed a need for improvement. Surprisingly, 4 laboratories were unable to detect EV71 RNA in any samples, even those containing the highest concentration of 10(7) IU/ml. Furthermore, the detection sensitivity for EV71 among all laboratories (82.1%) was substantially lower than that for EV (97.4%) or CA16 (95.1%). Overall, the results of the present study indicate that EQA should be performed periodically to help laboratories monitor their ability to detect HFMD viruses and to improve the comparability of results from different laboratories.
- SourceAvailable from: jcm.asm.org[show abstract] [hide abstract]
ABSTRACT: The widespread use of sensitive assays for the detection of viral and cellular RNA sequences has created a need for stable, well-characterized controls and standards. We describe the development of a versatile, novel system for creating RNase-resistant RNA. "Armored RNA" is a complex of MS2 bacteriophage coat protein and RNA produced in Escherichia coli by the induction of an expression plasmid that encodes the coat protein and an RNA standard sequence. The RNA sequences are completely protected from RNase digestion within the bacteriophage-like complexes. As a prototype, a 172-base consensus sequence from a portion of the human immunodeficiency virus type 1 (HIV-1) gag gene was synthesized and cloned into the packaging vector used to produce the bacteriophage-like particles. After production and purification, the resulting HIV-1 Armored RNA particles were shown to be resistant to degradation in human plasma and produced reproducible results in the Amplicor HIV-1 Monitor assay for 180 days when stored at -20 degreesC or for 60 days at 4 degreesC. Additionally, Armored RNA preparations are homogeneous and noninfectious.Journal of Clinical Microbiology 01/1999; 36(12):3590-4. · 4.07 Impact Factor
- [show abstract] [hide abstract]
ABSTRACT: The aims of this study were the establishment of a WHO International standard for HCV RNA for nucleic acid amplification technology (NAT) assays and the determination of the HCV RNA content of the candidate standard. Twenty-two laboratories evaluated three candidate materials (two lyophilised, AA and BB, which were derived from the same source and one a liquid preparation, CC). All samples were HCV genotype 1 with a concentration of approximately 10(5) genome equivalents/ml. The methods used included the Roche Amplicor assay (version 1), Chiron Quantiplex (bDNA) assay, Organon Teknika NASBA assay, Transcription Mediated assay and various in-house assays, using single or nested primers. There was reasonable agreement between the overall mean NAT detectable units/ml obtained by the different assays except for some of the in-house assays using single primers which gave substantially lower estimates. These titres were 5.0 log10 for samples AA and BB and 4.6 log10 for sample CC. Sample AA was accepted as the candidate standard and assigned a titre of 10(5) international units (IU)/ml. The International Standard consists of a batch of vials each containing 50,000 IU/vial. Preliminary studies indicated that the material is stable at +4 degrees C and +20 degrees C for up to 200 days.Vox Sanguinis 02/1999; 76(3):149-58. · 2.85 Impact Factor
- [show abstract] [hide abstract]
ABSTRACT: The genetic and phylogenetic characteristics of human enterovirus 71 (EV71) and coxsackievirus A16 (CA16) sampled from children with hand, foot, and mouth disease in Shenzhen, People's Republic of China, over a 6-year period (1999 to 2004) were examined with reverse transcription-PCR and DNA sequencing. Out of 147 stool specimens, 60 showed positive signals when screened with EV71- and CA16-specific primers. EV71 was identified in 19 specimens, and CA16 was identified in 41 specimens; coinfection by EV71 and CA16 was not observed. Phylogenetic analysis of all EV71 strains isolated from the mainland Chinese samples established C4 as the predominant genotype. Only one other known strain (3254-TAI-98; AF286531), isolated in Taiwan in 1998, was identified as belonging to genotype C4. Phylogenetic analysis of CA16 strains allowed us to identify three new genetic lineages (A, B, and C), with lineage C recently predominating in Asian countries, such as the People's Republic of China, Malaysia, and Japan. These new observations indicate that CA16 circulating in the People's Republic of China is genetically diverse, and additional surveillance is warranted.Journal of Clinical Microbiology 09/2005; 43(8):3835-9. · 4.07 Impact Factor
JOURNAL OF CLINICAL MICROBIOLOGY, Oct. 2011, p. 3591–3595
Copyright © 2011, American Society for Microbiology. All Rights Reserved.
Vol. 49, No. 10
External Quality Assessment for Enterovirus 71 and Coxsackievirus
A16 Detection by Reverse Transcription-PCR Using Armored
RNA as a Virus Surrogate?†
Liqiong Song,1,2Shipeng Sun,1,2Bo Li,3Yang Pan,1,2Wenli Li,1,2Kuo Zhang,2and Jinming Li2*
Graduate School, Peking Union Medical College, Chinese Academy of Medical Sciences, Beijing, People’s Republic of China1;
National Center for Clinical Laboratories, Beijing Hospital, Beijing, People’s Republic of China2; and Undergraduate School,
College of Liberal Arts and Sciences, University of Colorado Denver, Denver, Colorado3
Received 7 April 2011/Returned for modification 29 April 2011/Accepted 12 August 2011
Three armored RNAs (virus-like particles [VLPs]) containing target sequences from enterovirus 71
(EV71) and coxsackievirus A16 (CA16) and a pan-enterovirus (pan-EV) sequence were constructed and
used in an external quality assessment (EQA) to determine the performance of laboratories in the
detection of EV71 and CA16. The EQA panel, which consisted of 20 samples, including 14 positive samples
with different concentrations of EV and either EV71 or CA16 armored RNAs, 2 samples with all 3 armored
RNAs, and 4 negative-control samples (NaN3-preserved minimal essential medium [MEM] without
VLPs), was distributed to 54 laboratories that perform molecular diagnosis of hand, foot, and mouth
disease (HFMD) virus infections. A total of 41 data sets from 41 participants were returned; 5 (12.2%)
were generated using conventional in-house reverse transcription-PCR (RT-PCR) assays, and 36 (87.8%)
were generated using commercial real-time RT-PCR assays. Performance assessments of laboratories
differed; 12 (29.3%) showed a need for improvement. Surprisingly, 4 laboratories were unable to detect
EV71 RNA in any samples, even those containing the highest concentration of 107IU/ml. Furthermore, the
detection sensitivity for EV71 among all laboratories (82.1%) was substantially lower than that for EV
(97.4%) or CA16 (95.1%). Overall, the results of the present study indicate that EQA should be performed
periodically to help laboratories monitor their ability to detect HFMD viruses and to improve the
comparability of results from different laboratories.
Hand, foot, and mouth disease (HFMD) is an infectious
disease commonly diagnosed in young children and character-
ized by mucocutaneous papulovesicular rashes on the hands,
feet, mouth, and buttocks. Human enterovirus 71 (EV71) and
coxsackievirus A16 (CA16) are the principal causative agents
of HFMD. EV71 is of special concern, because it is more often
associated with major outbreaks and causes complications of
greater severity and higher rates of mortality than other en-
teroviruses. In contrast, CA16-associated HFMD has a milder
outcome and is accompanied by a much lower incidence of
severe complications (7, 16, 21).
HFMD is an important public health concern worldwide
(20). Since 1997, several large epidemics of HFMD have been
reported in the Asia-Pacific region, especially in Southeast
Asia, including Malaysia (2), Taiwan (24), and Singapore (5).
In March 2008, an EV71-associated HFMD outbreak occurred
in Fuyang City, Anhui Province, China, and subsequently
spread rapidly and extensively across the entire country; a total
of 488,955 HFMD cases, 126 of which were fatal, were re-
ported nationwide in 2008 (28).
As no vaccine or antiviral drug is currently available, early
and rapid detection is critical for HFMD prevention and con-
trol (30). At present, the causative agents of HFMD can be
effectively diagnosed by the detection of infectious virus, viral
antigens, viral genomic RNA, or antiviral antibodies (11, 17,
25, 27). Due to their speed, sensitivity, and specificity, reverse
transcription-PCR (RT-PCR)-based molecular diagnostic as-
says are increasingly used to detect EV71 and CA16 RNA for
HFMD diagnosis (8, 14, 15). Various commercial and in-house
assays are currently available. Due to the limited availability of
well-characterized reference reagents, most HFMD detection
assays lack critical standardization of performance, which
causes difficulties in comparing results between laboratories
and consequently complicates the clinical interpretation of lab-
oratory results for HFMD.
To date, many biological materials, including live virus, in-
activated virus, and naked RNA, have been used as controls for
the molecular diagnosis of RNA viruses (1, 19). However, each
of these materials has some inherent disadvantages, such as the
biosecurity risk of live virus, the risk of residual activity of
inactivated virus, and the inability of clinicians to evaluate
sample processing and RNA extraction and the relative insta-
bility of naked RNA. A potential alternative is armored RNA,
a noninfectious and quantifiable synthetic substitute for live or
inactivated RNA virus that can be spiked into clinical speci-
mens without risking degradation, thus enabling simultaneous
monitoring of nucleic acid extraction efficiency and the ampli-
fication process of the detection assay (3, 12, 18).
In this study, we constructed 3 types of armored RNAs
carrying genomic sequence fragments from EV71 and CA16:
one RNA specific for EV71, one RNA specific for CA16, and
* Corresponding author. Mailing address: National Center for Clin-
ical Laboratories, Beijing Hospital, 1 Dahua Road, Dongdan, Beijing
100730, People’s Republic of China. Phone: 86-10-58115053. Fax: 86-
10-65212064. E-mail: email@example.com.
† Supplemental material for this article may be found at http://jcm
?Published ahead of print on 24 August 2011.
one RNA designed to detect all EV serotypes. Moreover, a
pioneering national external quality assessment (EQA) study
was organized within the nationwide HFMD reference labo-
ratory network to evaluate the accuracy of EV71 and CA16
PCR assays used in laboratories in China.
MATERIALS AND METHODS
Preparation of armored RNAs. Three gene fragments were amplified by
RT-PCR using EV71 and CA16 RNA as templates. Both EV71 RNA and
CA16 RNA were kindly provided by the Beijing Center for Disease Preven-
tion and Control (CDC). The EV71 5? untranslated region (5?UTR) pack-
aged into virus-like particles (EV-VLPs) is a pan-enterovirus sequence that is
conserved among all enteroviruses. The primers used in this study are de-
scribed in Table S1 in the supplemental material. As described in previous
studies (13, 26), inclusion of a hepatitis C virus (HCV) 5?UTR in the chimeric
armored RNA sequence allows the ready assignment of an international unit
value to the chimeric armored RNA for quantitative detection. The HCV
5?UTR sequence was amplified from a pNCCL-HCV plasmid (constructed by
our laboratory) and ligated to the EV 5?UTR by overlapping-extension PCR.
Therefore, 3 armored RNAs, AR-HCV-EV(UTR)-994b (EV-VLP), AR-
EV71(VP)-2557b (EV71-VLP), and AR-CA16(VP)-2601b (CA16-VLP),
were generated by the conventional armored RNA technique (18) and further
optimized as previously described in a report from our laboratory (29). The
three armored RNAs were confirmed to contain the target sites for all PCR
assays evaluated in this study.
EV-VLP armored RNA was calibrated using the WHO HCV RNA Interna-
tional Standard (version 3.0 [http://www.nibsc.ac.uk/documents/ifu/06-100.pdf]),
and the other 2 armored RNAs (EV71-VLP and CA16-VLP) were quantified
using calibrated EV-VLP as a standard. Here, the armored RNA (EV-VLP)
content data are expressed in international units per milliliter based on the WHO
HCV RNA International Standard (22).
Temperature and time stability assessments were performed for the three
armored RNAs prepared in the study. The quantified armored RNAs were
diluted with NaN3-preserved minimal essential medium (MEM) to yield 103and
105IU/ml dilutions. For each dilution, a single batch was separated into 50
aliquots in individual time point samples of 0.5 ml. Two samples were stored at
?80°C as controls. The remaining 48 samples were divided into four groups (12
samples/group) and incubated at 37°C, 25°C, 4°C, and ?20°C, respectively. At
each of the time points of 1 week, 2 weeks, 3 weeks, 4 weeks, 8 weeks, and 6
months, two samples of each group were removed and stored at ?80°C until the
completion of the experiment. All of the 50 samples were quantified in a single
procedure using an enterovirus nucleic acid detection kit (real-time RT-PCR
assay; Guangzhou Daan Gene Co., Ltd., Guangzhou, China).
In addition, two aliquots of the 103IU/ml and 105IU/ml dilutions were
subjected to 5 freeze-thaw cycles and quantified using the enterovirus nucleic
acid detection kit.
Performance verification of VLP in two different matrices. Two of the VLP
mixtures (EV-EV71-VLP and EV-CA16-VLP [EV-VLP mixed with EV71-VLP
and with CA16-VLP, respectively, at a copy number ratio of 2:1:1]) were spiked
in serial 10-fold dilutions into MEM and pooled negative throat swab fluid,
respectively, at dilutions from 103IU/ml to 106IU/ml. Two panels that included
7 positive samples and 2 negative controls were prepared. After nucleic acid
extraction using a QIAmp viral RNA minikit (Qiagen, Hilden, Germany), the
two panel samples were tested for each of the aforementioned three targets in
duplicate experiments in a single run by using the enterovirus nucleic acid
detection kit. Finally, the cycle threshold (CT) values determined for each target
in the two panels were compared by a paired t test (see Table S2 in the supple-
Participants. The laboratories that perform HFMD molecular diagnosis in
China were invited to participate in the EQA study. The invitees are all members
of the Chinese Laboratory Network of HFMD Diagnosis, which is composed of
municipal and provincial laboratories. These laboratories receive training and
evaluation from the national Chinese Centers for Disease Control (CDC) lab-
oratory annually, take responsibility for testing and surveillance of HFMD in
their respective districts, and submit the surveillance data and 10 positive strains
to the national laboratory each month during periods of peak HFMD prevalence.
Proficiency panel. The panel samples were designed and coded as indicated
in Table 1. Tenfold serial dilutions of the 3 armored RNAs were made using
NaN3-preserved MEM. The final panel consisted of a set of 16 positive
samples and four NaN3-preserved MEM samples containing no VLPs for use
as negative controls. Aliquots of 500 ?l were assigned code numbers and
stored at ?20°C until distribution. Before distribution to the participants, the
panel samples were tested using an in-house RT-PCR assay recommended
by the Chinese CDC (http://www.chinacdc.cn/n272442/n272530/n3479265/n3479308
/appendix/fujian1%20shouzukoubiaobencaijijijiancejishufangan.pdf) to confirm the
positivity or negativity of the sample results.
Since the armored RNA was stable at higher temperatures, samples and
detailed instructions were sent by express shipping at ambient temperature
(approximately 10°C to 20°C) to 54 laboratories, and it took less than 1 week for
them to reach those destinations. The recipients were requested to submit the
results and other information on the assay details (RT-PCR method and RNA
extraction procedure) within 6 weeks of receiving the panel.
Evaluation of results. Results were scored as optimal (20/20 [100%] correct
responses), acceptable (at least 18 [?90%] correct responses), or improvable
(fewer than 18 [?90%] correct responses). Scored results were released to
participants in an anonymous manner.
Statistical analysis. Data collected were entered into a spreadsheet in Mi-
crosoft Excel (Microsoft Corp., Bellingham, WA) and analyzed using SPSS 13.0
for Windows. The rates of correct responses for EV-, EV71-, and CA16-positive
samples were compared using Pearson’s chi-square test, which, together with
Fisher’s exact test, was also used to compare the sensitivity and specificity of
participating laboratory PCR assays versus those of the Chinese CDC RT-PCR
assay performed in the laboratory of one of the coauthors (J.I.) of the present
study. Paired t tests were performed to compare the cycle threshold values of
VLPs determined using two different specimen matrices. A P value of ?0.05 was
considered statistically significant.
Construction of armored RNAs. Armored RNAs for EV,
EV71, and CA16 were constructed successfully and were con-
firmed by sequencing to contain the corresponding full-length
target sequence of all available assays for HFMD. Quantifica-
tion using the WHO HCV international RNA standard yielded
a concentration of 1.5 ? 1012IU/ml. As expected, all 3 types of
TABLE 1. EQA panel description and composite laboratory score
for each panel member
No. of correct results
reported/total no. of
aEqual copy numbers of EV-VLP were mixed with EV71-VLP or CA16-VLP
as indicated, except for EV-EV71-VLP and EV-CA16-VLP, which were mixed
with EV71-VLP and CA16-VLP, respectively, at a copy number ratio of 2:1:1.
The EV-EV71 and EV-CA16 samples were designed as specificity controls for
bThe target concentrations of the panel samples were selected to range from
103to 107IU/ml, closely matching the concentration ranges that were detected
in clinical samples as reported by Chang et al. (6).
3592SONG ET AL.J. CLIN. MICROBIOL.
armored RNA exhibited strong resistance to both RNase and
At a concentration of 105IU/ml, the three armored RNAs
were stable in NaN3-preserved MEM for 2 weeks at 37°C, 4
weeks at 25°C, 2 months at 4°C, and more than 6 months at
?20°C. At a concentration of 103IU/ml, reduced stabilities
were observed (1 week at 37°C, 2 weeks at 25°C, 4 weeks at
4°C, and more than 2 months at ?20°C). Stability was defined
as a decrease of no more than 0.5 log10IU/ml compared to the
control tube concentrations stored at ?80°C for the entire
duration. The current stability findings are consistent with
those of prior reports (3, 12, 18, 26, 29).
Performance comparison between armored RNAs in MEM
and clinical sample matrices. There were no significant differ-
ences between the two panels in the cycle threshold (CT) val-
ues determined for each of three targets (P ? 0.05) (see Table
S2 in the supplemental material), consistent with previous
studies (4, 12).
Laboratory PCR methods and EQA performance. A total of
54 laboratories included in the national HFMD detection lab-
oratory network were invited to participate in this EQA study.
Among them, 41 laboratories replied with their results as re-
quested, with 1 data set from each participant. The response
rate was 75.9%.
As shown in Table 2, laboratories used various methods,
including five monoplex commercial real-time RT-PCR
kits (PCR assays A to E), one triplex commercial real-time RT-
PCR assay (PCR assay F), and one in-house conventional RT-
PCR assay recommended by the Chinese CDC (http://www
detect EV71 and CA16. Of the 41 data sets, 5 (12.2%) were
generated using conventional in-house assays and 36 (87.8%)
were produced by commercial real-time RT-PCR assays. The
performance results differed substantially among all participating
laboratories and among the laboratories using the same PCR
assay (Table 2). Generally, laboratories using commercial
assay B and the in-house PCR performed better than
laboratories using assay A or C. Laboratories using assay D
performed very poorly. Four laboratories, 2 using assay A (of
the total of 17 laboratories using assay A) and both of the
laboratories using assay D, failed to detect EV71 in any
samples. Evaluations of the cumulative data according to
comparisons of assay results (Table 3) support these findings,
showing that the sensitivity of assay A for detection of EV71
was 74.5% (P ? 0.05) and that assay D failed to detect any
samples positive for EV71. Additionally, assay D showed
decreased sensitivity for CA16. Only 31.7% (13/41) of the
laboratories met criteria for optimal performance; 16 (39%) of
the 41 participants showed acceptable performance, and the
results from 12 (29.3%) of 41 laboratories indicated a need for
improvement in HFMD diagnosis.
We next assessed the sensitivities of detection for the 3
targets (EV, EV71, and CA16) (Table 4). The sensitivity for
EV71 (82.1%) was much lower than that for EV (97.4%) or
CA16 (95.1%) (P ? 0.05), but no obvious differences between
the sensitivities for EV and CA16 were observed (P ? 0.05). As
expected, assay accuracy declined with decreasing concentra-
tion, and most of the detection failures occurred at the 1 ? 103
IU/ml target concentration. At that target concentration, the
rates of detection were 90.2% for EV, 59.8% for EV71, and
84.1% for CA16.
All of the false-negative results were reported by 23 lab-
oratories, and the overall proportion of false-negative re-
sults was 101/1,394 (7.3%). Considering that over 40% (17/
41) of laboratories in the EQA program used assay A, the
lower sensitivity of that assay for EV71 may contribute to
false-negative results during routine testing of suspected
TABLE 2. Performance of 41 laboratories participating
in the EQA study
No. of correct
aQIAamp, QIAamp viral RNA minikit (Qiagen) (n ? 36 ?87.8%?); RNeasy,
RNeasy minikit (Qiagen) (n ? 2 ?4.9%?); TRIzol (Invitrogen) (n ? 3 ?7.3%?).
bA, B, C, D, E, and F represent 6 commercial TaqMan real-time RT-PCR kits
for the detection of hand, foot, and mouth disease. A, Kinghawk Pharmaceutical,
Beijing, China; B, Guangzhou Daan Gene Co., Ltd., Guangzhou, China; C,
Promega Corporation, Fitchburg, WI; D, Bioperfects Technologies, Shanghai,
China; E, Beijing Ipe-bio Technologies Co., Ltd., Beijing, China; F, Beijing
Sunbiostar Gene Technologies Co., Ltd., Beijing, China. A to E are monoplex
assays, and F is a triplex assay. In-house, in-house conventional RT-PCR assay
for HFMD recommended by the Chinese CDC.
cThe laboratory (n ? 4) failed to detect EV71 in any of the samples.
dThe laboratory (n ? 14) returned one false-positive result.
eThe laboratory (n ? 5) returned two false-positive results.
VOL. 49, 2011EXTERNAL QUALITY ASSESSMENT OF EV71 AND CA163593
HFMD cases. The performance of assay D was unacceptable
during this EQA, as the two laboratories that used that assay
had scores of 30 and 55 (Table 2) and the assay failed to
detect any samples containing EV71 and also had lower
sensitivity for CA16 (Table 3).
A total of 19 (1.8%) false-positive results (of the total of
1,066 results) were reported in the study. The individual spec-
ificities of detection for EV, EV71, and CA16 were 157/164
(95.7%), 442/451 (98%), and 448/451 (99.3%), respectively,
with no significant differences found among the 3 targets.
There were also no statistical differences observed in specificity
comparisons of each commercial assay versus the in-house
assay (Table 3).
This study was the first national EQA program designed to
evaluate molecular diagnosis of EV71 and CA16 by the use of
armored RNAs as virus surrogates. Our data demonstrate that
armored RNA serves as a robust and stable alternative to
infectious or inactivated virus in proficiency programs.
The 41 laboratories that participated in this EQA used six
different TaqMan RT-PCR assays and one conventional RT-
PCR assay. Our results in this study were not completely in
agreement with those of previous EQA studies (9, 10), which
reported that commercial real-time PCR technologies exhib-
ited better sensitivity than conventional PCR. The less-than-
perfect scores observed in this EQA study were likely due to a
combination of poor assay performance (in particular, com-
mercial assays A and D) and poor laboratory proficiency (con-
tamination observed in 14 laboratories). However, several
commercial assays performed well and represent a standard-
ized method that would facilitate interpretation and compara-
bility of results among different laboratories.
In this EQA, a triplex commercial real-time RT-PCR assay
(kit F) was used by only one participant. Our communications
with all participants revealed that most laboratories prefer
single assays over multiplex methods, as they indicated that
single assays customarily produce more sensitive and reliable
results. The inclusion of multiple templates in a single reaction
mixture has the potential to reduce the sensitivity of multiplex
RT-PCR assays due to competition of reagents. Recently, ac-
curate multiplex RT-PCR assays that simultaneously distin-
guish among EV71, CVA16, and other enteroviruses have
been described (8, 23).
Laboratory use of commercial RT-PCR assay A in this EQA
resulted in a wide range of scores, from 50 to 100%. Poor
sensitivity for EV71 was the primary factor in less-than-perfect
scores (Table 3). Further investigation of assay A revealed that
higher rates of false-negative results were associated with spe-
cific lots of the EV71-specific reagents. While assay D was used
by only two participants in this study, its false-negative rate for
the EV71 samples reached a startling 100%. More studies are
required to determine whether there are some subgenotypes of
EV71 virus that are out of the detection scope of this assay.
The lower sensitivity of some commercial assays for EV71
TABLE 3. Comparison of sensitivity and specificity data between different commercial and in-house assays
EV detectionEV71 detectionCA16 detection
No. of correct positive
results/total no. of
No. of correct negative
results/total no. of
No. of correct postivie
results/total no. of
No. of correct negative
results/total no. of
No. of correct positive
results/total no. of
No. of correct negative
results/total no. of
Total41 639/656 (97.4)157/164 (95.7)303/369 (82.1)442/451 (98) 351/369 (95.1)448/451 (99.3)
aA, B, C, D, E, F, and In-house represent the same assays as described for Table 2.
bStatistically significant result compared to in-house assay result (P ? 0.0083). The criteria of test ? were adjusted using the Bonferroni method (22a). ? ?
0.05/(group no. ? 1) ? 0.0083.
TABLE 4. Summary of results for the 4 types of target
specimens in the EQA
No. of positive
samples tested (%)
Total no. of positive
samples/total no. of
samples tested (%)
Negative04 12b/164 (7.9)
aEV71 and CA16 were used as specificity controls for each other. False-
positive results for EV (n ? 7) and EV71 (n ? 6) were present in 12 negative
samples. Negative sample 11 was simultaneously contaminated by EV and EV71
in laboratory 20. No false-positive CA16 results were reported for negative
bBecause positive samples contained two or three targets, the total number of
replicates exceeds the number of samples tested (n ? 20).
3594 SONG ET AL.J. CLIN. MICROBIOL.
indicated an urgent need to improve kit performance charac-
teristics, internal quality control by the manufacturers, and
external quality assurance in laboratories. In addition, false-
positive results in some of the laboratories indicated the need
for improved laboratory practices and quality management.
The present report emphasizes that EQA is a very important
tool for assessing the quality of diagnostic laboratory tests.
In summary, 3 armored RNAs for HFMD viruses were suc-
cessfully constructed and used in a nationwide EQA in China.
The results of this first nationwide EQA not only verified the
feasibility of the use of those armored RNAs to serve as con-
trol samples but also highlighted a series of problems regarding
HFMD diagnosis, such as the low sensitivity exhibited by some
commercial assays and the overall poor performance of some
participating laboratories. An EQA should be performed pe-
riodically to help laboratories monitor their ability to detect
HFMD viruses and to improve the concordance of results from
different laboratories. In addition, given the adaptability of
RNA viruses, more HFMD viruses with a wider range of ge-
notypes should be included in future EQA.
This study was supported by the National Natural Science Founda-
tion of China (30371365, 30571776, and 30972601).
We gratefully acknowledge the contributions of all of the partners
and participant laboratories.
All authors declare that we have no financial conflicts of interest.
1. Aarthi, D., K. Ananda Rao, R. Robinson, and V. A. Srinivasan. 2004. Vali-
dation of binary ethyleneimine (BEI) used as an inactivant for foot and
mouth disease tissue culture vaccine. Biologicals 32:153–156.
2. AbuBakar, S., et al. 1999. Identification of enterovirus 71 isolates from an
outbreak of hand, foot and mouth disease (HFMD) with fatal cases of
encephalomyelitis in Malaysia. Virus Res. 61:1–9.
3. Beld, M., et al. 2004. Highly sensitive assay for detection of enterovirus in
clinical specimens by reverse transcription-PCR with an armored RNA in-
ternal control. J. Clin. Microbiol. 42:3059–3064.
4. Bressler, A. M., and F. S. Nolte. 2004. Preclinical evaluation of two real-time,
reverse transcription-PCR assays for detection of the severe acute respira-
tory syndrome coronavirus. J. Clin. Microbiol. 42:987–991.
5. Chan, K. P., et al. 2003. Epidemic hand, foot and mouth disease caused by
human enterovirus 71, Singapore. Emerg. Infect. Dis. 9:78–85.
6. Chang, L. Y., et al. 2004. Transmission and clinical features of enterovirus 71
infections in household contacts in Taiwan. JAMA 291:222–227.
7. Chang, L. Y., et al. 1999. Comparison of enterovirus 71 and coxsackie-virus
A16 clinical illnesses during the Taiwan enterovirus epidemic, 1998. Pediatr.
Infect. Dis. J. 18:1092–1096.
8. Chen, T. C., et al. 2006. Combining multiplex reverse transcription-PCR and
a diagnostic microarray to detect and differentiate enterovirus 71 and cox-
sackievirus A16. J. Clin. Microbiol. 44:2212–2219.
9. de Pagter, P. J., R. Schuurman, N. M. de Vos, W. Mackay, and A. M. van
Loon. 2010. Multicenter external quality assessment of molecular methods
for detection of human herpesvirus 6. J. Clin. Microbiol. 48:2536–2540.
10. Domingo, C., et al. 2010. 2nd international external quality control assess-
ment for the molecular diagnosis of dengue infections. PLoS Negl. Trop. Dis.
11. Foo, D. G., et al. 2008. Identification of immunodominant VP1 linear epitope
of enterovirus 71 (EV71) using synthetic peptides for detecting human anti-
EV71 IgG antibodies in Western blots. Clin. Microbiol. Infect. 14:286–288.
12. Hietala, S. K., and B. M. Crossley. 2006. Armored RNA as virus surrogate
in a real-time reverse transcriptase PCR assay proficiency panel. J. Clin.
13. Huang, Q. Y., Y. J. Cheng, Q. W. Guo, and Q. G. Li. 2006. Preparation of a
chimeric armored RNA as a versatile calibrator for multiple virus assays.
Clin. Chem. 52:1446–1448.
14. Jiang, T., et al. 2011. Development and evaluation of a reverse transcription-
loop-mediated isothermal amplification assay for rapid detection of entero-
virus 71. J. Clin. Microbiol. 49:870–874.
15. Kessler, H. H., et al. 1997. Rapid diagnosis of enterovirus infection by a new
one-step reverse transcription-PCR assay. J. Clin. Microbiol. 35:976–977.
16. Li, L., et al. 2005. Genetic characteristics of human enterovirus 71 and
coxsackievirus A16 circulating from 1999 to 2004 in Shenzhen, People’s
Republic of China. J. Clin. Microbiol. 43:3835–3839.
17. Lipson, S. M., R. Walderman, P. Costello, and K. Szabo. 1988. Sensitivity of
rhabdomyosarcoma and guinea pig embryo cell cultures to field isolates of
difficult-to-cultivate group A coxsackieviruses. J. Clin. Microbiol. 26:1298–
18. Pasloske, B. L., C. R. Walkerpeach, R. D. Obermoeller, M. Winkler, and
D. B. DuBois. 1998. Armored RNA technology for production of ribonu-
clease-resistant viral RNA controls and standards. J. Clin. Microbiol. 36:
19. Pisani, G., et al. 2008. External quality assessment for the detection of HCV
RNA, HIV RNA and HBV DNA in plasma by nucleic acid amplification
technology: a novel approach. Vox Sang. 95:8–12.
20. Qiu, J. 2008. Enterovirus 71 infection: a new threat to global public health?
Lancet Neurol. 7:868–869.
21. Ryu, W. S., et al. 2010. Clinical and etiological characteristics of enterovirus
71-related diseases during a recent 2-year period in Korea. J. Clin. Microbiol.
22. Saldanha, J., N. Lelie, A. Heath, and the WHO Collaborative Study Group.
1999. Establishment of the first international standard for nucleic acid am-
plification technology (NAT) assays for HCV RNA. Vox Sang. 76:149–158.
22a.Simes, R. 1986. An improved Bonferroni procedure for multiple tests of
significance. Biometrika 73:751–754.
23. Thao, N. T., et al. 2010. Development of a multiplex polymerase chain
reaction assay for simultaneous identification of human enterovirus 71 and
coxsackievirus A16. J. Virol. Methods 170:134–139.
24. Wang, J. R., et al. 2002. Change of major genotype of enterovirus 71 in
outbreaks of hand-foot-and-mouth disease in Taiwan between 1998 and
2000. J. Clin. Microbiol. 40:10–15.
25. Wang, S. Y., T. L. Lin, H. Y. Chen, and T. S. Lin. 2004. Early and rapid
detection of enterovirus 71 infection by an IgM-capture ELISA. J. Virol.
26. Wei, Y. X., et al. 2008. RNase-resistant virus-like particles containing long
chimeric RNA sequences produced by two-plasmid coexpression system.
J. Clin. Microbiol. 46:1734–1740.
27. Xu, F., et al. 2010. Performance of detecting IgM antibodies against entero-
virus 71 for early diagnosis. PLoS One 5:e11388.
28. Yang, F., et al. 2009. Enterovirus 71 outbreak in the People’s Republic of
China in 2008. J. Clin. Microbiol. 47:2351–2352.
29. Zhan, S. E., et al. 2009. Armored long RNA controls or standards for
branched DNA assay for detection of human immunodeficiency virus type 1.
J. Clin. Microbiol. 47:2571–2576.
30. Zhang, D., J. Lu, and J. Lu. 2010. Enterovirus 71 vaccine: close but still far.
Int. J. Infect. Dis. 14:e739–e743.
VOL. 49, 2011EXTERNAL QUALITY ASSESSMENT OF EV71 AND CA163595