Content uploaded by Lorenzo M Donini
Author content
All content in this area was uploaded by Lorenzo M Donini on Jan 03, 2014
Content may be subject to copyright.
©
2005, Editrice Kurtis
Vol. 10: e28-e32, June 2005
BRIEF
REPORT
e28
Key words:
Orthorexia nervosa
Correspondence to:
Prof. Lorenzo M. Donini,
MD PhD
Istituto di Scienza
dell’Alimentazione
Università degli Studi di
Roma “La Sapienza”
P.le Aldo Moro 5,
00185 Rome, Italy
E-mail:
lorenzomaria.donini@uniroma1.it
Received: March 22, 2004
Accepted: July 16, 2004
Orthorexia nervosa: Validation of a
diagnosis questionnaire
L.M. Donini*, D. Marsili*, M.P. Graziani**, M. Imbriale*, and C. Cannella*
*Istituto di Scienza dell’Alimentazione, Università degli Studi di Roma “La Sapienza, and ** Istituto di Scienze
dell’Alimentazione, CNR, Avellino, Italy
ABSTRACT. Aim: To validate a questionnaire for the diagnosis of orthorexia nervosa, an
eating disorder defined as “maniacal obsession for healthy food”. Materials and Methods:
525 subjects were enrolled. Then they were randomized into two samples (sample of 404
subjects for the construction of the test for the diagnosis of orthorexia ORTO-15; sample of
121 subjects for the validation of the test). The ORTO-15 questionnaire, validated for the
diagnosis of orthorexia, is made-up of 15 multiple-choice items. Results and Conclusion:
The test we proposed for the diagnosis of orthorexia (ORTO 15) showed a good predictive
capability at a threshold value of 40 (efficacy 73.8%, sensitivity 55.6% and specificity 75.8%)
also on verification with a control sample. However, it has a limit in identifying the obsessive
disorder. For this reason we maintain that further investigation is necessary and that new
questions useful for the evaluation of the obsessive-compulsive behavior should be added to
the ORTO-15 questionnaire.
(Eating Weight Disord. 10: e28-e32, 2005).
©
2005, Editrice Kurtis
INTRODUCTION
For some time now, the mass-media and
experts in the field of nutrition have noticed
a new eating behavior disorder not yet rec-
ognized as a disease by DSM IV, called
“orthorexia nervosa (ON)” (1-5).
Generally, orthorexia can be considered
when the eating disorder is long-term and
not transitory, and when such behavior
has a significant negative impact on the
quality of life of the individual (3, 4, 6-9).
In extreme cases, orthorexic subjects pre-
fer to starve themselves rather than to eat
food they consider “impure” and harmful
to their health (3, 6-9).
In view of these considerations, orthorexia
may be considered to be a more or less seri-
ous personality or behavioral disorder that
has very little to do with trends or behaviors
linked to religious or philosophical customs.
In a previous work (10) we verified the
prevalence of the orthorexia phenomenon
diagnosed with a questionnaire on eating
habits and the presence of obsessive-pho-
bic personality traits.
Of the 404 subjects examined, 28 were
found to suffer from ON (prevalence of 6.9%).
The aim of this work was to validate a
questionnaire for the diagnosis of
orthorexia nervosa that could be easily
administered.
MATERIALS AND METHODS
Sample selection
The study was carried out at the Institute
of Food Sciences, University of Rome “La
Sapienza”, directed by Prof. Carlo Cannella
between February and August 2001.
Enrolment of volunteers and the collec-
tion of data were both carried out by
trained personnel in the field of Food
Science and Research on Eating Behavior.
525 subjects were enrolled. Spontaneous
enrolment gave us subjects with various
different occupational characteristics:
employees came from the Institute of
Biochemistry “La Sapienza” University,
from the Ministry of the Italian Air Force,
from the Sat 2000 television channel; stu-
dents enrolled at the Plinio Scientific High
School and at “La Sapienza” University;
parents of children in the 4th class of the
San Giuseppe Junior School and parents of
patients attending the Pediatric Dietetics
Service at the Umberto I Hospital in Rome;
a group of residents from Frosinone, near
Rome, etc, etc.
Subjects under the age of 16 were excluded
because they were considered insufficiently
autonomous in the choice of their food.
The 525 subjects were divided into two
samples upon randomization:
- sample of 404 subjects for the set-up of a
For personal use only
©
2005, Editrice Kurtis
Orthorexia nervosa
questionnaire for the diagnosis of orthorexia
(ORTO-15);
- sample of 121 subjects for the validation of
the ORTO-15 test.
The characteristics of the sample of 404 sub-
jects were described in an earlier prevalence
study (10).
In this study, we defined 4 groups of subjects
on the basis of their eating behavior (assessed
using a questionnaire on eating habits, with
special emphasis on the choices between food
normally considered “healthy” and unhealthy,
done by the subjects) and obsessive-phobic
personality traits, using the Minnesota
Multiphasic Personality Inventory (11). Based
on the concept that ON is a disorder character-
ized by a combination of eating, behavioural
and obsessive-phobic personality traits, we
diagnosed ON in the presence of both:
• “health fanatic” eating habits. In particular
we emphasized the choice made by the sub-
ject of food normally considered “healthy”
(fresh, wholemeal, biological produce….)
and that not normally considered healthy
(frozen, tinned….). To classify each food
group selected (cereals, milk, meat, fish,
vegetables, fruit, fast-food, snacks, sweets
and biscuits, drinks), a points system was
used which awarded “0” for eating behav-
iour considered “healthy” and ”1” for “non
healthy”. The final result was the ratio of
the sum of points awarded for each single
item with the maximum of points that each
subject could obtain without including the
items to which we failed to obtain a
response. From the distribution of points
obtained it was decided to consider those
subjects who were classified below the 25
th
percentile (score <0.57) as “health fanatic”
• obsessive-compulsive traits and phobia
linked to the personality of the subject,
based on scale 7 of the MMPI test consider-
ing a score of >65 for women and >66 for
men as modified.
By this way we found 4 groups of subjects:
1. normal eating behavior and MMPI
2. normal eating behavior and pathological
MMPI
3. “healthy” eating behavior and normal MMPI
4. “orthorexic” in which “healthy” eating
behavior is associated with a pathological
MMPI
The ORTO-15 questionnaire for the
diagnosis of orthorexia
The ORTO-15 questionnaire, a tool for the
diagnosis of orthorexia and made-up of 15 mul-
tiple-choice items, was constructed. The test
was created starting from a previously existing
model used by Bratman on a population in the
U.S.A. (3). The total structure of the test and of
the single questions was obtained at the end of
a series of preliminary questionnaires that
were reviewed, after administration to “pilot”
samples (Table 1).
Answers that indicated orthorexia were
given a score of “1”, while the “healthier” ones
had a score of “4”. The sum of the scores was
the final score of the test.
We defined the threshold value of the ORTO-
15 questionnaire based on the study sample
(404 subjects). This value could give a diagnosis
of orthorexia comparing the score of the four
groups of subjects defined on the basis of their
eating behavior and the MMPI score.
Validation of the ORTO-15 test
Thus, we validated the ORTO-15 test on the
sample of 121 subjects and the related thresh-
old value identifying agreement with the
results with the diagnosis of orthorexia (true
positives, true negatives, false positives, false
negatives). We measured the predictive capa-
bility for diagnosing orthorexia through the
calculation of efficacy (agreement between
the response to the test and the “truth”), sen-
sitivity (incidence of true positives on the
totality of the positives: capability of the test
to single out the orthorexic subjects identify-
ing the positive cases and avoiding the false
negatives), specificity (incidence of true nega-
tives on the totality of the negatives: capabili-
ty of the test to identify healthy subjects con-
sidering only the true positives as positive
and avoiding false positives), positive predic-
tive value (probability of being sick in pres-
ence of a positive test) and negative (the
probability that the subject has not got the
disorder when the test is negative).
Elaboration of data
Student t test and ANOVA were used to
assess differences in group means. Statistical
significance was set at the p<0.05 level. To
identify optimal threshold values for predicting
Orthorexia, receiver-operating-characteristics
(ROC) curve analysis was performed by com-
puting the sensitivity and 1-specificity of the
test at various cut-off levels. The area under the
ROC curve was evaluated. A value of 0.5 under
the ROC curve indicates that the variable per-
forms no better than chance while a value of
1.0 indicates perfect discrimination (12, 13).
Data were collected and analysed using SPSS
software for Windows 10.0 (SPSS Inc 1989-
1999) and Win Episcope 2.0 [Facultad de
Veterinaria de Zaragoza (E) Wageningen
University (N), University of Edinburgh (GB)].
e29
Eating Weight Disord., Vol. 10: N. 2- 2005
Eating Weight Disord. 10: e28-e32, 2005
©2005, Editrice Kurtis
©
2005, Editrice Kurtis
L.M. Donini, D. Marsili, M.P. Graziani
, et al.
RESULTS
Identification of a threshold value and of
the predictive value of the ORTO-15 test
The variance analysis showed a statistically
significant difference in the ORTO-15 score
between the different groups of subjects
(F=11.9, p=0.000) (Table 2).
We noted particularly how in groups with
“healthy” eating behavior the ORTO-15 score
is significantly lower (39.3±4 vs 42.3±4; t=5.9,
p=0.000), while the differences are less
marked and not statistically significant as a
function of the MMPI class (pathological
41.1±5, non pathological 41.7±4; t=1.1, p=0.3).
We tested three different threshold values
for ORTO-15 as a function of the scores
obtained by the different groups of subjects:
<35, <40, <45. Below these cut-offs the test is
supposed to give a diagnosis of orthorexia
(Table 3).
We calculated the predictive value of the test
in differentiating the orthorexic subjects from
healthy ones as a function of those threshold
values (Table 4).
At a threshold value of 35 points the test
has an efficacy of 86.5%, with a high specifici-
ty (94.2%) and a high negative predictive
value (91.1%). When the threshold value
increases, the sensitivity increases too (55.6%
at 40 points and 85.2% at 45 points), while
specificity and efficacy decrease. At the cut-
e30
Eating Weight Disord., Vol. 10: N. 2- 2005
Eating Weight Disord. 10: e28-e32, 2005
©2005, Editrice Kurtis
TABLE 1
Test for the diagnosis of orthorexia nervosa.
ORTO-15
Always Often Sometimes Never
1) When eating, do you pay attention to the calories of the food? OOOO
2) When you go in a food shop do you feel confused? OOOO
3) In the last 3 months, did the thought of food worry you? O O O O
4) Are your eating choices conditioned by your worry about your health status? OOOO
5) Is the taste of food more important than the quality when you evaluate food? OOOO
6) Are you willing to spend more money to have healthier food? O O O O
7) Does the thought about food worry you for more than three hours a day? OOOO
8) Do you allow yourself any eating transgressions ? OOOO
9) Do you think your mood affects your eating behavior? OOOO
10) Do you think that the conviction to eat only healthy food increases self-esteem? OOOO
11) Do you think that eating healthy food changes your life-style (frequency of eating out, friends, …)? OOOO
12) Do you think that cosuming healthy food may improve your appearance? OOOO
13) Do you feel guilty when transgressing ? OOOO
14) Do you think that on the market there is also unhealthy food? OOOO
15) At present, are you alone when having meals? OOOO
SCORING GRID FOR ORTO-15 TEST RESPONSES
ITEMS RESPONSES
Always Often Sometimes Never
2-5-8-9 4321
3-4-6-7-10-11-12-14-15 1234
1-13 2431
TABLE 2
Mean of scores on the ORTO-15 test for the different groups
of subjects of the study sample.
Score of ORTO-15
(Mean±SD)
Orthorexic 39.4±4
Normal eating behavior and pathological MMPI 41.9±4
“Healthy” eating behavior and normal MMPI 39.3±4
Normal eating behavior and normal MMPI 42.4±4
©
2005, Editrice Kurtis
Orthorexia nervosa
off point of 45 the test becomes unreliable
since its efficacy is of 37.4%.
Validation of the ORTO-15 test
The ORTO-15 test and the previously select-
ed threshold values (<35 ed <40), were applied
to the validation sample (Table 3). The results
confirmed the substantial validity of the test
only for the threshold value of 40 points (sen-
sitivity 100.0%, specificity 73.6%, positive pre-
dictive value 17.6%, negative predictive value
100%) (Table 4).
Instead, at a threshold value of 35 the test
had a sensitivity and a positive predictive
value of 0%.
The area under the ROC curve, representing
the overall accuracy of the ORTHO-15 test as a
test for the diagnosis of ON was found to be
0.696 (95% CI: 0.585-0.807).
DISCUSSION
The first to speak out about orthorexia was
Dr. Steven Bratman, author of the book
“Health–Food Junkies”(3).
The desire to eat healthy foods is not a disor-
der in itself, but an obsession for these foods,
along with a loss of moderation and balance
and the withdrawal from life caused by this
food habit, may then lead to orthorexia.
The orthorexic sufferer spends a great deal
of his time thinking about food, frequently
dedicating his whole existence to the plan-
ning, purchase, preparation and consumption
of the food that he considers healthy. His eat-
ing behavior becomes the only one possible,
and generates a feeling of superiority over
the lifestyle and eating habits of others.
Selection of sample subjects
As far as the subjects for the study are con-
cerned, a defect in the selection method must
be pointed out, since the subjects completed
the questionnaire only on the basis of volun-
tary enrolment. This could limit the possibility
of extending the results to apply to the entire
population, and consequently implies the
necessity of further studies.
e31
Eating Weight Disord., Vol. 10: N. 2- 2005
Eating Weight Disord. 10: e28-e32, 2005
©2005, Editrice Kurtis
TABLE 3
Distribution of orthorexic and non-orthorexic subjects,
according to ORTO-15 with different threshold values, in study
and validation samples.
Study Sample Validation Sample
Positive Negative Positive Negative
Threshold Value Orto-15: <35
Orthorexic 5 22 0 3
Normal eating behavior 5 57 2 25
and pathological MMPI
Pathological eating behavior 11 53 4 32
and normal MMPI
Normal eating behavior 14 226 3 50
and normal MMPI
Threshold Value Orto-15: <40
Orthorexic 15 12 3 0
Normal eating behavior 15 47 8 19
and pathological MMPI
Pathological eating behavior 30 34 11 25
and normal MMPI
Normal eating behavior 58 182 14 39
and normal MMPI
Threshold Value Orto-15: <45
Orthorexic 23 4
Normal eating behavior 44 18
and pathological MMPI
Pathological eating behavior 58 6
and normal MMPI
Normal eating behavior 164 76
and normal MMPI
Note: 11 subjects of the study sample did not complete the ORTO-15 test
TABLE 4
Predictive value of ORTO-15 in giving the diagnosis of orthorexia in the validation and study samples.
Study Sample Validation Sample
Threshold Values Efficacy Sensitivity Specificity Positive Negative Efficacy Sensitivity Specificity Positive Negative
Orto-15: predictive predictive predictive predictive
value value value value
%% % % % %% %%%
<35 86.5 18.5 94.2 26.3 91.1 89.3 0.0 94.3 0 94.3
<40 73.8 55.6 75.8 20.5 93.8 75.0 100.0 73.6 17.6 100.0
<45 37.4 85.2 31.7 12.3 95.0
©
2005, Editrice Kurtis
L.M. Donini, D. Marsili, M.P. Graziani
, et al.
The test for the diagnosis of orthorexia:
ORTO-15
To construct the test for the diagnosis of
orthorexia we started with the study of Bratman
on the US population (3). The test done by
Bratman is made up of 10 items with a dichoto-
mous choice (YES/NO). The number of YES
answers increases with the degree of orthorexia.
Instead, our test was made-up of 15 closed
multiple choice items (always, often, some-
times, never). The items investigate the obses-
sive attitude of the subjects in choosing, buy-
ing, preparing and consuming food they con-
sider to be healthy.
We kept some items from Bratman’s test (1,
3, 7, 8, 9, 10) even though some verbal aspects
of them were modified. We disguised some
excessive assertiveness since, in our opinion,
they could induce obvious answers.
For example, item 10 of Bratman that states:
“when eating in a correct way do you feel a
sense of total control?”, could imply an affirma-
tive answer, but also a negative one for opposi-
tion. Our reformulation of the question (“do
you allow yourself any eating transgressions?”)
asks the subject a definition of his/her behavior
in a less rigid form and gives him/her a scale of
values that goes from “always” to “never”.
In our opinion, this makes the test responses
more truthful. Also the rigidity of the response
(YES/NO) of Bratman’s test did not appear to
be useful for us for our Latin sample, that is
socially more dialectic and, therefore, more
prone to modulate the behavior in a scale of
value that goes from “always” to “never”, than
an Anglo-Saxon one.
We wanted to use the test to investigate both
the emotional and the rational aspects of the
subjects to whom it was administered: some
items keep to the cognitive-rational area (1, 5,
6, 11, 12, 14), other ones to the clinical area (3,
7-9, 15), and others to the emotional area (2, 4,
10 and 13).
We then gave a score of “1” to the response
that was more indicative of orthorexia and that
of “4” to those that indicated a normal eating
behavior. We added everything up to obtain the
final test score and, as predictable, subjects with
healthy eating behavior had a lower score.
Then we identified the threshold value below
which the diagnosis of orthorexia could be
given. A cut-off of 40 was considered to be
more predictive either in the study sample (sen-
sitivity 55.6%, specificity 75.8%, positive pre-
dictive value 20.5%, negative predictive value
93.8%) or in the validation one (sensitivity
100.0%, specificity 73.6%, positive predictive
value 17.6%, negative predictive value 100.0%).
Cut-off point values can be set depending on
the purpose for which the scales are used. For
diagnosis purposes, as it is in this case, a high
specificity is generally required whereas
screening purposes require a high sensitivity.
We found that the test has a threshold value
of 40 points and a notable predictive capability
concerning healthy eating behavior, while it is
less efficient in discriminating the other com-
ponent in the diagnosis of orthorexia, that is
the presence of obsessive traits. In fact, the
mean score of the test does not vary signifi-
cantly as a function of the MMPI class (patho-
logical/non pathological).
Therefore, we maintain that further investi-
gation is necessary and that new questions use-
ful for the evaluation of the obsessive-compul-
sive behavior should be added to the ORTO-15
questionnaire.
Please note that the original test validated is
in Italian and the present version was translat-
ed into English for editorial purposes. This ver-
sion needs further validation in an Anglo-
Saxon population.
REFERENCES
1. “Progetto Obiettivo Anoressia e Bulimia”, Ministero
della Sanità, Ufficio stampa, Comunicato n. 222, Roma,
20 settembre 2000
2. Andreoli V., Cassano G., Rossi R. (Eds): DSMIV,
Manuale diagnostico e statistico dei disturbi mentali.
Milano, Masson, 1996.
3. Bratman S., Knight D.: Health food junkies. New York,
Broadway Books, 2000.
4. www.something-fishy.org: “Other types of eating dis-
orders”
5. www.anred.com: “Anorexia nervosa and related eat-
ing disorders”
6. www.albanesi.it: “L’ortoressia”
7. www.beyondveg.com: “Obsession with dietary purity
as an eating disorder”, Tom Billings
8. www.rai.it: “Cibo sano: che ossessione”
9. Nymah H.: A direct question: is orthorexia a correct
word for a wrong concept? Lakartdningen, 99, 433-
434, 2002.
10. Donini L.M., Marsili D., Graziani M.P., Imbriale M.,
Cannella C.: Orthorexia nervosa: a preliminary study
with a proposal for diagnosis and an attempt to mea-
sure the dimension of the phenomenon. Eat. Weight
Disord., 9, 151-157, 2004.
11. Mosticoni R., Chiari G.: Una descrizione obiettiva della
personalità-Il Minnesota Multiphasic Personalità
Inventory-MMPI. Firenze, Organizzazioni Speciali,
1985.
12. Murphy J.M., Berwick D.M., Weinstein M.C., Borus
J.F., Budman S.H., Klerman G.L.: Performance of
screening and diagnostic tests. Arch. Gen. Psychiatry,
44, 550, 1987.
13. Hanley J.A., McNeil B.J.: A method of comparing the
areas under receiver operating characteristics curves
derived from the same cases. Radiology, 148, 839, 1983.
e32
Eating Weight Disord., Vol. 10: N. 2- 2005
Eating Weight Disord. 10: e28-e32, 2005
©2005, Editrice Kurtis
