ArticlePDF Available

A contribution to the validation of Italian eHEALS scale for the Italian population

Authors:

Abstract and Figures

Background: Modern technology allows people to search for various information on the Internet, including health information. The eHEALS scale measures and assesses the ability for consumers to find, judge and apply health information found towards health problems. The Italian version of the eHEALS scale was validated using the Principal-Component Analysis (PCA) technique. Even if the results were satisfactory and the scale was considered validated,
Content may be subject to copyright.
Open Access Original Research
A contribution to the validation of Italian eHEALS scale
for the Italian population
Paolo Soraci1, Eleonora Guaitoli2, Fulvia Lagattolla3, Francesco M. Melchiori4,
Roberta Cimaglia5, Carla Nesci6, Grazia Parente7, Mariachiara Trovato, Sabina
Spagna4, Lara Scali5, Valentina Roggero4, Micol Lucaselli9, Giulia Bravo10
1Istituto Mediterraneo di Psicologia APS (I.Me.P.), Reggio Calabria, Italy.
2Department of General Surgery P.O. Valle d’Itria, Martina Franca, Italy.
3Servizio di Psiconcologia, IRCCS Istituto Tumori "Giovanni Paolo II" di Bari, Italy.
4University Niccolò Cusano, Faculty of Psychology, Rome, Italy.
5Istituto Romano di Psicoterapia Psicodinamica Integrata, Rome, Italy.
6Scuola di Psicoterapia Analisi Transazionale Reggio Calabria, Italy.
7Associazione Matrice Orientamento e Formazione Onlus, Bari, Italy.
9 Associazione BluMedia, Rome, Italy
10Department of Medicine, University of Udine, Udine, Italy.
Corresponding author: Paolo Soraci, paolo.soraci85@gmail.com
Abstract: Background: Modern technology allows people to search for various
information on the Internet, including health information. The eHEALS scale
measures and assesses the ability for consumers to find, judge and apply health
information found towards health problems. The Italian version of the eHEALS
scale was validated using the Principal-Component Analysis (PCA) technique.
Even if the results were satisfactory and the scale was considered validated,
psychometric scaling literature is also recommended to subject the scale itself to a
Confirmatory Factor Analysis (CFA) for a more sound and complete validation
process. Methods: The sample consisted of 349 Italian participants. Each
participant was administered Italian versions of the eHEALS scale, the
Rosenberg’s self-esteem scale, The Anxiety - Adult (PROMIS Emotional Distress
Anxiety) - Short Form, the Locus of Control of Behavior Test and the General
Self-Efficacy Scale. Several psychometric tests were also performed to investigate
the validity and reliability of the test, including the CFA. Results: Analysis of the
data showed satisfactory psychometric characteristics and confirmed the scale’s
unidimensional properties. The eHEALS eight items scale items had acceptable
correlations with the eHEALS test total (min=0.780, max=0.867). Furthermore,
factor loadings were significant (min=0.836, max=0.948). The measure of internal
consistency was excellent = 0.931). Construct validity for the eHEALS scale
was supported by significant positive correlations with the Internal Locus of
Control of Behavior and the General Self-Efficacy Scale, the frequency of
searching for information on one's health, perceived expertise with technology,
frequency of Internet use, perceived Internet expertise and a negative correlation
Citation: Soraci, P.,
Guaitoli, E., Lagattolla,
F., Melchiori, F.M.,
Cimaglia, R., Nesci, C.,
Parente, G., Trovato, M.,
Spagna, S., Scali, L.,
Roggero, V., Lucaselli,
M, Bravo, G. (2022). A
contribution to validation
of eHEALS scale for the
Italian population.
Journal of Concurrent
Disorders.
Founding Editor:
Masood Zangeneh, PhD
Editor: Fayez Mahamid,
PhD
Received: 08/21/2022
Accepted: 09/10/2022
Published: 10/17/2022
Copyright: ©2022
Soraci, P., Gualtieri, E.,
Lagattolla, F., Melchiori,
F.M., Cimaglia, R.,
Nesci, C., Parente, G.,
Trovato, M., Spagna, S.,
Scali, L., Roggero, V.,
Bravo, G. Licensee CDS
Press, Toronto, Canada.
This article is an open
access article distributed
under the terms and
conditions of the Creative
Commons Attribution
(CC BY) license
(http://creativecommons.
org/licenses/by/4.0/)
Journal of Concurrent Disorders, 2022 https://cdspress.ca/
Journal of Concurrent Disorders, 2022
2
with external locus of control. Conclusions: The Italian version of the Health
Literacy Scale (eHEALS) is valid and reliable in assessing the ability to collect,
evaluate, and apply health information to health problems amongst the general
Italian population.
Keywords: eHeals, Health Literacy, Confirmatory Factor Analysis, Italian
validation.
Introduction
Since the onset of the SARS-CoV-2 pandemic, digital technology
across the global population has become the primary method of both
communication and to access health information; specifically, obtaining
answers on health inquiries, recognizing symptoms, methods of precaution
to be taken, and the most effective treatments to follow for health problems.
As a result, the need for digital literacy has grown exponentially.
Health literacy represents the individual’s ability to obtain, process
and understand basic health information, the functioning of health services,
and information relating to personal health (e.g., individual pathology,
eating habits, physical behavior, etc.), that are critical for individuals to
make appropriate health decisions (Ratzan et al., 2000). Health literacy
represents not only a range of skills going over the individual ability to read
and take in information, but also the ability to control individual and societal
factors that may have an impact on their health (Ratzan, et al. 2000).
Modern technology allows people to search for various information
on the Internet, including health information, through a wide range of
electronic devices such as smartphones, tablets and PCs connected to Wi-
Fi or mobile networks. For example, the Pew Research Internet Project
estimates that over 85% of American adults use the Internet, and nearly
three-quarters of them have searched for health information online. (Pew
Research Center, 2016). Despite this, it is useful to specify that the ability
to search for health information differs from the ability to interpret health
information (Norman & Skinner, 2006). Specifically, people may not have
the skills to understand such information or may not have sufficient
knowledge for the correct interpretation of information found on the
Internet (Norman & Skinner, 2006; Institute of Medicine, 2004; 2011).
For this reason, assessing the degree of health literacy, for the
general population, is considered a fundamental prerequisite for health
professionals who intend to promote e-health resources to patients who may
need them (Norman & Skinner, 2006).
Digital health (eHealth) literacy is defined as "the ability to browse
and obtain health information" (Nguyen et al., 2016). Digital health literacy
(eHealth) can be a challenge for the general population and patients, given
Journal of Concurrent Disorders, 2022 https://cdspress.ca/
Journal of Concurrent Disorders, 2022
3
the need to understand its many components, including "i) traditional
literacy, ii) health literacy, iii) information literacy, iv) scientific literacy, v)
media literacy and vi) information literacy" (Norman & Skinner, 2006).
More specifically, users should have the knowledge to access, retrieve, and
evaluate the information they obtain online (Norman & Skinner, 2006).
Users are likely to be exposed to different types and qualities of information,
highlighting the need to compare and further evaluate the information. Also,
due to the rapid change in both care routines and technology, health
information is updated quickly. Furthermore, searching online about one's
own health state, sometimes fairly accurate or inexact (for example, one
consumes information from sources whose reliability and trustworthiness is
unknown) is correlated with subsequent worries, anguish, and anxiety
(Starcevic & Berle, 2013).
In 2006, Norman and Skinner (2006) developed the eHealth Literacy
Scale (eHEALS) as a tool to measure digital health literacy. The eHEALS
is a self-assessment scale which provides data on the perception of an
individual’s own knowledge and skills when collecting and understanding
health-related information online. Thanks to its simple and fast
administration, it has been validated in different populations and translated
into various languages such as Japanese (Mitsutake et al., 2012; 2016),
Chinese (Koo et al., 2012), Dutch (Van der Vaart et al., 2011), Spanish
(Aponte & Nokes, 2015), Swedish (Wångdahl et al., 2020) and finally,
Italian (Bravo et al., 2018). The eHEALS test is designed to provide a
general estimate of mainstream eHealth literacy that can be used to tailor
clinical decision making and in health promotion planning.
It can be assumed that there is a link between digital health literacy
and the general use of technology (Norman & Skinner, 2006). Individuals
accustomed to technology will most likely increase their ability to use it as
a tool to find information online on health problems. Consequently, it
becomes unsafe when such information is false, misleading or of low quality
(Eysenbach, 2001; 2002). The availability of tools and resources to evaluate
the qualitative level of people’s comprehension and ability to use health
information on the web allows healthcare professionals (e.g., doctors,
psychologists) to protect online users from possible risks, and at the same
time increase their responsibility (Eysenbach, 2001; 2002).
Health portals developed by governmental and non-profit
organizations are very useful, as they offer reliable information and
discourage populations from consuming information from unsafe sources,
with some limits (e.g., the impossibility of analyzing all information
accessible on the web). (Norman & Skinner, 2006). Individuals without
digital and health literacy skills should be also considered. Consequently,
the figure of an "expert", able to facilitate the process of screening the most
relevant health information for the users, could be necessary (Klecun,
2010).
Journal of Concurrent Disorders, 2022 https://cdspress.ca/
Journal of Concurrent Disorders, 2022
4
Regardless of the population of interest, the need to consume
information on the Internet with confidence is particularly important when
considering health-related problems; where the consequences of trusting
low-quality, misleading or even false information are not negligible and can
undermine the trust between patients and healthcare professionals with a
negative impact on the effectiveness of treatments (Fan et al., 2020).
Complementarily, Hirji (2004) points out that many users are not
adequately trained in information retrieval skills, and this is overlooked by
many website designers and health care providers who publish information
online. The author cites studies which seem to indicate that people
overestimate their ability to judge accurate material online. Essential skills
for evaluating web-based information are identified by Edgar et al. (2002),
such as the ability to conduct a search to find the "right" sites; the ability to
judge the quality of information; and the ability to synthesize such
information in a context useful for personal/individual health. Although this
is a useful approach, it presents some limitations, including the inability to
track all health information online. Therefore, the promotion of people's
ability to critically analyze the data found remains a priority (Hirji, 2004;
Wyatt et al., 2003; Klecun 2010).
Many researchers (Robbins & Dunn, 2019; Wakefield et al., 2017;
Vaart & Drossaert, 2017) point out that digital health literacy requires skills
complementary to those of general and health literacy skills. The eHEALS
scale, can allow the clinician/researcher to understand the person's skills and
awareness on health issues. The eHealth scale could therefore be an
adequate tool to assess the degree of capability for a patient to seek health
information online (Cuthbert & Aggarwal, 2020).
The Italian version of the eHEALS scale (Bravo et al., 2018) was
validated using the PCA (Principal-Component Analysis) technique.
However, existing literature recommends a more solid and complete
validation, for instance, subjecting the scale itself also to a Confirmatory
Factor Analysis (CFA), as the PCA can also be used as an initial step in
CFA because it can provide information regarding the maximum number
and nature of factors (Kim, 2008). However, because measuring health
constructs is complex, scale development and construct validation studies
usually suggest CFA only after having used exploratory techniques to
investigate the latent structure (Edwards, 2010).
Furthermore, convergent construct validity was examined using
mainly demographic variables (e.g., age, sex, educational qualification),
frequency of Internet access to search for health information. To strengthen
validity ( i.e. construct, nomology, convergent and discriminant), in addition
to the original research (Bravo et al., 2018), further tests, theoretically
connected to the eHEALS (eHealth Literacy Scale), were evaluated in the
sample: (i) Self-esteem (Rosemberg's self-esteem scale); (ii) Anxiety (The
Anxiety Adult PROMIS Emotional Distress - Anxiety - Short Form) (iii)
Locus of Control of Behavior (Test Locus of Control of Behavior -LCB)
Journal of Concurrent Disorders, 2022 https://cdspress.ca/
Journal of Concurrent Disorders, 2022
5
(iv) and General Self-Efficacy (General Self-Efficacy Scale). These scales
were used because accurate research of information about one's health,
discovered using electronic tools, seemed to be associated with lower levels
of anxiety (e.g., Bayrampour et al., 2019), higher self-esteem (Wolf et al.,
2016), higher internal locus of control (Vajaean & Baban, 2015) and a
higher sense of self-efficacy (Ditzler et al., 2016).
The objective of the study is: (i) to calculate the main indicators of
good fit (e.g., Goodness Fit Index) performing the Confirmatory Factor
Analysis (CFA), (ii) to evaluate the correlation of the eHEALS test with
self-esteem, anxiety, self-efficacy, locus of control, general efficacy and
(iii) to confirm the one-dimensional factorial structure of the eHEALS.
The research was conducted in accordance with the Declaration of
Helsinki for medical research involving human subjects and was approved
by the Roman Institute of Integrated Psychodynamic Psychotherapy
(IRPPI) in Rome, Italy. All participants gave their consent to participate in
the study. The identity of the participants remains anonymous, and the data
was stored in an encrypted online archive, accessible only to the authors of
this study.
Methods
Participants
Between January and February 2021, a link to the online survey was
published across several Italian social network community forums (i.e.,
Facebook). Inclusion criteria were the following: (i) at least 18 years old;
(ii) understanding of the Italian language; (iii) acceptance of informed
consent. Participant anonymity was guaranteed ((the data was stored in an
encrypted online database). Three hundred and forty-nine volunteers
completed the online survey. Participants who joined voluntarily provided
consent online.
Females comprised the majority of the sample (n= 274; 78.5%), with
a median level of education falling in the “High school” category (see Table
1), and mean age of 40 years (SD ± 13). Furthermore, the mode (the
category in which the most participants identified with) of the romantic
relationship status resulted in the category "Married" (42.7%, n = 149, see
Table 2). The mode for the occupation category was found to be “Worker”
(54%, n = 189, the other category was 11.5% Unemployed, 16.5% Student,
3.5% Retired, 14.5% Other). Furthermore, the most used tool to search for
information was the “Smartphone” category (71%, n = 248, the other
category was 3.7% Other, 22.6% PC-Notebook, 2.7% Tablet). The results
of the main test used (mean ± SD) are summarized in Table 3. Lastly, all
participants completed the entire online form, therefore there was no
missing data.
Journal of Concurrent Disorders, 2022 https://cdspress.ca/
Journal of Concurrent Disorders, 2022
6
Table 1. Education level of Sample (n=349)
Education level
Frequency
Percent
Cumulative Percent
Post graduate title
59
16.905
16.905
16.905
Middle school
12
3.438
3.438
20.344
High school
124
35.530
35.530
55.874
University degree
154
44.126
44.126
100.000
Total
349
100.000
Table 2. Relationship status (n=349)
Relationship status
Frequency
Percent
Valid Percent
Cumulative Percent
Divorced
11
3.152
3.152
3.152
Fiancé
93
26.648
26.648
29.799
Separated
15
4.298
4.298
34.097
Single
75
21.490
21.490
55.587
Married
149
42.693
42.693
98.281
Widowed
6
1.719
1.719
100.000
Total
349
100.000
Instruments
Socio-demographic questions and the use of the Internet for health
research. Socio-demographic information of participants (e.g., gender, age,
educational level, relationship status, employment status) were collected. In
addition, participants were asked questions pertaining to their use of the
Internet and subsequent search behaviours for health information using the
following questions: How often have you searched for information about
your health on the Internet in the last 12 months?”? (using a 5-point Likert
scale, where 1 is never and 5 is very often; “What devices do you generally
use to search for information about your health?with multiple answers:
PC, Tablet, Smartphone, and others. A further question was “How often do
you use the Internet in a week?”, with a 5-point Likert scale, where 1 =
Never and 5 = Very often. In addition, the following questions were asked
How competent do you feel in using the Technology” and “How competent
do you feel in using the Internet?” with response on a Likert scale (1 = Not
competent at all and 5 = Very competent). A further question was “During
the Covid-19 quarantine, did your research on health information on the
Journal of Concurrent Disorders, 2022 https://cdspress.ca/
Journal of Concurrent Disorders, 2022
7
web increase?” with a Likert response from 1 to 5 points, where 1 = Not at
all and 5 = Very Often.
eHEALS scale. The eHEALS (eHealth Literacy Scale) includes 8
items evaluated on a 5-point Likert scale (score 1 as strongly disagree; score
5 as strongly agree), where a higher score indicates a higher level of
confidence in the ability to find, rate and use the health information for
making health-related decisions. In short, a higher score represents
increased perceived eHealth literacy (Paige et al., 2017). Example of an item
is I know how to find health information on the Internet”. The Italian
version of the eHEALS was applied (Bravo et al., 2018). The alpha of
Cronbach in this study was 0.931.
Rosenberg’s Self-Esteem Scale (RSES; Rosenberg, 1965): The 10-
item Italian version (Prezza et al. 1997) was used to assess self-esteem (for
example, "Overall, I am satisfied with myself ") using a four point Likert-
type scale from 0 (strongly in disagree) to 3 (strongly agree). Scores vary
between 0 and 30, with the highest scores indicating greater self-esteem.
The alpha of Cronbach in this study was 0.864. This test reveals that a
person with higher self-esteem is associated with better management of
health information (Wolf et al., 2016).
Anxiety Adult (PROMIS Emotional Distress Anxiety Short
Form). The 7-Item Adult PROMIS Emotional Distress / Anxiety-Short
Form (APEDA-SF) test (Pilkonis et al., 2011; Italian version: Fossati et al.
2015) evaluates anxiety among adults. The seven items (e.g., "I feel
anxious") are rated on a scale of 1 (never) to 5 (very often) with scores
ranging from 7 to 35. A higher score indicates higher levels of anxiety. The
Cronbach alpha in this study was 0.918. The use of correct health
information seems to be associated with low anxiety (Deady et al., 2017;
Bayrampour et al., 2019).
Locus of Control of Behaviour (LCB, Craig et al., 1984). The LCB
is a test consisting of 17 items (e.g., “When I make plans, I am almost certain
that I can make them work”) rating on Likert scale from 0 to 5 (0 =
completely disagree, 5 = completely agree). Seven questions (1,5,7,8,13,15
and 16) evaluate the internal locus control, the others evaluate external locus
control. The indicative value of the 17 answers consists of the sum of the
scores on the external control in addition to the inverted scores of the
questions relating to internal control (Farma & Cortinovis, 2001). The alpha
of Cronbach in this study was 0.700. The use of correct health information
seems to be associated with a major locus of internal control (Vajaean &
Baban, 2015).
General Self-Efficacy Scale (Italian version Sibilia et al., 1995;
original version Schwarzer & Jerusalem, 1995). This scale was created to
assess the general level of perceived self-efficacy in predicting and planning
how to cope and adapt to everyday problems after experiencing stressful life
events. The scale is usually self-administered, as part of a more
Journal of Concurrent Disorders, 2022 https://cdspress.ca/
Journal of Concurrent Disorders, 2022
8
comprehensive questionnaire. Answers are scored on a 5-point Likert scale
(1= Not at all true 5= Exactly true). To calculate the overall score, the
responses to all 10 items are added together to obtain the final composite
score with a range of 10 to 50. A higher score indicates greater overall self-
efficacy. Cronbach's alpha in the present study was 0.928. The use of correct
health information seems to be associated with greater self-efficacy (Ditzler
et al., 2016). An example is "I can always manage to solve difficult problems
if I try hard enough".
Preliminary statistical analysis
The univariate normality of the data was investigated first using the
guidelines proposed by Muthén and Kaplan (1985) which outline an
asymmetry and a kurtosis in the interval from −1 to +1 as the ideal range of
items or Shapiro-Wilk normality test are not significant for p <0.01 (Mishra
et al., 2019). Furthermore, descriptive statistics concerning the items (i.e.,
frequencies, percentages) were calculated. The performed statistical
analysis were the following: (i) descriptive statistics of the eHEALS test
items (i.e., means and standard deviations); (ii) criterion/convergent/
concurrent validity of the eHEALS test; (iii) the reliability of the scale,
examined by composite reliability (CR) (CR values greater than 0.7 are
associated with a strong test reliability; Fornell & Larcker 1981).
The evaluation of the factorial structure and the dimensionality of
the Italian eHEALS was evaluated using Confirmatory Factor Analysis
(CFA). The best sample size to carry out a factor analysis varies between
30 and 500 units (Roscoe, 1975). Moreover, the minimum sample size for
this study must be 340, considering the following factors: Confidence Level
(95%), Margin of Error (5%), population size (> 20.000) (e.g., Kadam,
2010). In addition, from 5 to 10 observations for each variable are needed
(Hair et al., 2010). Specific indicators were also calculated to ascertain the
one-dimensionality of the test (Ferrando & Lorenzo-Seva, 2017): UNICO
(one-dimensional congruence> 0.95), ECV (common variance> 0.80.)
Furthermore, the indices recommended by Kline (2015), for the CFA, were
adopted to delineate a good factorial model in the following way (as
follows): NNFI (Non-Normed Fit Index 0.95), CFI (Comparative Fit
Index ≥ 0.95), GFI (Goodness Fit Index ≥ 0.95), AGFI (Adjusted Goodness
Fit Index 0.95), RMSEA (Root Mean Square Error of Approximation ≤
0.08), and RMSR (Root Mean Square of Residuals 0.8) and with an
acceptable saturation on all items (λij ≥ 0.50, Ferguson & Cox, 1993). The
reliability of the data was assessed through the following indicators:
Cronbach's Alpha (α) (Cronbach, 1951; 1955), McDonald's Omega )
(Mcdonald, 1999) and Composite Reliability (CR). The analyzes were
performed using FACTOR v.10.10.3 (Lorenzo-Seva & Ferrando, 2006),
SPSS Statistics v.20 (IBM Corporation, 2011), Jasp version 0.13.1 (JASP
Team, 2020), the Mann Whitney U calculator (2017).
Journal of Concurrent Disorders, 2022 https://cdspress.ca/
Journal of Concurrent Disorders, 2022
9
Table 3. Descriptive Statistics of main tests used
eHEALS
Anxiety
GSE
Self esteem
LOCINT
LOCEST
Valid
349
349
349
349
349
349
Missing
0
0
0
0
0
0
Mean
27.499
20.461
32.897
22.066
24.937
18.837
Std. Error of Mean
0.401
0.380
0.339
0.307
0.248
0.474
Std. Deviation
7.494
7.106
6.326
5.727
4.635
8.852
Minimum
8.000
7.000
11.000
3.000
9.000
0.000
Maximum
40.000
35.000
45.000
30.000
35.000
49.000
Note: eHEALS= eHEALS scale, Anxiety= PROMIS Emotional Distress Anxiety Short Form, GSE= General self-
efficacy scale, Self-esteem= Rosenberg’s Self-Esteem Scale, LOCINT= Internal locus of control of behaviour scale,
LOCEST= External locus of control of behaviour scale
Results
Confirmatory Factorial Analysis (CFA)
The present study analyzed the distribution of the eight items of the
Italian eHEALS scale. Most items (see Table 4) were distributed
asymmetrically (i.e., negative asymmetry, with the highest frequencies in
the high values). As for asymmetry and kurtosis, some items were
distributed in a substantially non-normal way (the items do not fall within
the range of ± 1 or Shapiro-Wilk normality test are significant for p <0.01,
this confirms that the items are not all normally distributed, see Muthén &
Kaplan 1985; Mishra et al., 2019). Moreover, the Italian eHEALS scale
appeared to have a unidimensional structure (i.e., a single factor); it had
eigenvalues > 1 in a single factor model (i.e., Gorsuch, 1983) which
suggests one factor as the optimal usable model (more specifically, the
eigenvalues = 5.91 with ECV=0.73) and UNICO= 0.988, confirming the
findings in the original research (Bravo et al., 2018).
Since there is no single consensus in the literature (Bollen & Long
1993; Boomsma, 2000), different goodness of fit (GOF) indices were used
to confirm the dimensionality of the eHEALS. In this specific case, since
the elements (see Table 4) were distributed in a substantially non-normal
way (some items outside the range of ± 1), we used the Diagonal Weighted
Least Squares (DWLS, polychromic correlation) method in the
confirmatory factor analysis (estimation method), whit 95% Confidence
Interval and 1000 Bootstrap samples (Koğar & Yilmaz, 2015). The results
showed the following: χ²=30.602 (df=20, p=0.061, i.e. not significant) whit
Journal of Concurrent Disorders, 2022 https://cdspress.ca/
Journal of Concurrent Disorders, 2022
10
χ²/df= 1.53 (Chi Square/degree of freedom ratio <3 for a good model, Hu &
Bentler, 1999; Schumacker, & Lomax, 2010), Comparative Fit Index
(CFI)= 0.99, Tucker-Lewis Index (TLI)=0.99, Bentler-Bonett Non-Normed
Fit Index (NNFI)= 0.99, Root mean square error of approximation
(RMSEA)=0.039, Goodness of fit index (GFI)=0.99.
Furthermore, all items have a significant (p<0.01) and high (>0.50)
factor loading, ranging from 0.836 to 0.948 (see Figure 1, Table 5, 6 for
details). These results confirm an excellent factorial structure and excellent
validity of the investigated construct (Cronbach & Meehl, 1955). Although
the single eHEALS items are distributed in a non-normal way, the total
score of tests used (including the eHEALS test) and the sample size, allows
to use the data approximated to normality (the assumptions of normality are
respected (for the total score of the test) as assessed by standardized residue
analysis, see Doob, 1938; Tchebycheff, 1980) in subsequent analysis (e.g.,
ANOVA, Pearson's r correlation).
Table 4. Descriptive Statistics of the eight items (eHEALS)
ITEM 1
ITEM 2
ITEM 3
ITEM 4
ITEM 5
ITEM 6
ITEM 7
ITEM 8
Valid
349
349
349
349
349
349
349
349
Missing
0
0
0
0
0
0
0
0
Mean
3.542
3.427
3.496
3.453
3.567
3.542
3.756
2.716
Std. Error of Mean
0.056
0.058
0.059
0.059
0.057
0.064
0.062
0.070
Std. Deviation
1.054
1.093
1.095
1.107
1.072
1.197
1.165
1.312
Skewness
-0.510
-0.377
-0.405
-0.417
-0.606
-0.483
-0.711
0.221
Std. Error of Skewness
0.131
0.131
0.131
0.131
0.131
0.131
0.131
0.131
Kurtosis
-0.160
-0.531
-0.481
-0.431
-0.097
-0.648
-0.428
-1.045
Std. Error of Kurtosis
0.260
0.260
0.260
0.260
0.260
0.260
0.260
0.260
Minimum
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
Maximum
5.000
5.000
5.000
5.000
5.000
5.000
5.000
5.000
Journal of Concurrent Disorders, 2022 https://cdspress.ca/
Journal of Concurrent Disorders, 2022
11
Table 5. Fit indices (eHEALS CFA)
Index
Value
Comparative Fit Index (CFI)
0.997
Tucker-Lewis Index (TLI)
0.995
Bentler-Bonett Non-normed Fit Index (NNFI)
0.995
Bentler-Bonett Normed Fit Index (NFI)
0.990
Parsimony Normed Fit Index (PNFI)
0.707
Bollen's Relative Fit Index (RFI)
0.986
Bollen's Incremental Fit Index (IFI)
0.997
Relative Noncentrality Index (RNI)
0.997
Table 6. Factor loadings eight item of eHEALS
95% Confidence Interval
Indicator
Symbol
Estimate
Std. Error
z-value
p
Lower
Upper
Std. Est.
ITEM 1
λ1
0.885
0.033
27.106
< .001
0.787
0.975
0.839
ITEM 2
λ2
0.940
0.034
28.021
< .001
0.854
1.015
0.861
ITEM 3
λ3
0.920
0.034
27.277
< .001
0.827
1.001
0.841
ITEM 4
λ4
0.948
0.034
27.815
< .001
0.853
1.030
0.856
ITEM 5
λ5
0.920
0.034
27.360
< .001
0.829
0.998
0.858
ITEM 6
λ6
0.897
0.035
25.872
< .001
0.777
1.001
0.749
ITEM 7
λ7
0.836
0.034
24.881
< .001
0.717
0.936
0.718
ITEM 8
λ8
0.869
0.035
24.915
< .001
0.751
0.967
0.662
Journal of Concurrent Disorders, 2022 https://cdspress.ca/
Journal of Concurrent Disorders, 2022
12
Figure 1. Factor loading - eight items of eHEALS scale
Criterion/ Construct Validity
After Confirmatory Factorial Analysis, different types of reliability (e.g.,
internal consistency) and validity (e.g., construct validity, criterion validity)
were investigated. To carry out these analysis, different items of the
eHEALS test were correlated with each other, alongside the total of the
eHEALS test itself (Cronbach, & Meehl, 1955). Furthermore, the eHEALS
test total was correlated with several other tests and variables, theoretically
related to the “eHEALS” construct (Campbell & Fiske, 1959). All eight
items of the eHEALS test were found to be significantly and positively
correlated with each other (min=0.496, max=0.834, p<0.01) and with the
eHEALS total test score (min=0.780, max=0.867, p <0.01). The total
eHEALS test correlates significantly (p <0.05) and positively with; the GSE
test (r = 0.178), the internal locus of control test (r = 0.224), the frequency
of searching for information on one's health (r = 0.246), perceived
Technology expertise (r = 0.446), frequency of Internet use (r = 0.282),
frequency of searching for information during quarantine (r = 0.235) and
with perceived Internet expertise (r = 0.494). In addition, the total of the
eHEALS test is positively correlated with self-esteem (r = 0.032, p= 0.550),
although not statistically significant. The total eHEALS test is negatively
correlated with anxiety, although not statistically significant (r = -0.93, p =
0.178) and to the external locus of control (r=-0.146, p <0.05). See Table
7,8 for details.
Journal of Concurrent Disorders, 2022 https://cdspress.ca/
Journal of Concurrent Disorders, 2022
13
Table 7. Pearson's Correlations among the main tests used
Variable
eHEALS
Anxiety
GSE
Self esteem
LOCINT
LOCEST
eHEALS
Anxiety
-0.093
GSE
0.178
***
-0.276
***
Self-esteem
0.032
-0.483
***
0.561
***
LOCINT
0.224
***
-0.137
*
0.576
***
0.367
***
LOCEST
-0.146
**
0.474
***
-0.254
***
-0.419
***
-0.217
***
Note: * p < .05, ** p < .01, *** p < .001. eHEALS= eHEALS scale, Anxiety= PROMIS Emotional Distress Anxiety Short
Form, GSE= General self-efficacy scale , Self-esteem= Rosenberg’s Self-Esteem Scale, LOCINT= Internal locus of control of
behaviour scale , LOCEST= External locus of control of behaviour scale
Journal of Concurrent Disorders, 2022 https://cdspress.ca/
Journal of Concurrent Disorders, 2022
14
Reliability
To analyze the reliability of the eHEALS and internal consistency,
Cronbach’s alpha, Composite Reliability (CR) and McDonald’s Omega
were calculated. In our study, Cronbach's alpha was α= 0.931, McDonald's
Omega was ω= 0.932 and the CR was 0.932 (for a defined construct with
eight items is necessary to meet a minimum threshold of 0.80, Netemeyer
et. al., 2003). These results confirm a strong reliability of the test (see Table
9).
Table 8. Correlations’ matrix: eHealth vs Technological and digital attitude/expertise
Variable
Health
information
search
Technology
expertise
Internet use
Search
information
during
quarantine
Internet
expertise
eHEALS
Health
information
search
Technology
expertise
0.166
**
Internet use
0.059
0.345
***
Search
information
during
quarantine
0.456
***
0.181
***
0.123
*
Internet
expertise
0.113
*
0.751
***
0.342
***
0.174
**
eHEALS
0.246
***
0.446
***
0.282
***
0.235
***
0.494
***
Note: * p < .05, ** p < .01, *** p < .001
Journal of Concurrent Disorders, 2022 https://cdspress.ca/
Journal of Concurrent Disorders, 2022
15
In addition, we conducted an ANOVA (Analysis of Variance)
between the education level groups (defined as A = Middle and High school,
B [n= 136] = University degree [n=154], C = Post-graduate Degree [i.e.,
Ph.D., n= 59]) and the eHEALS total. The results were as follows: F =
11.644, p <0.01 (η² = 0.063) with group C (Post-graduate Degree) having
the highest average (see Table 10,11). The model is therefore significant. In
addition, we verified whether there was a difference in gender in the use of
eHEALS (total scoring), by carrying out an ANOVA with the following
results: F = 0.027, p = 0.871, with the results being statistically insignificant.
Table 9. Item Reliability Statistics (eHeals scale)
If item dropped
Item
ω
α
ITEM 1
0.920
0.920
ITEM 2
0.918
0.918
ITEM 3
0.920
0.920
ITEM 4
0.919
0.918
ITEM 5
0.918
0.918
ITEM 6
0.925
0.925
ITEM 7
0.927
0.927
ITEM 8
0.931
0.930
Table 10. ANOVA Between eHEALS total score and Education level
Cases
Sum of Squares
df
Mean Square
F
p
η²
η² p
Education Level
1232.443
2
616.221
11.644
< .001
0.063
0.063
Residuals
18310.806
346
52.921
Journal of Concurrent Disorders, 2022 https://cdspress.ca/
Journal of Concurrent Disorders, 2022
16
Limitations
A few limitations of this study must be discussed. In the first place,
a convenience sampling was adopted, therefore it may not be as
representative of the reference population compared to a random one.
Secondly, even if the internal coherence of the data was analyzed, it is
acknowledged that self-reports suffer from bias (ultimately classified as
content-related or content-free (Althubaiti, 2016). Lastly, as previously
mentioned, the higher proportion of female participants may also interfere
with results’ inference.
Discussion and Conclusions
Despite these limitations, the eHEALS test appears to be a valid and
reliable scale to measure eHealth competence (i.e., eHealth literacy)
amongst the Italian adult population. The information available to the public
can influence personal health decisions and, subsequently, the effectiveness
and outcome of public health measures implemented by health services. A
critical review of the accessibility, quality and nature of information sources
is now required.
As a result, there is a need for higher quality online health resources
to facilitate public information, this useful for promoting better cooperation
with public health.
The goal of this study was to contribute to strengthen the Italian
version of eHEALS scale (validation and initial translation by Bravo et al.,
2018) making it more valid and reliable, using various statistical techniques
(e.g., Confirmatory Factor Analysis, CFA), and to confirm the relationship
between eHEALS and other measures of convergent/divergent constructs.
The results indicate a one-dimensional structure of the test, confirming what
was previously found by the original study (Norman & Skinner, 2006) and
supporting other international validations (e.g., Pérez et al., 2015; Aponte
& Nokes, 2015; (Mitsutake et al., 2012, 2016; Koo et al., 2012; Van der
Journal of Concurrent Disorders, 2022 https://cdspress.ca/
Journal of Concurrent Disorders, 2022
17
Vaart et al., 2011). Psychometric analysis has shown that eHEALS has good
internal reliability and consistency.
The construct and criterion validity were confirmed by the
significant correlation between the test items and test totals, and by
significant correlations with the GSE test (general self-efficacy scale),
Internal locus of control of behavior scale, Health information search,
technology expertise, Internet use and Internet expertise. Furthermore, even
if not statistically significant, the total test is positively correlated with the
self-esteem test (Rosenberg's Self-Esteem Scale) and negatively correlated
with the Anxiety test (PROMIS Emotional Distress - Anxiety - Short Form).
These results support findings in previous studies (Norman & Skinner,
2006; Starcevic & Berle, 2013). The use of this methodology and scales,
theoretically related to the measurement of eHEALS, allow the present
study to provide strong validity of criterion and construct (e.g., James,
1973).
In fact, a greater sense of self-efficacy was found to be associated
with a greater ability to search for health information correctly (Norman &
Skinner, 2006) which in turn leads to reduced anxiety (Norman & Skinner,
2006). It is also not surprising that a greater perceived sense of technology
and Internet expertise is associated with a higher scoring in the eHEALS
test. In fact, generally, as shown by previous studies (Norman & Skinner,
2006), a greater sense of perceived self-efficacy (in this case, feeling
competent in the use of technology and in the use of the Internet) is
associated with a general better research information about health.
The frequency of searching for information, the frequency of
internet use, and the frequency of use of technology in general, have
previously been found positively correlated to the eHEALS test (Norman &
Skinner, 2006; Wångdahl et al., 2020). Individuals who use technology and
the Internet will most likely indicate an increase in the ability to use
technology as a help tool over time.
The positive correlation between the eHEALS test and an increase
in the search for information in a period of quarantine also supports our
initial hypotheses: the SARS-CoV-2 (COVID-19) pandemic led many
people to research information about the virus, the pandemic, and health in
general (Do et al., 2020).
Furthermore, in previous research (e.g., De Caro et al., 2016), the
eHEALS test was found to be significantly correlated with self-esteem. In
our study, we found a positive, however not statistically significant
correlation for self-esteem, and a positive and statistically significant
correlation with self-efficacy. Self-efficacy and self-esteem, although
connected to each other, are not the same construct (self-esteem is the extent
to which one appreciates, loves, and values oneself (e.g., Smith et al., 2007),
while self-efficacy is defined as judgments that people have about their
abilities to be able to obtain certain types of services (e.g., Bandura, 2010).
This difference may be due to the cultural differences in which these
searches were carried out, or the type of sample recruited.
Journal of Concurrent Disorders, 2022 https://cdspress.ca/
Journal of Concurrent Disorders, 2022
18
The level of education, on the other hand, seems to make a
difference. In our study, a higher total score on the eHEALS test is
associated with higher levels of education. This is not surprising, as more
education is generally associated with better and more effective search
habits for health information (Wångdahl et al., 2020) although literacy
levels decline in the general population after age 45 (Barrett & Riddell,
2019). In conclusion, future research is needed considering an alternative
recruitment method that could guarantee a more representative sample of
the Italian adult population.
Funding
None.
Declaration of conflict of interest
The authors declare that they have no conflict of interest.
Availability of data and material: The data is available in case of a
reasonable request.
Ethics Approval and informed consent
All procedures performed in this study involving human participants
were in accordance with the ethical standards of the research team's
organizational Ethics Board and the 1975 Helsinki Declaration. Informed
consent was obtained from all participants.
Authors Contributions
All authors participated in the writing of the following research.
Appendix:
List of items examined (Italian eHEALS scale):
Item 1: So come trovare su Internet informazioni utili alla salute
Item 2: So come usare Internet per rispondere alle domande riguardanti la
mia salute
Item 3: So quali informazioni sulla salute sono disponibili su Internet
Item 4: So dove trovare su Internet informazioni utili sulla salute
Item 5: So come usare le informazioni sulla salute che trovo su Internet in
modo che mi possano essere d'aiuto
Item 6: Ho le capacità che mi servono per valutare le informazioni sulla
salute che trovo su Internet
Item 7: Posso distinguere la bassa o alta qualità delle informazioni sulla
salute che trovo su Internet
Item 8: Mi sento sicuro nell'usare informazioni che trovo su Internet per
prendere decisioni riguardo la mia salute
Journal of Concurrent Disorders, 2022 https://cdspress.ca/
Journal of Concurrent Disorders, 2022
19
References
Aponte, J., & Nokes, K. M. (2015). Electronic health literacy of older Hispanics with diabetes.
Health Promotion International, 32, 482489.
Althubaiti A. (2016). Information bias in health research: definition, pitfalls, and adjustment
methods. Journal of multidisciplinary healthcare, 9, 211217.
https://doi.org/10.2147/JMDH.S104807.
Bandura, A. (2010). SelfEfficacy. In the corsini encyclopedia of psychology (eds I.B. Weiner and
W.E. Craighead). https://doi.org/10.1002/9780470479216.corpsy0836.
Barrett, G. F., & Riddell, W. C. (2019). Ageing and skills: The case of literacy skills. European
Journal of Education, 54(1), 6071. https://doi.org/10.1111/ejed.12324
Bayrampour, H., Trieu, J., & Tharmaratnam, T. (2019). Effectiveness of eHealth interventions to
reduce perinatal anxiety: A systematic review and meta-analysis. The Journal of clinical
psychiatry, 80(1), 18r12386. https://doi.org/10.4088/JCP.18r12386.
Bollen, K. A., & Long, J. S. (Eds.). (1993). Sage focus editions, Vol. 154.Testing structural
equation models. Sage Publications, Inc.
Boomsma, A. (2000). Reporting analyses of covariance structures. Structural Equation Modeling
A Multidisciplinary Journal. 7. 461-483. doi:10.1207/S15328007SEM0703_6.
Bravo G., Del Giudice P., Poletto M., Battistella C., Conte A., De Odorico A.,Lesa L., Menegazzi
G. & Brusaferro S. (2018). Validazione della versione italiana del questionario di
alfabetizzazione sanitaria digitale (IT-eHEALS). Bollettino epidemiologico nazionale.
Retrieved December 25, 2020, from https://www.epicentro.iss.it/ben/2018/luglio-agosto/2.
Campbell, D. T., & Fiske, D. W. (1959). Convergent and discriminant validation by the multitrait-
multimethod matrix. Psychological Bulletin, 56(2), 81105.
https://doi.org/10.1037/h0046016,
Craig, A. R., Franklin, J. A., & Andrews, G. (1984). A scale to measure locus of control of
behaviour. British Journal of Medical Psychology, 57(2), 173180.
https://doi.org/10.1111/j.2044-8341.1984.tb01597.x.
Cronbach, L. J. (1951). Coefficient alpha and the internal structure of tests. Psychometrika, 16(3),
297 334. https://doi.org/10.1007/bf02310555.
Cronbach, L. J., & Meehl, P. E. (1955). Construct validity in psychological tests. Psychological
Bulletin, 52(4), 281302. https://doi.org/10.1037/h0040957
Cuthbert, G., & Aggarwal, G. (2020, September 08). Digital literacy is even more important today.
Retrieved December 29, 2020, from https://www.capgemini.com/2020/09/digital-literacy-
in-the-times-of-covid/).
De Caro, W., Corvo, E., Marucci, A. R., Mitello, L., Lancia, L., & Sansoni, J. (2016). eHealth
literacy scale: An nursing analisys and italian validation. Studies in health technology and
informatics, 225, 949.
Deady, M., Choi, I., Calvo, R. A., Glozier, N., Christensen, H., & Harvey, S. B. (2017). eHealth
interventions for the prevention of depression and anxiety in the general population: a
systematic review and meta-analysis. BMC psychiatry, 17(1), 310.
https://doi.org/10.1186/s12888-017-1473-1.
Ditzler, N., & Greenhawt, M. (2016). Influence of health literacy and trust in online information
on food allergy quality of life and self-efficacy. Annals of allergy, asthma & immunology
: official publication of the American College of Allergy, Asthma, & Immunology, 117(3),
258263.e1. https://doi.org/10.1016/j.anai.2016.07.011.
Journal of Concurrent Disorders, 2022 https://cdspress.ca/
Journal of Concurrent Disorders, 2022
20
Do, B. N., Tran, T. V., Phan, D. T., Nguyen, H. C., Nguyen, T., Nguyen, H. C., Ha, T. H., Dao,
H. K., Trinh, M. V., Do, T. V., Nguyen, H. Q., Vo, T. T., Nguyen, N., Tran, C. Q., Tran,
K. V., Duong, T. T., Pham, H. X., Nguyen, L. V., Nguyen, K. T., Chang, P., … Duong, T.
V. (2020). Health Literacy, eHealth Literacy, Adherence to Infection Prevention and
Control Procedures, Lifestyle Changes, and Suspected COVID-19 Symptoms Among
Health Care Workers During Lockdown: Online Survey. Journal of medical Internet
research, 22(11), e22894. https://doi.org/10.2196/22894.
Lévy, P. (1939). L'addition des variables aléatoires définies sur une circonférence. Bulletin de la
Société mathématique de France, 67, 1-41.
Edgar, L., Greenberg, A. & Remmer, J. (2002). Providing Internet lessons to oncology patients
and family members: A shared project. Psycho-Oncology, 11, 439- 446.
Edwards, B. D. (2010). Book Review: Timothy A. Brown. (2006). Confirmatory factor analysis
for applied research. New York: Guilford. Organizational Research Methods, 13(1), 214
217. https://doi.org/10.1177/1094428108323758.
Eysenbach G, Jadad AR (2001). Consumer health informatics in the Internet age. In: Edwards A,
Elwyn G, editors. Evidence-based Patient Choice: inevitabile o impossibile. Oxford, UK:
Oxford University Press;. pp. 289 307.
Eysenbach G. (2002). Infodemiology: The epidemiology of (mis)information. The American
journal of medicine, 113(9), 763765. https://doi.org/10.1016/s0002-9343(02)01473-0.
Fan, K. S., Ghani, S. A., Machairas, N., Lenti, L., Fan, K. H., Richardson, D., . . . Raptis, D. A.
(2020). Covid-19 prevention and treatment information on the internet: A systematic
analysis and quality assessment. BMJ Open, 10(9). doi:10.1136/bmjopen-2020-040487.
Farma, T. & Cortinovis, I. (2001). “Un questionario sul “Locus of Control” : suo utilizzo nel
contesto italiano”. Ricerca in Psicoterapia. 1.
Ferguson, E., & Cox, T. (1993). Exploratory factor analysis: A users’ guide. International Journal
of Selection and Assessment, 1, 84-94. http://dx.doi.org/10.1111/j.1468-
2389.1993.tb00092.x.
Ferrando, P., & Lorenzo-Seva, U. (2017). Program FACTOR at 10: Origins, development and
future directions. Psicothema, 29, 236240. https://doi.org/10.7334/psicothema2016.304.
Fornell, C., & Larcker, D. F. (1981). Evaluating structural equation models with unobservable
variables and measurement error. Journal of Marketing Research, 18(1), 3950.
https://doi.org/10.2307/3151312.
Fossati, A., Borroni, S., & Del Corno, F. (2015). Scale di valutazione adulti - American Psychiatric
Association - Raffaello Cortina Editore - Ebook Raffaello Cortina Editore. Retrieved June
21, 2020, from http://www.raffaellocortina.it/scheda-ebook/american-psychiatric-
association/scale-di-valutazione-adulti-9788860307668-2150.html.
Gorsuch, R. L. (1983). Factor analysis (2nd Edition). Hillsdale, NJ: Erlbaum.
Hair, J. F., Black, W. C., Babin, B. J., & Anderson, R. E. (2010). Multivariate data analysis:
International version. New Jersey, Pearson.
Hirji, J. (2004). Freedom or folly? Canadians and the consumption of online health information.
Information, Communication and Society, 7(4), 445-465.
Hu, L.-t., & Bentler, P. M. (1999). Cutoff criteria for fit indexes in covariance structure analysis:
Conventional criteria versus new alternatives. Structural Equation Modeling, 6(1), 1
55. https://doi.org/10.1080/10705519909540118.
IBM Corp. (2011). IBM SPSS statistics for windows, Version 20.0. Armonk, NY: IBM Corp.
Journal of Concurrent Disorders, 2022 https://cdspress.ca/
Journal of Concurrent Disorders, 2022
21
Institute of Medicine (US) Committee on Health Literacy; Nielsen-Bohlman L, Panzer AM,
Kindig DA, editors. Health Literacy: A Prescription to End Confusion. Washington (DC):
National Academies Press (US); 2004. 1, Introduction. Available from:
https://www.ncbi.nlm.nih.gov/books/NBK216033/.
Institute of Medicine (US) Roundtable on Health Literacy. Innovations in Health Literacy
Research: Workshop Summary. Washington (DC): National Academies Press (US); 2011.
4, The Role of Health Literacy in Health Information Technology. Available from:
https://www.ncbi.nlm.nih.gov/books/NBK209676/.
James, L. R. (1973). Criterion models and construct validity for criteria. Psychological Bulletin,
80(1), 7583. https://doi.org/10.1037/h0034627.
JASP Team (2020). JASP (Version 0.13.1) [Computer software]. Retrieved October 21, 2020,
from: https://jasp-stats.org/.
Kadam, P., & Bhalerao, S. (2010). Sample size calculation. International journal of Ayurveda
research, 1(1), 5557. https://doi.org/10.4103/0974-7788.59946.
Kim, H. (2008). Common factor analysis versus principal component analysis: Choice for
symptom cluster research. Asian Nursing Research, 2(1), 17-24. doi:10.1016/s1976-
1317(08)60025-0.
Klecun, Ela. (2010). Digital literacy for health: The promise of health 2.0. IJDLDC. 1. 48-57.
10.4018/jdldc.2010070105.
Kline, R. B. (2015). Principles and practice of structural equation modeling (4th ed.). New York:
Guilford Publications.
Koğar, Hakan & Yilmaz Kogar, Esin. (2015). Comparison of different estimation methods for
categorical and ordinal data in confirmatory factor analysis. Eğitimde ve Psikolojide Ölçme
ve Değerlendirme Dergisi. 6. 10.21031/epod.94857.
Koo, M. & Norman, Cameron & Chang, H.-M. (2012). Psychometric evaluation of a Chinese
version of the eHealth Literacy Scale (eHEALS) in school age children. International
Electronic Journal of Health Education. 15. 29-36.
Lorenzo-Seva, U., & Ferrando, P. J. (2006). FACTOR: A computer program to fit the exploratory
factor analysis model. Behavior Research Methods, 38, 8891.
https://doi.org/10.3758/BF03192753.
Mann Whitney U test calculator [Internet]. Statistics Kingdom 2017. Retrieved October 21, 2020,
from: http://www.statskingdom.com/170median_mann_whitney.html .
Mishra, P., Pandey, C. M., Singh, U., Gupta, A., Sahu, C., & Keshri, A. (2019). Descriptive
statistics and normality tests for statistical data. Annals of cardiac anaesthesia, 22(1), 67
72. https://doi.org/10.4103/aca.ACA_157_18.
McDonald R. P. (1999). Test theory: A unified treatment. Mahwah, NJ: Erlbaum.
Mitsutake, S., Shibata, A., Ishii, K., & Oka, K. (2012). Association of eHealth literacy with
colorectal cancer knowledge and screening practice among internet users in Japan. Journal
of Medical Internet Research, 14, e153.
Mitsutake, S., Shibata, A., Ishii, K., & Oka, K. (2016). Associations of eHealth Literacy with
health behavior among adult internet users. Journal of medical Internet research, 18(7),
e192. https://doi.org/10.2196/jmir.5413.
Muthén, B., & Kaplan, D. (1985). A comparison of some methodologies for the factor analysis of
non-normal Likert variables. British Journal of Mathematical and Statistical Psychology,
38(2), 171189. https://doi.org/10.1111/j.2044-8317.1985.tb00832.x.
Journal of Concurrent Disorders, 2022 https://cdspress.ca/
Journal of Concurrent Disorders, 2022
22
Netemeyer, R. G., Bearden, W. O., & Sharma, S. (2003). Scaling procedures. Thousand Oaks,
CA: Sage Publications, Inc.
Nguyen, J., Moorhouse, M., Curbow, B., Christie, J., Walsh-Childers, K., & Islam, S. (2016).
Construct validity of the eHealth literacy scale (eHEALS) among two adult populations: A
Rasch analysis. JMIR Public Health and Surveillance, 2, e24.
Norman, C. D., & Skinner, H. A. (2006). eHEALS: The eHealth literacy scale. Journal of Medical
Internet Research, 8, e27.
Pew Research Center. (2016). Smartphone ownership and internet usage continues to climb in
emerging economies. Retrieved August 16, 2018, from http://www.
pewglobal.org/2016/02/22/smartphone-ownership-and-internet-usage-contin ues-to-
climb-in-emerging-economies/.
Paige, S. R., Krieger, J. L., Stellefson, M., & Alber, J. M. (2017). eHealth literacy in chronic
disease patients: an item response theory analysis of the eHealth literacy scale (eHEALS).
Patient Education and Counseling, 100(2), 320-326.
Paramio Pérez, G., Almagro, B. J., Hernando Gómez, Á., & Aguaded Gómez, J. I. (2015).
Validación de la escala eHealth Literacy (eHEALS) en población universitaria española
[Validation of the eHealth Literacy Scale (eHEALS) in Spanish University
Students]. Revista espanola de salud publica, 89(3), 329338.
https://doi.org/10.4321/S1135-57272015000300010.
Pilkonis, P. A., Choi, S. W., Reise, S. P., Stover, A. M., Riley, W. T., Cella, D., & PROMIS
Cooperative Group. (2011). Item banks for measuring emotional distress from the patient-
reported outcomes measurement information system (PROMIS®): Depression, anxiety,
and anger. Assessment, 18(3), 263283.
Prezza, M., Trombaccia, F. R., & Armento, L. (1997). La scala dell’autostima di Rosenberg.
Traduzione e validazione italiana [Rosenberg Self-Esteem Scale. Italian translation and
validation]. Bollettino di Psicologia Applicata, 223, 3544.
Ratzan, S & Parker, Ruth & Selden, C & Zorn, Marcia. (2000). National Library of Medicine
Current Bibliographies in Medicine: Health Literacy. Bethesda, MD: National Institutes
of Health.
Robbins, D., & Dunn, P. (2019). Digital health literacy in a person-centric world. International
Journal of Cardiology, 290, 154-155. doi:10.1016/j.ijcard.2019.05.033.
Roscoe, J.T. (1975) Fundamental research statistics for the behavioral science, International Series
in Decision Process, 2nd Edition, Holt, Rinehart and Winston, Inc., New York.
Rosenberg, M. (1965). Society and the adolescent self-image. Princeton: Princeton University
Press.
Schumacker, R. E., & Lomax, R. G. (2010). A beginner’s guide to structural equation
modeling (3rd ed.). New York, NY: Routledge Academic.
Schwarzer, R., & Jerusalem, M. (1995). Generalized self-efficacy scale. J. Weinman, S. Wright, &
M. Johnston, Measures in health psychology: A user’s portfolio. Causal and control beliefs,
35, 37.
Sibilia, L., Schwarzer, R., & Jerusalem, M. (1995). Italian adaptation of the general self-efficacy
scale. Available at: userpage. fu-berlin. de/~ health/italian. htm (Accessed 07 July 2020).
Smith, E. R.; Mackie, D. M. (2007). Social Psychology (Third ed.). Hove: Psychology Press.
Starcevic, V., Berle, D. (2013) Cyberchondria: towards a better understanding of excessive health-
related Internet use. Expert Rev Neurother; 13(2):205213. doi:10.1586/ern.12.162.
Journal of Concurrent Disorders, 2022 https://cdspress.ca/
Journal of Concurrent Disorders, 2022
23
Tchebycheff, P. (1890). Sur deux théorèmes relatifs aux probabilités. Acta mathematica, 14(1),
305.
Vaart, R. V., & Drossaert, C. (2017). Development of the digital health literacy instrument:
Measuring a broad spectrum of health 1.0 and health 2.0 skills. Journal of Medical Internet
Research, 19(1). doi:10.2196/jmir.6709.
Vajaean C. & Baban, A. (2015). Emotional and behavioral consequences of online health
information-seeking: The role of eHealth literacy. Cognition, Brain, Behavior. 19. 327-
345.
Van der Vaart, R., van Deursen, A. J., Drossaert, C. H., Taal, E., van Dijk, J. A., & van de Laar,
M. A. (2011). Does the eHealth literacy scale (eHEALS) measure what it intends to
measure? Validation of a Dutch version of the eHEALS in two adult populations. Journal
of Medical Internet Research, 13, e86.
Wångdahl, J., Jaensson, M., Dahlberg, K., & Nilsson, U. (2020). The Swedish Version of the
Electronic Health Literacy Scale: Prospective Psychometric Evaluation Study Including
Thresholds Levels. JMIR mHealth and uHealth, 8(2), e16316.
https://doi.org/10.2196/16316.
Wakefield, B. J., Turvey, C. L., Nazi, K. M., Holman, J. E., Hogan, T. P., Shimada, S. L., &
Kennedy, D. R. (2017). Psychometric properties of patient-facing eHealth evaluation
measures: Systematic review and analysis. Journal of medical Internet research, 19(10),
e346. https://doi.org/10.2196/jmir.7638.
Wolf, A., Fors, A., Ulin, K., Thorn, J., Swedberg, K., & Ekman, I. (2016). An eHealth diary and
symptom-tracking tool combined with person-centered care for improving self-efficacy
after a diagnosis of acute coronary syndrome: A substudy of a randomized controlled
trial. Journal of medical Internet research, 18(2), e40. https://doi.org/10.2196/jmir.4890.
Wyatt, J., Allison, S., Donoghue, D., Horton, P., & Kearney, K. (2003). Evaluation of CMF funded
UK online centres-final report. Hall Aitken/Department for Education and Skills, 96.
ResearchGate has not been able to resolve any citations for this publication.
Article
Full-text available
Objective To evaluate the quality of information regarding the prevention and treatment of COVID-19 available to the general public from all countries. Design Systematic analysis using the ‘Ensuring Quality Information for Patients’ (EQIP) Tool (score 0–36), Journal of American Medical Association (JAMA) benchmark (score 0–4) and the DISCERN Tool (score 16–80) to analyse websites containing information targeted at the general public. Data sources Twelve popular search terms, including ‘Coronavirus’, ‘COVID-19 19’, ‘Wuhan virus’, ‘How to treat coronavirus’ and ‘COVID-19 19 Prevention’ were identified by ‘Google AdWords’ and ‘Google Trends’. Unique links from the first 10 pages for each search term were identified and evaluated on its quality of information. Eligibility criteria for selecting studies All websites written in the English language, and provides information on prevention or treatment of COVID-19 intended for the general public were considered eligible. Any websites intended for professionals, or specific isolated populations, such as students from one particular school, were excluded, as well as websites with only video content, marketing content, daily caseload update or news dashboard pages with no health information. Results Of the 1275 identified websites, 321 (25%) were eligible for analysis. The overall EQIP, JAMA and DISCERN scores were 17.8, 2.7 and 38.0, respectively. Websites originated from 34 countries, with the majority from the USA (55%). News Services (50%) and Government/ Health Departments (27%) were the most common sources of information and their information quality varied significantly. Majority of websites discuss prevention alone despite popular search trends of COVID-19 treatment. Websites discussing both prevention and treatment (n=73, 23%) score significantly higher across all tools (p<0.001). Conclusion This comprehensive assessment of online COVID-19 information using EQIP, JAMA and DISCERN Tools indicate that most websites were inadequate. This necessitates improvements in online resources to facilitate public health measures during the pandemic.
Article
Full-text available
Background: To enhance the efficacy of information and communication, health care has increasingly turned to digitalisation. Electronic health is an important factor that influences the use and receipt of benefits from web-based health resources. Consequently, the concept of eHealth literacy has emerged and in 2006 Norman and Skinner developed an 8-item self-report eHealth literacy scale to measure these skills: the eHealth Literacy Scale (eHEALS). However, the eHEALS has not been tested for reliability and validity in the general Swedish population as well as there are no threshold levels for eHEALS. Objective: The aim of this study was to translate and adapt the eHEALS into a Swedish version, to evaluate convergent validity, psychometric properties as well as to determine threshold levels for inadequate, problematic, and sufficient eHealth literacy. Method: Prospective psychometric evaluation study including 323 participants equally distributed between sex and with a mean age of 49 years. Results: There were some difficulties translating the English concept Health resources. This resulted in this concept being translated as Health information, i.e. Hälsoinformation in Swedish. The eHEALS total score was 29.3 (SD 6.2), Cronbach’s alpha 0.94, Spearman-Brown coefficient 0.96 and with a response rate of 94.6%. All a priori hypotheses were confirmed, supporting convergent validity. The test-retest reliability indicated an almost perfect agreement, 0.86 (p< 0.001). An exploratory factor analysis found one component explaining 64% of the total variance. No floor or ceiling effect was noted. Thresholds levels were set at 8–20 = inadequate, 21–26 = problematic, and 27–40 = sufficient, and there were no significant differences in distribution of the three levels between Sw-eHEALS and HLS-EU-Q16. Conclusions: The Swedish version of eHEALS was assessed as being unidimensional and the internal consistency of the instrument high, making the reliability adequate. Adapted threshold levels for inadequate, problematic, and sufficient levels of eHealth literacy seem to be relevant. However, there are some linguistic issues relating to the concept of Health resources.
Article
Full-text available
Descriptive statistics are an important part of biomedical research which is used to describe the basic features of the data in the study. They provide simple summaries about the sample and the measures. Measures of the central tendency and dispersion are used to describe the quantitative data. For the continuous data, test of the normality is an important step for deciding the measures of central tendency and statistical methods for data analysis. When our data follow normal distribution, parametric tests otherwise nonparametric methods are used to compare the groups. There are different methods used to test the normality of data, including numerical and visual methods, and each method has its own advantages and disadvantages. In the present study, we have discussed the summary measures and methods used to test the normality of the data
Article
Full-text available
Background Significant resources are being invested into eHealth technology to improve health care. Few resources have focused on evaluating the impact of use on patient outcomes A standardized set of metrics used across health systems and research will enable aggregation of data to inform improved implementation, clinical practice, and ultimately health outcomes associated with use of patient-facing eHealth technologies. Objective The objective of this project was to conduct a systematic review to (1) identify existing instruments for eHealth research and implementation evaluation from the patient’s point of view, (2) characterize measurement components, and (3) assess psychometrics. Methods Concepts from existing models and published studies of technology use and adoption were identified and used to inform a search strategy. Search terms were broadly categorized as platforms (eg, email), measurement (eg, survey), function/information use (eg, self-management), health care occupations (eg, nurse), and eHealth/telemedicine (eg, mHealth). A computerized database search was conducted through June 2014. Included articles (1) described development of an instrument, or (2) used an instrument that could be traced back to its original publication, or (3) modified an instrument, and (4) with full text in English language, and (5) focused on the patient perspective on technology, including patient preferences and satisfaction, engagement with technology, usability, competency and fluency with technology, computer literacy, and trust in and acceptance of technology. The review was limited to instruments that reported at least one psychometric property. Excluded were investigator-developed measures, disease-specific assessments delivered via technology or telephone (eg, a cancer-coping measure delivered via computer survey), and measures focused primarily on clinician use (eg, the electronic health record). Results The search strategy yielded 47,320 articles. Following elimination of duplicates and non-English language publications (n=14,550) and books (n=27), another 31,647 articles were excluded through review of titles. Following a review of the abstracts of the remaining 1096 articles, 68 were retained for full-text review. Of these, 16 described an instrument and six used an instrument; one instrument was drawn from the GEM database, resulting in 23 articles for inclusion. None included a complete psychometric evaluation. The most frequently assessed property was internal consistency (21/23, 91%). Testing for aspects of validity ranged from 48% (11/23) to 78% (18/23). Approximately half (13/23, 57%) reported how to score the instrument. Only six (26%) assessed the readability of the instrument for end users, although all the measures rely on self-report. Conclusions Although most measures identified in this review were published after the year 2000, rapidly changing technology makes instrument development challenging. Platform-agnostic measures need to be developed that focus on concepts important for use of any type of eHealth innovation. At present, there are important gaps in the availability of psychometrically sound measures to evaluate eHealth technologies.
Article
Full-text available
Background Anxiety and depression are associated with a range of adverse outcomes and represent a large global burden to individuals and health care systems. Prevention programs are an important way to avert a proportion of the burden associated with such conditions both at a clinical and subclinical level. eHealth interventions provide an opportunity to offer accessible, acceptable, easily disseminated globally low-cost interventions on a wide scale. However, the efficacy of these programs remains unclear. The aim of this study is to review and evaluate the effects of eHealth prevention interventions for anxiety and depression. Method A systematic search was conducted on four relevant databases to identify randomized controlled trials of eHealth interventions aimed at the prevention of anxiety and depression in the general population published between 2000 and January 2016. The quality of studies was assessed and a meta-analysis was performed using pooled effect size estimates obtained from a random effects model. Results Ten trials were included in the systematic review and meta-analysis. All studies were of sufficient quality and utilized cognitive behavioural techniques. At post-treatment, the overall mean difference between the intervention and control groups was 0.25 (95% confidence internal: 0.09, 0.41; p = 0.003) for depression outcome studies and 0.31 (95% CI: 0.10, 0.52; p = 0.004) for anxiety outcome studies, indicating a small but positive effect of the eHealth interventions. The effect sizes for universal and indicated/selective interventions were similar (0.29 and 0.25 respectively). However, there was inadequate evidence to suggest that such interventions have an effect on long-term disorder incidence rates. Conclusions Evidence suggests that eHealth prevention interventions for anxiety and depression are associated with small but positive effects on symptom reduction. However, there is inadequate evidence on the medium to long-term effect of such interventions, and importantly, on the reduction of incidence of disorders. Further work to explore the impact of eHealth psychological interventions on long-term incidence rates.
Article
The digital age is beginning to impact the healthcare system. Smartphones and other devices based on cellular technology have made access to information ubiquitous among consumers throughout the world. There has been a shift from devices that collect data to systems for those medical conditions, such as atrial fibrillation. This changes the focus from health literacy to digital health literacy and the information-communication between the healthcare professional and the individual. Moving from health literacy to digital health literacy, therefore also means shifting from patients to persons and from managing health to empowering people to live a healthier life. Digital solutions will uncover an even greater tool, the engaged patient.
Article
The relationship between ageing and skills is of growing policy significance due to population ageing, the changing nature of work and the importance of literacy for social and economic well‐being. This article examines the relationship between age and literacy skills in a sample of OECD countries using three internationally comparable surveys. By pooling the survey data across time we can separate birth cohort and ageing effects. In doing so, we find that literacy skills decline with age and that, in most of our sample countries, successive birth cohorts tend to have poorer literacy outcomes. Therefore, once we control for cohort effects, the rate at which literacy proficiency falls with age is much more pronounced than that which is apparent, based on the cross‐sectional relationship between age and literacy skills at a point in time. Further, in studying the literacy‐age relationship across the skill distribution in Canada we find a more pronounced decline in literacy skills with age at lower percentiles, which suggests that higher initial literacy moderates the influence of cognitive ageing.
Article
Objective: eHealth interventions have been shown to be effective in improving anxiety among the general population. Despite the effectiveness of eHealth interventions for perinatal depression, a recent review reported mixed results for perinatal anxiety. The review, however, was not focused on anxiety, and studies with various designs were included. The aim of this systematic review is to summarize the evidence specific to anxiety and to conduct a meta-analysis to examine the effectiveness of eHealth interventions in reducing perinatal anxiety. Data sources: MEDLINE, CINAHL, EMBASE, and PsycINFO were searched beginning with the date that the databases were available through March 2018 using keywords such as perinatal period, web-based interventions, and anxiety. Study selection: Randomized controlled trials that were conducted during the perinatal period, examined the effectiveness of an eHealth mental health intervention, measured anxiety symptoms or disorders as a primary or secondary outcome, provided data on anxiety levels both pre-intervention and post-intervention, had a comparison group, and were published in English were included. A total of 770 articles were retrieved, and the full texts of 64 articles were reviewed. Five studies met the inclusion criteria, 4 of which fulfilled the quality criteria and were included in the meta-analysis. Data extraction: Data were extracted using a data extraction form developed for this study. The Cochrane Collaboration's Review Manager software was used to conduct the meta-analysis. Results: The test for heterogeneity (I² = 0%; P = .80) suggested a homogeneous sample. The meta-analysis for the total effect size showed that at post-intervention, the eHealth group had significantly lower anxiety scores than the control group, with a standardized mean difference of -0.41 (95% CI, -0.71 to -0.11; P = .007). Conclusions: eHealth interventions are promising in improving perinatal anxiety. The content of these interventions should account for common comorbid mental health conditions during the perinatal period and provide opportunities to tailor further treatment if necessary.
Background: One of the scales most used to measure quickly and easily eHealth Literacy is the eHealth LiteracyScale (eHEALS); however, there was no validation of this scale in Italian. Therefore, the aim of this study was to adapt and validate the eHealth Literacy Scale (eHEALS) to the italian context. Methods: Italian translation of eHEALS was administered along unit to another two scale for measure lifestyle habits self-esteem and life satisfaction). A sample of 650 university students aged between 18 and 45 years was selected. An exploratory factor analysis, confirmatory factor analysis, analysis of invariance, reliability, stability and bivariate correlations were performed. Results: Exploratory factor analysis revealed a monofactorial structure that explained 67% of variance. Reliability of 0.87 and test-retest correlation of 0.78 was obtained. The questionnaire was invariant by gender. Regarding the criterion validity, a statistically significant and positive correlations between 0.05 and 0.15 with three indicators was obtained (self-esteem, lifestyle habits and life satisfaction). The italian version of the eHEALS tested in this work has shown to be a valid and reliable scale to measure eHealth competence in university students.