PreprintPDF Available

Pandemic Programming How COVID-19 affects software developers and how their organizations can help

Authors:

Abstract and Figures

Background. As a novel coronavirus swept the world in early 2020, thousands of software developers began working from home. Many did so on short notice , under difficult and stressful conditions. Aim. This paper seeks to understand the effects of the pandemic on developers' wellbeing and productivity. Method. A questionnaire survey was created mainly from existing, validated scales. The questionnaire ran in 12 languages, with region-specific advertising strategies. The data was analyzed using non-parametric inferential statistics and structural equation modeling. Results. The questionnaire received 2225 usable responses from 53 countries. Factor analysis supported the validity of the scales and the structural model achieved a good fit (χ 2 fit = 10.8, CFI = 0.961, RMSEA = 0.051, SRMR = 0.067). Findings include: (1) developers' wellbeing and productivity are suffering; (2) productivity and wellbeing are closely related ; (3) disaster preparedness, fear related to the pandemic and home office ergonomics all affect wellbeing or productivity; (4) women, parents and people with disabilities may be disproportionately affected. Conclusion. To improve employee productivity, software companies should focus on maximizing employee wellbeing and improving the ergonomics of employees' home offices. Women, parents and disabled persons may require extra support.
Content may be subject to copyright.
Empirical Software Engineering manuscript No.
(will be inserted by the editor)
Pandemic Programming
How COVID-19 affects software developers and how their
organizations can help
Paul Ralph ·Sebastian Baltes ·Gianisa
Adisaputri ·Richard Torkar ·Vladimir
Kovalenko ·Marcos Kalinowski ·
Nicole Novielli ·Shin Yoo ·Xavier
Devroey ·Xin Tan ·Minghui Zhou ·
Burak Turhan ·Rashina Hoda ·Hideaki
Hata ·Gregorio Robles ·Amin Milani
Fard ·Rana Alkadhi
Received: date / Accepted: date
Abstract
Background. As a novel coronavirus swept the world in early 2020, thousands
of software developers began working from home. Many did so on short no-
tice, under difficult and stressful conditions. Aim. This paper seeks to under-
stand the effects of the pandemic on developers’ wellbeing and productivity.
Method. A questionnaire survey was created mainly from existing, validated
scales. The questionnaire ran in 12 languages, with region-specific advertising
strategies. The data was analyzed using non-parametric inferential statistics
and structural equation modeling. Results. The questionnaire received 2225 us-
able responses from 53 countries. Factor analysis supported the validity of the
scales and the structural model achieved a good fit (χ2
fit = 10.8, CFI = 0.961,
RMSEA = 0.051, SRMR = 0.067). Findings include: (1) developers’ wellbeing
P. Ralph, Dalhousie University, E-mail: paulralph@dal.ca
S. Baltes, The University of Adelaide, E-mail: sebastian.baltes@adelaide.edu.au
G. Adisaputri, Adisa Emergency and Disaster Management, E-mail: gianisaa@gmail.com
R. Torkar, Chalmers and University of Gothenburg, E-mail: richard.torkar@cse.gu.se
V. Kovalenko, JetBrains E-mail: vladimir.kovalenko@jetbrains.com
M. Kalinowski, Pontifical Catholic Uni. of Rio de Janeiro, E-mail: kalinowski@inf.puc-rio.br
N. Novielli, University of Bari Aldo Moro E-mail: nicole.novielli@uniba.it
S. Yoo, KAIST, E-mail: shin.yoo@kaist.ac.kr
X. Devroey, Delft University of Technology E-mail: X.D.M.Devroey@tudelft.nl
X. Tan, Peking University, E-mail: tanxin16@pku.edu.cn
M. Zhou, Peking University, E-mail: zhmh@pku.edu.cn
B. Turhan, Monash University & University of Oulu, E-mail: burak.turhan@monash.edu
R. Hoda, Monash University E-mail: rashina.hoda@monash.edu
H. Hata, Nara Institute of Science and Technology E-mail: hata@is.naist.jp
G. Robles, Universidad Rey Juan Carlos E-mail: grex@gsyc.urjc.es
A. Milani Fard, New York Institute of Technology E-mail: amilanif@nyit.edu
R. Alkadhi, King Saud University E-mail: ralkadi@ksu.edu.sa
2 Paul Ralph et al.
and productivity are suffering; (2) productivity and wellbeing are closely re-
lated; (3) disaster preparedness, fear related to the pandemic and home office
ergonomics all affect wellbeing or productivity; (4) women, parents and people
with disabilities may be disproportionately affected. Conclusion. To improve
employee productivity, software companies should focus on maximizing em-
ployee wellbeing and improving the ergonomics of employees’ home offices.
Women, parents and disabled persons may require extra support.
Keywords Software development ·Work from home ·Crisis management ·
Disaster management ·Emergency management ·Wellbeing ·Productivity ·
COVID-19 ·Pandemic ·Questionnaire ·Structural equation modeling
1 Introduction
In December 2019, a novel coronavirus disease (COVID-19) emerged in Wuhan,
China. While symptoms vary, COVID-19 often produces fever, cough, short-
ness of breath, and in some cases, pneumonia and death. By April 30, 2020,
The World Health Organization (WHO) recorded more than 3 million con-
firmed cases and 217,769 deaths (WHO, 2020a). With wide-spread transmis-
sions in 214 countries, territories or areas, the WHO declared it a Public Health
Emergency of International Concern (WHO, 2020b) and many jurisdictions
declared states of emergency or lockdowns (Kaplan et al., 2020). Many tech-
nology companies told their employees to work from home (Duffy, 2020).
Quarantine work !== Remote work. I’ve been working remotely with suc-
cess for 13 years, and I’ve never been close to burn out. I’ve been working
quarantined for over a month and I’m feeling a tinge of burn out for the
first time in my life. Take care of yourself folks. Really.
–Scott Hanselman (April 19, 2020)
Thinking of this situation as a global natural experiment in working from
home—the event that would prove once and for all that working from home,
well, works—would be na¨ıve. This is not normal working from home. This is
working from home, unexpectedly, during an unprecedented crisis. The normal
benefits of working from home do not apply (Donnelly and Proctor-Thomson,
2015). Rather than working in a remote office or well-appointed home office,
some people are working in impromptu in bedrooms, at kitchen tables and on
sofas while partners, children, siblings, parents, roommates, and pets distract
them. Others are spending all day alone in a studio or one-bedroom apartment.
With schools and daycare closed, many parents must juggle work with not only
childcare but also home schooling or monitoring remote schooling activities
and keeping children engaged. Some professionals have the virus or are caring
for family members with the virus.
Pandemic Programming 3
While numerous studies have investigated remote work, few investigate
working from home during disasters. There are no modern studies of working
from home during a pandemic of this magnitude because there has not been a
pandemic of this magnitude since before there was a world wide web. There-
fore, software companies have limited evidence on which to base attempts to
support their workers through this crisis, which raises the following research
question.
Research questions: How is working from home during the COVID-19 pan-
demic affecting software developers’ emotional wellbeing and productivity?
In addressing this question, this paper generates and evaluates a theoreti-
cal model for explaining and predicting changes in wellbeing and productivity
while working from home during a crisis. Moreover, we provide recommenda-
tions for professionals and organizations to support employees who are working
from home due to COVID-19 or future disasters.
This paper is organized as follows. Section 2 provides needed background
on pandemics, productivity and working from home. Section 3 introduces our
hypotheses and nomological model. Section 4 describes the design of the pan-
demic programming questionnaire and our sampling strategy. Next, we de-
scribe our analysis and results (Section 5), followed by the study’s limitations
and implications (Section 6). Section 7 concludes the paper with a summary
of its contributions.
2 Background
To fully understand this study, we need to review three areas of related work:
(1) pandemics, bioevents and disasters; (2) working from home; (3) produc-
tivity.
2.1 Pandemics, bioevents and disasters
Madhav et al. (2017) defines pandemics as “large-scale outbreaks of infec-
tious disease over a wide geographic area that can greatly increase morbidity
and mortality and cause significant economic, social, and political disruption”
(p. 35). Pandemics can be very stressful not only for those who become in-
fected but also for those caring for the infected and worrying about the health
of themselves, their families and their friends (Kim et al., 2015; Prati et al.,
2011). In a recent poll, “half of Canadians (50%) report[ed] a worsening of
their mental health” during the COVID-19 lockdown (ARI, 2020).
A pandemic can be mitigated in several ways including social distancing
(Anderson et al., 2020). “Social distancing refers to a set of practices that aim
to reduce disease transmission through physical separation of individuals in
community settings” (Rebmann, 2009, p. 120-14). It may include public facility
shutdowns, home quarantine, cancelling large public gatherings, working from
home or reducing the number of workers in the same place at the same time,
4 Paul Ralph et al.
maintaining a distance of at least 1.5–2m between people, and other attempts
to reduce contact rate (Rebmann, 2009; Anderson et al., 2020).
The extent to which individuals comply with recommendations varies sig-
nificantly and is affected by many factors. People are more likely to comply
when they have more self-efficacy; that is, confidence that they can stay at
home or keep working during the pandemic, and when they perceive the risks
as high (Teasdale et al., 2012). However, this “threat appraisal” depends on:
the psychological process of quantifying risk, sociocultural perspectives (e.g.
one’s worldview and beliefs; how worried one’s friends are), “illusiveness of pre-
paredness” (e.g. fatalistic attitudes and denial), beliefs about who is respon-
sible for mitigating risks (e.g. individuals or governments) and how prepared
one feels (Yong et al., 2017; Yong and Lemyre, 2019; Prati et al., 2011).
People are less likely to comply when they are facing loss of income, per-
sonal logistical problems (e.g. how to get groceries), isolation, and psycho-
logical stress (e.g. infection fears, boredom, frustration, stigma) (DiGiovanni
et al., 2004). Barriers to following recommendations include job insecurity, lack
of childcare, guilt and anxiety about work not being completed, and personal
cost of following government advice (Teasdale et al., 2012; Blake et al., 2010).
For employees, experiencing negative life events such as disasters is asso-
ciated with absenteeism and lower quality of workdays (North et al., 2010).
Employers therefore need work-specific strategies and support for their em-
ployees. Employers can give employees a sense of security and help them return
to work by continuing to pay full salaries on time, reassuring employees they
they are not going to lose their job, having flexible work demands, implement-
ing an organized communication strategy, and ensuring access to utilities (e.g.
telephone, internet, water, electricity, sanitation) and organisational resources
(North et al., 2010; Donnelly and Proctor-Thomson, 2015; Blake et al., 2010).
Work-specific strategies and support are also needed to ensure business
continuation and survival. The disruption of activities in disasters simultane-
ously curtails revenues and reduces productive capacity due to the ambiguity
and priorities shifting in individuals, organizations and communities (Don-
nelly and Proctor-Thomson, 2015). As social distancing closes worksites and
reduces commerce, governments face increased economic pressure to end social
distancing requirements prematurely (Loose et al., 2010). Maintaining remote
workers’ health and productivity is therefore important for maintaining social
distancing as long as is necessary (Blake et al., 2010).
2.2 Working from home
erez et al. (2004) defines teleworking (also called remote working) as “or-
ganisation of work by using information and communication technologies that
enable employees and managers to access their labour activities from remote
locations, such as home-based teleworking, mobile teleworking, and telecenters
or teleworking centers” (p. 280). Telework can help restore and maintain oper-
ational capacity and essential services during and after disasters (Blake et al.,
Pandemic Programming 5
2010), especially when workplaces are inaccessible. Indeed, many executives
are already planning to shift “at least 5% of previously on-site employees to
permanently remote positions post-COVID 19” (Lavelle, 2020).
However, many organisations lack appropriate plans, supportive policies,
resources or management practices for practising home-based telework. In dis-
asters such as pandemics where public facilities are closed and people are
required to stay at home, their experience and capacity to work can be limited
by lack of dedicated workspace at home, caring responsibilities and organisa-
tional resources (Donnelly and Proctor-Thomson, 2015).
In general, working from home is often claimed to improve productivity
(Davenport and Pearlson, 1998; McInerney, 1999; Cascio, 2000) and telework-
ers consistently report increased perceived productivity (Duxbury et al., 1998;
Baruch, 2000). Interestingly, Baker et al. (2007) found that organisational and
job-related factors (e.g. management culture, human resources support, struc-
ture of feedback) are more likely to affect teleworking employees’ satisfaction
and perceived productivity than work styles (e.g. planning vs. improvising)
and household characteristics (e.g. number of children).
However, individuals’ wellbeing while working remotely is influenced by
their emotional stability (that is, a person’s ability to their control emotions
when stressed). Working from home gives people with high emotional stability
more autonomy and fosters their wellbeing. In contrast, working from home
can exacerbate physical, social and psychological strain in employees with low
emotional stability (Perry et al., 2018), and the COVID-19 pandemic has not
been good for emotional stability (ARI, 2020).
Research on working from home has been criticized for reliance on self-
reported perceived productivity, which may inflate the benefits of working
from home (Bailey and Kurland, 2002); however, objective measures often
lack construct validity (Ralph and Tempero, 2018) and perceived productivity
correlates well with managers’ appraisals (Baruch, 1996). (The perceived pro-
ductivity scale we use below correlates well with objective performance data;
cf. Section 4.3).
2.3 Productivity
Productivity is the amount of work done per unit of time.1Measuring time
is simple but quantifying the work done by a software developer is not. Some
researchers (e.g. Jaspan and Sadowski, 2019) argue for using goal-specific met-
rics; others reject the whole idea of measuring productivity (e.g. Ko, 2019)
not least because people tend to optimize for whatever metric is being used, a
phenomenon known as Goodhart’s law (Goodhart, 1984; Chrystal and Mizen,
2003).
Furthermore, simple productivity measures such as counting the number of
commits or the number of modified lines of code in a certain period suffer from
1https://dictionary.cambridge.org/dictionary/english/productivity
6 Paul Ralph et al.
low construct validity (Ralph and Tempero, 2018). The importance and added
value of a commit does not necessarily correlate with its size. Similarly, some
developers might prefer very dense one-line solutions, while others like to ar-
range their contributions in several lines. Comparing the two above-mentioned
solutions by counting lines of code must yield biased results. Nevertheless, large
companies including Microsoft still use controversial metrics such as number
of pull requests as a “proxy for productivity” (Spataro, 2020), and individual
developers also use them to monitor their own performance (Baltes and Diehl,
2018). Copious time tracking tools exist for that purpose—some specifically
tailored for software developers.2
While researchers have adapted existing scales to measure related phenom-
ena like happiness (e.g. Graziotin and Fagerholm, 2019), there is no widespread
consensus about how to measure developers’ productivity or the main an-
tecedents thereof. Many researchers use simple, unvalidated productivity scales;
for example, Meyer et al. (2017) used a single question asking participants to
rate themselves from “not productive” to “very productive.” (The perceived
productivity scale we use below has been repeatedly validated in multiple do-
mains; cf. see Section 4.3).
3 Hypotheses
The discussion above suggests numerous hypotheses, as follows. Here we hy-
pothesize about “developers” even though our survey was open to all soft-
ware professionals because most respondents were developers (see Section
5.3). These hypotheses were generated contemporaneously with questionnaire
design—before data collection began.
Hypothesis H1: Developers will have lower wellbeing while working from home
due to COVID-19. Stress, isolation, travel restrictions, business closures and
the absence of educational, child care and fitness facilities all take a toll on
those working from home. Indeed, a pandemic’s severity and the uncertainty
and isolation it induces create frustration, anxiety and fear (Taha et al., 2014;
DiGiovanni et al., 2004; Teasdale et al., 2012). It therefore seems likely that
many developers will be experiencing reduced emotional wellbeing.
Hypothesis H2: Developers will have lower perceived productivity while working
from home due to COVID-19. Similarly, stress, moving to an impromptu home
office, and lack of child care and other amenities may have a negative impact
on many developers’ productivity. Many people are likely more distracted by
the people they live with and their own worrisome thoughts. People tend
to experience lower motivation, productivity and commitment while working
from home in a disaster situation (Donnelly and Proctor-Thomson, 2015).
2See: https://wakatime.com/
Pandemic Programming 7
H3
Change in wellbeing
Home office ergonomics Change in perceived
productivity
Fear (of bioevent)
Disaster preparedness H4
H8
H6
H5
H9
H7
H10
Fig. 1 Theoretical model of developer wellbeing and productivity
Assuming Hypotheses H1 and H2 are supported, we want to propose a model
that explains and predicts changes in wellbeing and productivity (Figure 1).
Hypotheses H1 and H2 are encapsulated in the change in wellbeing and change
in perceived productivity constructs. The model only makes sense if wellbeing
and productivity have changed since developers began working from home.
Hypothesis H3: Change in wellbeing and change in perceived productivity are
directly related. We expect wellbeing and productivity to exhibit reciprocal
causality. That is, as we feel worse, we become less productive, and feeling less
productive makes us feel even worse, in a downward spiral. Many studies show
that productivity and wellbeign covary (cf. Dall’Ora et al., 2016). Moreover,
Evers et al. (2014) found that people with increasing health risks have a lower
wellbeing and higher dissatisfaction in life, leading to higher rates of depression
and anxiety. On the other hand, decreasing health risk will increase physical
and emotional wellbeing and productivity.
Hypotheses H4 and H5: Disaster preparedness is directly related to change in
wellbeing and change in perceived productivity. Disaster preparedness is the
degree to which a person is ready for a natural disaster. It includes behav-
iors like having an emergency supply kit and complying with directions from
authorities. We expect lack of preparedness for disasters in general and for
COVID-19 in particular to exacerbate reductions in wellbeing and productiv-
ity, and vice versa (cf. Paton, 2008; Donnelly and Proctor-Thomson, 2015).
Hypotheses H6 and H7: Fear (of the pandemic) is inversely related to change in
wellbeing and change in perceived productivity. Fear is a common reaction to
bioevents like pandemics. Emerging research on COVID-19 is already showing
a negative effect on wellbeing, particularly anxiety (Harper et al., 2020; Xiang
et al., 2020). Meanwhile, fear of infection and public health measures cause
psychosocial distress, increased absenteeism and reduced productivity (Shultz
et al., 2016; Thommes et al., 2016).
8 Paul Ralph et al.
Hypotheses H8 and H9: Home office ergonomics is directly related to change in
wellbeing and change in perceived productivity. Here we use ergonomics in its
broadest sense of the degree to which an environment is safe, comfortable and
conducive to the tasks being completed in it. We are not interested in measur-
ing the angle of a developer’s knees and elbows, but in a more general sense
of their comfort. Professionals with more ergonomic home offices should have
greater wellbeing and be more productive. Donnelly and Proctor-Thomson
(2015) found that availability of a dedicated work-space at home, living cir-
cumstances, and the availability of organisational resources to work relate to
the capacity to return to work after a disaster and employees’ productivity.
Hypothesis H10: Disaster preparedness is inversely related to fear (of the pan-
demic). It seems intuitive that the more prepared we are for a disaster, the
more resilient and less afraid we will be when the disaster occurs. Indeed, Ro-
nan et al.’s 2015 systematic review found that programs for increasing disaster
preparedness had a small- to medium-sized negative effect on fear. People who
have high self-efficacy and response-efficacy (i.e. perceive themselves as ready
to face a disaster) will be less afraid (Roberto et al., 2009).
4 Method
On March 19, 2020, the first author conceived of a survey to investigate how
COVID-19 affects developers, and recruited the second and third authors for
help. We created the questionnaire and it was approved by Dalhousie Univer-
sity’s research ethics board in less than 24 hours. We began data collection
on March 27th. We then recruited authors 5 through 17, who translated and
localized the questionnaire into Arabic, (Mandarin) Chinese, English, French,
Italian, Japanese, Korean, Persian, Portuguese, Spanish, Russian and Turk-
ish, and created region-specific advertising strategies. Translations launched
between April 5 and 7, and we completed data collection on April 14. Next,
we recruited the fourth author to assist with the data analysis, which was
completed on April 29th. The manuscript was prepared primarily by the first
four authors with edits from the rest of team.
This section details our approach and instrumentation.
4.1 Replication pack
A comprehensive replication package including our (anonymous) dataset, in-
struments and analysis scripts is stored in the Zonodo open data archive.3
3https://zenodo.org/record/3780082
Pandemic Programming 9
4.2 Population and inclusion criteria
This study’s target population is software developers anywhere in the world
who switched from working in an office to working from home because of
COVID-19. Of course, developers who had been working remotely before the
pandemic and developers who continued working in offices throughout the pan-
demic are also important, but this study is about the switch, and the questions
are designed for people who switched from on-site to at-home work.
In principle, the questionnaire was open to all sorts of software profes-
sionals, including designers, quality assurance specialists, product managers,
architects and business analysis, but we are mainly interested in developers,
our marketing focuses on software developers, and therefore most respondents
identify as developers (see Section 5.3).
4.3 Instrument design
We created an anonymous questionnaire survey. We did not use URL tracking
or tokens. We did not collect contact information.
Questions were organized into blocks corresponding to scale or question
type. The order of the items in each multi-item scale was randomized to mit-
igate primacy and recency effects. The order of blocks was not randomized
because our pilot study (Section 4.4) suggested that asking the questionnaire
was more clear when the questions that distinguish between before and after
the switch to home working came after those that did not.
The questionnaire used a filter question to exclude respondents who do not
meet the inclusion criteria. Respondents who had not switched from working
in an office to working from home because of COVID-19 simply skipped to the
end of the questionnaire. It also included not only traditional demographic
variables (e.g. age, gender, country, experience, education) but also how many
adults and children (under twelve) participants lived with, the extent to which
participants are staying home and whether they or any friends or family had
tested positive for COVID-19.
The questionnaire used validated scales as much as possible to improve
construct validity. A construct is a quantity that cannot be measured directly.
Fear, disaster preparedness, home office ergonomics, wellbeing and productiv-
ity are all constructs. In contrast, age, country, and number of children are
all directly measurable. Direct measurements are assumed to have inherent
validity, but latent variables have to be validated to ensure that they measure
the right properties(cf. Ralph and Tempero, 2018).
The exact question wording can be seen in our replication pack. This sec-
tion describes the scales and additional questions.
Emotional wellbeing (WHO-5). To assess emotional wellbeing, we used the
WHO’s five-item wellbeing index (WHO-5).4Each item is assessed on a six-
4https://www.psykiatri-regionh.dk/who- 5/Documents/WHO5_English.pdf
10 Paul Ralph et al.
point scale from “at no time” (0) to “all of the time” (5). The scale can be
calculated by summing the items or using factor analysis. The WHO-5 scale is
widely used, widely applicable, and has high sensitivity and construct validity
(Topp et al., 2015). Respondents self-assessed their wellbeing twice: once for
the four weeks prior to beginning to work from home, and then again for the
time they have been working from home.
Perceived Productivity (HPQ). To assess perceived productivity we used items
from the WHO’s Health at Work Performance Questionnaire (HPQ).5The
HPQ measures perceived productivity in two ways: (1) using an eight-item
summative scale, with multiple reversed indicators, that assesses overall and
relative performance; and (2) using 11-point general ratings of participants’
own performance and typical performance of similar workers. These scales are
amenable to factor analysis or summation. Of course, people tend to overesti-
mate their performance relative to their peers, but we are comparing partic-
ipants to their past selves not to each other. HPQ scores are closely related
to objective performance data in diverse fields (Kessler et al., 2003). Again,
respondents self-assessed their productivity for both the four weeks prior to
working from home, and for the time they have been working from home.
Disaster Preparedness (DP). To assess disaster preparedness, we adapted Yong
et al.’s (2017) individual disaster preparedness scale. Yong et al. developed
their five-item, five-point, Likert scale based on common, important behaviors
such as complying with government recommendations and having emergency
supplies. The scale was validated using a questionnaire survey of a “weighted
nationally representative sample” of 1084 Canadians. We adapted the items to
refer specifically to COVID-19. It can be computed by summing the responses
or using factor analysis.
Fear and Resilience (FR). The Bracha-Burkle Fear and Resilience (FR) check-
list is a triage tool for assessing a patient’s reaction to a bioevent (e.g. infec-
tious disease pandemics, bioterrorism). The FR checklist places the patient on
a scale from intense fear to hyper-resilience (Bracha and Burkle, 2006). We
dropped some of the more extreme items (e.g. “Right now are you experiencing
shortness of breath?”) because respondents are at home taking a survey, not
arriving in a hospital emergency room. The FR checklist is a weighted sum-
mative scale so it has to be computed manually using Bracha and Burkle’s
formula rather than using factor analysis. It has multiple reversed indicators.
Ergonomics. We could not find a reasonable scale for evaluating home office
ergonomics. There is comparatively less research on the ergonomics of home of-
fices (Inalhan and Ng, 2010) and ergonomic instruments tend to be too narrow
(e.g. evaluating hip angle). Based on our reading of the ergonomics literature,
5https://www.hcp.med.harvard.edu/hpq/info.php
Pandemic Programming 11
we made a simple six-item, six-point Likert scale concerning distractions, noise,
lighting, temperature, chair comfort and overall ergonomics.
Again, we evaluated the scale’s face and content validity using a pilot
study (see Section 4.4) and statistically evaluated convergent and discriminant
validity ex post in Section 5.2.
Organizational Support (OS). This study seeks to produce actionable advice
for software companies regarding how to support their employees. Conse-
quently, we need to elicit respondent’s beliefs about helpful actions. We could
not find any appropriate instrument, so the first author interviewed three ex-
perienced developers with experience in both co-located and distributed teams
as well as office work and working from home. Interviewees brainstormed ac-
tions companies could take to help, and we used open-coding (Salda˜na, 2015)
to organize their responses into five themes:
1. Equipment: providing equipment employees need in their home office (e.g.
a second monitor)
2. Reassurance: adopting a tone that removes doubt and fear (e.g. assuring
employees that lower productivity would be understood)
3. Connectedness: encouraging virtual socializing (e.g. through video chat)
4. Self-care: providing personal services not directly related to work (e.g. re-
sources for exercising or home-schooling children)
5. Technical infrastructure and practices: ensuring that remote infrastructure
(e.g. VPNs) and practices (e.g. code review) are in place.
We generated a list of 22 actions (four or five per theme) by synthesiz-
ing the ideas of interviewees with existing literature on working from home,
distributed development and software engineering more generally. For each
action, respondents indicate whether their employer is taking the action and
whether they think it is or would be helpful. Organizational support is not a
construct in our theory per se because we have insufficient a priori information
to produce a quantitative estimate, so we analyze these answers separately.
4.4 Pilot
We solicited feedback from twelve colleagues: six software engineering aca-
demics and six experienced software developers. Pilot participants made var-
ious comments on the questionnaire structure, directions and on the face
and content validity of the scales. Based on this feedback we made numer-
ous changes including clarifying directions, making the question order static,
moving the WHO-5 and HPQ scales closer to the end, dropping some prob-
lematic questions, splitting up an overloaded question, and adding some open
response questions. (Free-text answers are not analyzed in this paper; open
response questions were included mainly to inform future research; see Sec-
tion 6.3).
12 Paul Ralph et al.
4.5 Sampling, incentives and localization
We advertised our survey on social and conventional media, including Dev.to,
eveloppez.com, DNU.nl, eksisozluk, Facebook, Hacker News, Heise Online, InfoQ,
LinkedIn, Twitter, Reddit and WeChat. Upon completion, participants were pro-
vided a link and encouraged to share it with colleagues who might also like
to take the survey. Because this is an anonymous survey, we did not ask re-
spondents to provide colleagues’ email addresses or send messages on their
behalf.
We did not offer cash incentives for participation. Rather, we offered to
donate US$500 to an open source project chosen by participants (in one of the
open response questions).
We considered several alternatives, including scraping emails from software
repositories and stratified random sampling using company lists, but none of
these options seemed likely to produce a more representative sample.
Instead, we focused on increasing the diversity of the sample by localizing
the survey and promoting it in more jurisdictions. We translated the survey
into Arabic, (Mandarin) Chinese, English, French, Italian, Japanese, Korean,
Persian, Portuguese, Spanish, Russian and Turkish. We capitalized on each
authors’ local knowledge to reach the more people in their jurisdiction. Rather
than a single, global campaign, we used a collection of local campaigns.
Each localization involved small changes in wording. Only a few significant
changes were needed:
The Chinese version had to be re-implemented in a different questionnaire
system (wjx.cn) because Google Forms is not available in China.
Because the lockdowns in China and Korea are ending, we reworded some
questions from “since you began working from home” to “while you were
working from home.”
The Portuguese version promised to donate to a specific open-source project
in Brazil that is related to COVID-19 (which we have done).
5 Analysis and Results
We received 2668 total responses of which 439 did not meet our inclusion
criteria and 4 were effectively blank (see below) leaving 2225 responses that
fulfilled our inclusion criteria. This section describes how the data was cleaned
and analyzed.
5.1 Data cleaning
The data was cleaned as follows.
1. Delete responses that did not meet our inclusion criteria.
2. Delete almost empty rows, where the respondent apparently answered the
filter question correctly, then skipped all other questions.
Pandemic Programming 13
3. Delete the timestamp field (to preserve anonymity), the consent form con-
firmation field (because participants could not continue without checking
these boxes so the answer is always “TRUE”) and the filter question field
(because all remaining rows have the same answer).
4. Add a binary field indicating whether the respondent had entered text in
at least one long-answer question (see Section 5.2)
5. Remove all free-text responses to a separate file (to preserve anonymity).
6. Recode the raw data (which is in different languages with different alpha-
bets) into a common quantitative coding scheme; for example, from 1 for
“strongly disagree” to 5 for “strongly agree” The recoding instructions and
related scripts are included in our replication package.
7. Split select-multiple questions into one binary variable per checkbox. (Google
forms inanely saves this data as a comma-separated list of the text of se-
lected answers).
8. Add a field indicating the language of the response.
9. Combine the responses into a single data file.
10. Calculate the FR scale according to its formula (Bracha and Burkle, 2006).
5.2 Validity analysis
We evaluated construct validity using established guidelines (Ralph and Tem-
pero, 2018). First, we assessed content validity using a pilot study (Section 4.4).
Next, we assessed convergent and discriminant validity using a principle com-
ponent analysis (PCA) with Varimax rotation and Kaiser normalization. Bartlett’s
test is significant (chi square = 13229; df = 253; p < 0.001) and our KMO
measure of sampling adequacy is high (0.874), indicating that our data is ap-
propriate for factor analysis.
As Table 1 shows, the items load well but not perfectly. The bolded coef-
ficients suggest possible issues with Change in productivity (Productivity)
7 and 9, as well as Ergonomics 1. We dropped items one at a time until the
loadings stabilize, starting with Productivity 7, followed by Productivity
9. As shown in Table 2, dropping these two indicators solved the problem with
Ergonomics 1, so the latter is retained.
We evaluate predictive validity by testing our hypotheses in Section 5.4.
Response bias. There are two basic ways to analyze response bias. The first—
comparing sample parameters to known population parameters—is imprac-
tical because no one has ever established population parameters for software
professionals. The second—comparing late respondents to early respondents—
cannot be used directly here because we do not know the time between each
respondent learning of the survey and completing it. However, we can do some-
thing similar: we can compare respondents who answered one or more open
response questions (more keen on the survey) with those who skipped the open
response questions (less keen on the survey).
14 Paul Ralph et al.
Table 1 First Principle Components Analysis*
Variable Component
1 2 3 4
Productivity 8 0.740
Productivity 2 0.715
Productivity 9 0.704 0.304
Productivity 6 0.699
Productivity 4 0.669
Productivity 3 0.645
Productivity 5 0.64
Productivity 1 0.563
Productivity 7 0.356
Wellbeing 1 0.838
Wellbeing 2 0.791
Wellbeing 3 0.782
Wellbeing 5 0.734
Wellbeing 4 0.727
Ergonomics 6 0.802
Ergonomics 5 0.748
Ergonomics 2 0.666
Ergonomics 3 0.645
Ergonomics 1 0.306 0.640
Ergonomics 4 0.628
Disaster Preparedness 3 0.688
Disaster Preparedness 1 0.661
Disaster Preparedness 5 0.568
Disaster Preparedness 2 0.565
Disaster Preparedness 4 0.493
*Rotation converged in 5 iterations. Coefficients <0.3 suppressed.
As shown in Table 3, only number of adult cohabitants and age are signif-
icant, and in both cases, the effect size (η2) is very small. This is consistent
with minimal response bias.
5.3 Demographics
Respondents were disproportionately male (81% vs. 18% female and 1% non-
binary) and overwhelmingly employed full-time (94%) with a median age of
30–34. Participants were generally well-educated (Fig. 3). Most respondents,
53%, live with one other adult, while 18% live with no other adults and the
rest live with two or more people. 27% live with one or more children under
12. 8% indicate that they may have a disability that affects their work. Mean
work experience is 9.3 years (σ= 7.3). Mean experience working from home is
1.3 years (σ= 2.5); however, 58% of respondents have no experience working
from home.
Participants hail from 53 countries (Table 4) and organizations ranging
from 0–9 employees to more than 100,000 (Fig 2). Many participants listed
multiple roles but 80% included software developer or equivalent among them,
Pandemic Programming 15
Table 2 Second Principle Components Analysis
Variable Component
1234
productivity 2 0.721
productivity 8 0.718
productivity 6 0.703
productivity 4 0.679
productivity 3 0.651
productivity 5 0.649
productivity 1 0.566
wellbeing 1 0.845
wellbeing 2 0.797
wellbeing 3 0.790
wellbeing 5 0.740
wellbeing 4 0.732
ergonomics6 0.803
ergonomics5 0.745
ergonomics2 0.669
ergonomics1 0.646
ergonomics3 0.644
ergonomics4 0.629
disasterPreparedness3 0.685
disasterPreparedness1 0.666
disasterPreparedness2 0.570
disasterPreparedness5 0.565
disasterPreparedness4 0.490
Notes: Rotation convergonomicsed in 5 iterations; correlations <0.3suppressed.
Table 3 Analysis of Response Bias (One-way ANOVA)
Variable F Sig. η2
age 4.250 0.039 0.002
disability 0.117 0.733 0.000
education 0.153 0.696 0.000
adult cohabitants 19.037 0.000 0.009
child cohabitants 0.358 0.550 0.000
experience 3.381 0.066 0.002
remote experience 0.013 0.910 0.000
organization size 0.330 0.566 0.000
while the rest were other kinds of software professionals (e.g. project manager,
quality assurance analyst).
Seven participants (<1%) tested positive for COVID-19 and six more
(<1%) live with someone with COVID-19; 4% of respondents indicated that
a close friend or family member had tested positive, and 13% were currently
or recently quarantined.
16 Paul Ralph et al.
0-9 10-99 100-999 1,000-9,999 10,000-99,999 100,000
0100 200 300 400 500 600 700
Number of responses
Fig. 2 Organization sizes
No post-sec. Some post-sec. Undergraduate Masters PhD
0200 400 600 800 1000
Number of responses
Fig. 3 Participants’ education levels
Table 4 Respondents’ countries of residence
Country n % Country n %
Germany 505 22.7% Japan 53 2.4%
Russia 366 16.4% Spain 52 2.3%
Brazil 272 12.2% Iran 40 1.8%
Italy 173 7.8% Austria 29 1.3%
United States 99 4.4% Canada 27 1.2%
South Korea 81 3.6% Switzerland 20 0.9%
Belgium 77 3.5% United Kingdom 20 0.9%
China 76 3.4% n/a 20 0.9%
Turkey 66 3.0% Other 194 8.7%
India 55 2.5%
Pandemic Programming 17
5.4 Change in wellbeing and productivity
Hypothesis H1: Developers will have lower wellbeing while working from home
due to COVID-19. The WHO5 wellbeing scale is a five-item summative scale.
Participants answered the scale twice—once referring to the 28-day period
before switching to work from home and once referring to the period while
working from home. To assess the effect of switching to work from home,
we sum each scale and compare their distributions. Since the summed scales
deviate significantly from a normal distribution (K-S test; p < 0.001), we
compare the distributions using the Wilcoxon signed rank test.
Hypothesis H1 is supported (Wilcoxon signed rank test Z= 9.107; p <
0.001). Effect size for the Wilcoxon signed rank test is calculated as z/n
where nis the total number of observations over both data points, which gives
an effect size of 0.137.
Hypothesis H2: Developers will have lower perceived productivity while working
from home due to COVID-19. Like the wellbeing scale, participants answered
the HPQ productivity scale twice. We sum the scales (omitting items 7 and
9; see Section 5.2) and compare their distributions. Again, the summed scales
deviate significantly from a normal distribution (K-S test; p < 0.001), so we
use the Wilcoxon signed rank test.
Hypothesis H2 is supported (Wilcoxon signed rank test Z= 10.614; p <
0.001; effect size= 0.164).
5.5 Structural equation model
To test our remaining hypotheses, we use structural equation modeling (SEM).
Briefly, SEM is used to test theories involving constructs (also called latent
variables). A construct is a quantity that cannot be measured directly (Ralph
and Tempero, 2018). Fear, disaster preparedness, home office ergonomics, well-
being and productivity are all constructs. In contrast, age, country, and num-
ber of children are all directly measurable.
To design a structural equation model, we first define a measurement model,
which maps each reflective indicator into its corresponding construct. For ex-
ample, each of the five items comprising the WHO5 wellbeing scale is modeled
as a reflective indicator of wellbeing. SEM uses confirmatory factor analysis
to estimate each construct as the shared variance of its respective indicators.
Next, we define the structural model, which identifies the expected rela-
tionships among the constructs. The constructs we are attempting to predict
are referred to as endogenous, while the predictors are exogenous.
SEM uses a path modeling technique (e.g. regression) to build a model that
predicts the endogenous (latent) variables based on the exogenous variables,
18 Paul Ralph et al.
Table 5 Confirmatory Factor Analysis
Construct Indicator Estimate Std.Err z-value P(>|z|)
Wellbeing =WB1 1.000
WB2 0.896 0.016 54.518 0
WB3 0.955 0.016 58.917 0
WB4 0.804 0.018 44.686 0
WB5 0.848 0.017 51.041 0
Productivity =P1 1.000
P2 -1.268 0.053 -24.084 0
P3 -1.120 0.053 -20.979 0
P4 -1.239 0.053 -23.263 0
P5 -1.229 0.055 -22.266 0
P6 -1.306 0.058 -22.677 0
P8 1.460 0.057 25.512 0
Ergonomics =Erg1 1.000
Erg2 0.964 0.035 27.395 0
Erg3 0.820 0.037 22.128 0
Erg4 0.937 0.035 26.663 0
Erg5 1.064 0.034 31.535 0
Erg6 1.258 0.035 35.821 0
Disaster DP1 1.000
Preparedness =DP2 0.716 0.089 8.079 0
DP3 1.181 0.112 10.521 0
DP4 0.923 0.105 8.805 0
DP5 1.186 0.120 9.888 0
Notes: converged after 50 iterations with 185 free parameters with (n= 1377); estimates
may exceed 1.0 because they are regression coefficients, not correlations as in Principal
Component Analysis; negative estimates indicate reversed items
and to estimate both the strength of each relationship and the overall accuracy
of the model.6
As mentioned, the first step in a SEM analysis is to conduct a confirma-
tory factor analysis to verify that the measurement model is consistent (Table
5). Here, the latent concepts Ergonomics and DisasterPreparedness are cap-
tured by their respective exogenous variables. Fear is not included because it is
computed manually (see Section 4.3). Wellbeing is the difference in a partic-
ipant’s emotional wellbeing before and after the corona outbreak. This latent
concept is captured by five exogenous variables, DeltaW1,...,DeltaW5. Simi-
larly, Productivity represents the difference in perceived productivity, before
and after the corona outbreak.
The confirmatory factor analysis converged (not converging would suggest
a problem with the measurement model) and all of the indicators load well on
their constructs. The lowest estimate, 0.716 for DP2, is still quite good. The
estimates for DeltaP2 through 6 are negative because these items were reversed
6Data was analyzed using the Rpackage lavaan 0.6-5. available at http://lavaan.ugent.
be/.
Pandemic Programming 19
(i.e. higher score = worse productivity). Note that factor loadings greater than
one do not indicate a problem because they are regression coefficients, not
correlations (J¨oreskog, 1999).
Having reached confidence in our measurement model, we construct our
structural model by representing all of the hypotheses stated in Section 3 as
regressions (e.g. Wellbeing DisasterPreparedness + Fear + Ergonomics).
In principle, we use all control variables as predictors for all latent variables.
In practice, however, this leads to too many relationships and prevents the
model from converging. Therefore, we evaluate the predictive power of each
control variable, one at a time, and include it in a regression only where it
makes at least a marginally significant (p < 0.1) difference. Here, using a
higher than normal p-value is more conservative because we are dropping
predictors rather than testing hypotheses. Country (of residence) and language
(of questionnaire) are not included because SEM does not respond well to
nominal categorical variables (see Section 5.6).
Since the exogenous variables are ordinal, the weighted least square mean
variance (WLSMV) estimator was used. WLSMV uses diagonally weighted
least squares to estimate the model parameters, but it will use the full weight
matrix to compute robust standard errors, and a mean- and variance-adjusted
test statistic. In short, the WLSMV is a robust estimator which does not
assume a normal distribution, and provides the best option for modelling or-
dinal data in SEM (Brown, 2006). We use the default Nonlinear Minimization
subject to Box Constraints (NLMINB) optimizer.
For missing data, we use pairwise deletion: we only keep those observations
for which both values are observed (this may change from pair to pair). By
default, since we are also dealing with categorical exogenous variables, the
model is set to be conditional on the exogenous variables.
The model was executed and all diagnostics passed, that is, lavaan ended
normally after 97 iterations with 212 free parameters with n= 1377. We
evaluate model fit by inspecting several indicators (cf. Hu and Bentler, 1999,
for cut-offs):
The ratio of the χ2-test statistic and degrees of freedom (2302/212 = 10.8)
should be greater than two.
The Comparative Fit Index (CFI = 0.961) and Tucker-Lewis Index (TLI =
0.979), which compare the model’s fit against the null model, should be at
least 0.95.
The Root Mean Square Error of Approximation (RMSEA = 0.051, 90%
CI [0.048; 0.053]) should be less than 0.06.
The Standardized Root Mean Square Residual (SRMR = 0.067) should be
less than 0.08 (for large sample sizes).
In summary, all diagnostics indicate the model is safe to interpret (i.e.
χ2
fit = 10.8, CFI = 0.961, RMSEA = 0.051, SRMR = 0.067).
Figure 4 illustrates the supported structural equation model. Numbers are
path coefficients, which indicate the relative strength and direction of relation-
ships. Arrows indicate hypothesized causal direction.
20 Paul Ralph et al.
Change in
wellbeing
Home office
ergonomics
Change in
perceived
productivity
Fear (of
bioevent)
Disaster
preparedness
0.097
0.125
-0.031
0.242
-0.336
ΔWB1 ΔWB2 ΔWB3 ΔWB4 ΔWB5
ΔP1 ΔP2 ΔP3 ΔP4 ΔP5 ΔP6 ΔP8
1.0
.896
.955
.804
.848
1.0
-1.268
-1.120
-1.239
-1.229
-1.306
1.460
DP1
DP2
DP3
DP4
DP5
1.0
.716
1.181
.923
1.186
Erg1 Erg2 Erg3 Erg4 Erg5
1.0
0.964
0.820
0.937
1.064
Erg6
1.258
0.822
Fig. 4 Supported model of developer wellbeing and productivity
Note: error terms, unsupported hypotheses and control variables are omitted for clarity
Based on this model, Hypotheses H1–H3, H5, H6, H8–H10 are sup-
ported; Hypotheses H4 and H7 are not supported. That is, change
in wellbeing and change in perceived productivity are directly related;
change in perceived productivity depends on home office ergonomics and
disaster preparedness; change in wellbeing depends on ergonomics and fear;
and disaster preparedness is inversely related to fear.
5.6 Exploratory findings
Inspecting the detailed SEM results (Table 6) reveals many interesting pat-
terns.
Many direct effects are obvious, for example:
People who live with small children have significantly less ergonomic home
offices. This is not surprising because the ergonomics scale included items
related to noise and distractions.
Women tend to be more afraid. This is consistent with studies on the SARS
epidemic, which found that women tended to perceive the risk as higher
(Brug et al., 2004).
People with disabilities are less prepared for disasters, have less ergonomic
offices and are more afraid.
Pandemic Programming 21
Table 6 SEM Regressions
Construct Predictor Estimate Std.Err z-value P(>|z|)
Disaster adultCohabitants 0.080 0.019 4.234 0.000
Preparedness disability -0.179 0.059 -3.035 0.002
covidStatus 0.073 0.032 2.260 0.024
education -0.050 0.026 -1.882 0.060
Ergonomics children -0.163 0.031 -5.184 0.000
adultCohabitants -0.047 0.019 -2.457 0.014
disability -0.110 0.057 -1.932 0.053
remoteExperience 0.044 0.026 1.709 0.087
Fear isolation 0.502 0.105 4.764 0.000
DisasterPreparedness -0.336 0.106 -3.161 0.002
role -0.356 0.116 -3.056 0.002
covidStatus 0.196 0.075 2.607 0.009
gender 0.273 0.122 2.241 0.025
disability 0.265 0.119 2.227 0.026
education -0.122 0.060 -2.047 0.041
children 0.116 0.063 1.831 0.067
Wellbeing Ergonomics 0.125 0.033 3.813 0.000
covidStatus -0.121 0.040 -3.041 0.002
Fear -0.031 0.012 -2.542 0.011
age 0.097 0.044 2.204 0.028
DisasterPreparedness -0.020 0.049 -0.416 0.678
Productivity Ergonomics 0.242 0.024 10.233 0.000
DisasterPreparedness 0.097 0.035 2.788 0.005
adultCohabitants 0.041 0.015 2.752 0.006
disability 0.124 0.049 2.513 0.012
age 0.070 0.032 2.220 0.026
Fear -0.002 0.009 -0.204 0.838
Wellbeing Performance 0.822 0.045 18.361 0.000
Notes: converged after 97 iterations; Latent variables capitalized (e.g. Fear); direct mea-
surements in camelCase (e.g. age, adultCohabitants)
People who live with other adults are more prepared for disasters.
People who live alone have more ergonomic home offices.
People who have COVID-19 or have family members, housemates or close
friends with COVID-19 tend to be more afraid, more prepared, and have
worse wellbeing since working from home.
People who are more isolated (i.e. not leaving home at all, or only for
necessities) tend to be more afraid.
It is more difficult to interpret indirect effects. For example, changes in
productivity and wellbeing are closely related. Hypothesis H4 may be unsup-
ported because change in productivity may be mediating the effect of disaster
preparedness on change in wellbeing. Similarly, Hypothesis H7 may not be un-
supported because change in wellbeing is mediating the relationship between
fear and change in productivity. Furthermore, control variables including gen-
22 Paul Ralph et al.
der, children and disability may have significant effects on wellbeing or produc-
tivity that are not obvious because they are mediated by another construct.
Moreover, some variables have conflicting effects. For example, disability has
not only a direct positive effect on productivity but also an indirect nega-
tive effect (through fear). So, is the pandemic hitting people with disabilities
harder? More research is needed to explore these relationships.
Above, we mentioned omitting language and country because SEM does
not respond well to nominal categorical variables. We tried anyway, and both
language and country were significant predictors for all latent variables, but,
as expected, including so many binary dummy variables makes the model im-
possible to interpret. While our analysis suggests that country, language (and
probably culture) have significant effects on disaster preparedness, ergonomics,
fear, wellbeing and productivity, more research is need to understand the na-
ture of these effects (see Section 6.3).
5.7 Organizational support
Table 7 shows participants’ opinions of the helpfulness of numerous ways their
organizations could support them. Several interesting patterns stand out from
this data:
Only action #1—paying developer’s home internet charges—is perceived
as helpful by more than half of participants and less than 10% of companies
appear to be doing that.
The action most companies are taking (#20, having regular meetings) is
not perceived as helpful by most participants.
There appears to be no correlation between things developers believe would
help and things employers are actually doing.
There is little consensus among developers about what their organizations
should do to help them.
5.8 Summary interpretation
This study shows that software professionals who are working from home dur-
ing the pandemic are experiencing diminished emotional wellbeing and pro-
ductivity, which are closely related. Furthermore, poor disaster preparedness,
fear related to the pandemic, and poor home office ergonomics are exacerbat-
ing this reduction in wellbeing and productivity. Moreover, women, parents
and people with disabilities may be disproportionately affected. In addition,
dissensus regarding what organizations can do to help suggests that no single
action is universally helpful; rather, different people need different kinds of
support.
Pandemic Programming 23
Table 7 Organizational support actions in order of perceived helpfulness*
# Action Helpful Following
1 My organization will pay for some or all of my internet
charges
51.9% 9.8%
2 My organization will buy new equipment we need to work
from home
49.2% 30.9%
3 My organization is encouraging staff to use this time for
professional training
47.7% 24.3%
4 My organization has reassured me that they understand if
my work performance suffers
47.4% 40.5%
5 My organization is providing activities to occupy staff mem-
ber’s children
46.4% 7.2%
6 My organization is sending food to staff working from home 44.5% 4.0%
7 My organization is providing at-home exercise programs 41.4% 15.8%
8 My organization has reassured me that I will keep my job 40.2% 62.4%
9 My organization has reassured me that I can take time off
if I’m sick or need to care for dependents
40.1% 65.5%
10 My organization is improving documentation of its pro-
cesses (e.g. how code changes are approved)
37.4% 34.7%
11 My organization will pay for software we need to work from
home
36.8% 54.7%
12 My team is peer reviewing commits, change requests or pull
requests (peer code review)
36.5% 63.1%
13 I can (or could) take equipment (e.g. monitors) home from
my workplace
36.0% 73.9%
14 My organization has reassured me that I will continue to be
paid
34.7% 75.2%
15 My team uses a build system to automate compilation and
testing
34.3% 62.9%
16 Someone is keeping high priority work ready and our back-
log organized
33.1% 60.0%
17 My team has good work-from-home infrastructure (e.g.
source control, VPN, remote desktop, file sharing)
32.6% 86.4%
18 My team is having virtual social events (e.g. via video chat) 32.1% 56.1%
19 My organization is encouraging staff to touch base regularly
with each other
30.8% 62.4%
20 My team is continuing to have regular meetings (e.g. via
video chat)
28.5% 88.9%
21 My team is avoiding synchronous communication (e.g. video
chat)
25.5% 14.3%
22 For most of the day, I work with an open video or audio call
to some or all of my team
23.3% 26.7%
*number of respondents who indicated that this practice is or would be helpful and number
of respondents who indicated that their organizations are following this recommendation
(n=2225)
6 Discussion
6.1 Recommendations
Organizations need to accept that expecting normal productivity under these
circumstances is unrealistic. Pressuring employees to maintain normal produc-
tivity will likely make matters worse. Furthermore, companies should avoid
24 Paul Ralph et al.
making any decisions (e.g. layoffs, promotions, bonuses) based on productiv-
ity during the pandemic because any such decision may be prejudiced against
protected groups.
The best way to improve productivity is to help employees maintain their
emotional wellbeing. However, no single action appears beneficial to everyone,
so organizations should talk to their employees to determine what they need.
Helping employees improve the ergonomics of their work spaces, in partic-
ular, should help. However, micromanaging foot positions, armrests and hip
angles is not what we mean by ergonomics. Rather, companies should ask
broad questions such as “what do you need to limit distractions and be more
comfortable?” Shipping an employee a new office chair or noise cancelling
headphones could help significantly.
Meanwhile, professionals should try to accept that their productivity may
be lower and stop stressing about it. Similarly, professionals should try to
remember that different people are experiencing the pandemic in very different
ways—some people may be more productive than normal while others struggle
to complete any work through no fault of their own. It is crucial to support
each other and avoid inciting conflict over who is working harder. If a member
of a protected group feels discriminated against due to low productivity at
this time, we recommend contacting your local human rights commission or
equivalent organization.
6.2 Limitations and threats to validity
The above recommendations should be considered in the context of the study’s
limitations.
Sampling bias. Random sampling of software developers is rare (Amir and
Ralph, 2018) because there are no lists of all the software developers, projects,
teams or organizations in the world or particular jurisdictions (Baltes and
Ralph, 2020). We therefore combined convenience and snowball sampling with
a strategy of finding a co-author with local knowledge to translate, localize and
advertise the questionnaire in a locally effective way. On one hand, the conve-
nience/snowball strategy may bias the sample in unknown ways. On the other
hand, our translation and localization strategy demonstrably increased sample
diversity, leading to one of the largest and broadest samples of developers ever
studied, possible due to a large, international and diverse research team. Any
random sample of English-speaking developers is comparatively ethnocentric.
Response Bias. Meanwhile, we found minimal evidence of response bias (in
Section 5.2), however, because the questionnaire is anonymous and Google
Forms does not record incomplete responses, response bias can only be esti-
mated in a limited way. Someone could have taken the survey more than once
or entered fake data. Moreover, some internet service providers in Iran block
Google services, but developers tend to use proxies to bypass these restrictions.
Pandemic Programming 25
Additionally, large responses from within a single country could skew the data
but we correct for country, company size and language to mitigate this.
Construct validity. To enhance construct validity, we used validated scales
for wellbeing, productivity, disaster preparedness and fear/resilience. Post-
hoc construct validity analysis suggests that all four scales, as well as the
ergonomics scale we created, are sound (Section 5.2). However, perceived pro-
ductivity is not the same as actual productivity. Although the HPQ scale
correlates well with objective performance data in other fields (Kessler et al.,
2003), it may not in software development. Similarly, we asked respondents
their opinion of numerous potential mechanisms for organizational support.
Just because companies are taking some action (e.g. having regular meetings)
or respondents believe in the helpfulness of some action (e.g. paying their
internet bills), does not mean that those actions will lead to measurable im-
provements in productivity or wellbeing.
Measurement Validity. There is much debate about whether 5-point responses
such as used in the WHO5 scale should be treated as ordinal or interval.
CFA and SEM are often used with these kinds of variables in social sciences
despite assuming at least interval data. Some evidence suggests that CFA is
robust against moderate deviations from normality, including arguably-ordinal
questionnaire items (Flora and Curran, 2004, cf.). We tend not to worry about
treating data as interval as long as, in principle, the data is drawn from a
continuous distribution. Additionally, due to a manual error, the Italian version
was missing organizational support item 11: “My team uses a build system to
automate compilation and testing.” The survey may therefore under-count the
frequency and importance of this item by up to 10%.
Conclusion validity. We use structural equation modeling to fit a theoretical
model to the data. Indicators of model fit suggest that the model is sound.
Moreover, SEM enhances conclusion validity by correcting for multiple com-
parisons, testing the entire model as a whole (instead of one hypothesis at a
time) and measurement error (by inferring latent variables based on observable
variables). SEM is considered superior to alternative path modeling techniques
such as partial least squares path modeling (R¨onkk¨o and Evermann, 2013).
While a Bayesian approach might have higher conclusion validity (Furia et al.,
2019), none of the Bayesian SEM tools (e.g. Blaavan) we are aware of support
ordered categorical variables. The main remaining threat to conclusion valid-
ity is that the structural model is overfit to the data. More research is needed
to determine whether the model overstates any of the supported effects.
Internal validity. To infer causality, we must demonstrate correlation, prece-
dence and the absence of third variable explanations. SEM demonstrates cor-
relation. Inferring causal relationships from cross-sectional data is fraught be-
cause we cannot tell the direction of causality or control for all possible con-
founds. However, many of our propositions only make sense in one direction.
26 Paul Ralph et al.
For example, having COVID-19 may reduce one’s productivity, but feeling
unproductive cannot give someone a specific virus. Other relationships make
a lot more sense in one direction than the other. For example, having a more
ergonomic office might make you more productive, but being more produc-
tive does not make your office more ergonomic. We can be more confident
in causality where precedence only makes sense in one direction. That said,
while we include numerous control variables (e.g. age, gender, education level),
other third variable explanations cannot be discounted. Developers who work
more overtime, for example, might have lower wellbeing, worse home office
ergonomics, and reduced disaster preparedness.
6.3 Implications for researchers and future work
For researchers, this paper opens a new research area intersecting software
engineering and crisis, disaster and emergency management. Although many
studies explore remote work and distributed teams, we still need a better un-
derstanding of how stress, distraction and family commitments affect develop-
ers working from home during crises, bioevents and disasters. More research is
needed on how these events affect team dynamics, cohesion and performance.
More specifically, the dataset we publish alongside this paper can be sig-
nificantly extended. An enormous amount of quantitative data is available
regarding different countries, and how those countries reacted to the COVID-
19 pandemic. Country data could be merged with our dataset to investigate
how different contexts, cultures and political actions affect developers.
Moreover, the crisis continues. More longitudinal research is needed to
understand its long-term effects on software professionals, projects and com-
munities.
6.4 Lessons learned
This study taught us two valuable lessons about research methodology. First,
collaborating with a large, diverse, international research team and releasing
a questionnaire in multiple languages with location-specific advertising can
generate a large, diverse, international sample of participants.
Second, Google Forms should not be used to conduct scientific question-
naire surveys. It is blocked in some countries. It does not record partial re-
sponses or bounce rates, hindering analysis of response bias. URL parameter
passing, which is typically used to determine how the respondent found out
about the survey, is difficult. Exporting the data in different ways gives differ-
ent variable orders, encouraging mistakes. Responses are recorded as (some-
times long) strings instead of numbers, overcomplicating data analysis. We
should have used a research focused survey tool such as Qualtrics7.
7qualtrics.com
Pandemic Programming 27
7 Conclusion
The COVID-19 pandemic has created unique conditions for many software
developers. Stress, isolation, travel restrictions, business closures and the ab-
sence of educational, child care and fitness facilities are all taking a toll. Work-
ing from home under these conditions is fundamentally different from normal
working from home. This paper reports the first large-scale study of how work-
ing from home during a pandemic affects software developers. It makes several
key contributions:
Evidence that productivity and wellbeing are suffering;
Evidence that productivity and wellbeing are closely related;
A model that explains and predicts the effects of the pandemic on produc-
tivity and wellbeing;
A ranked list of suggestions for supporting developers working from home;
Some indication that the pandemic may disproportionately affect women,
parents and people with disabilities.
Furthermore, this study is exceptional in several ways: (1) the questionnaire
used previously validated scales, which we re-validated using both principal
components analysis and confirmatory factor analysis; (2) the questionnaire
attracted an unusually large sample of 2225 responses; (3) the questionnaire
ran in 12 languages, mitigating cultural biases; (4) the data was analyzed us-
ing highly sophisticated methods (i.e. structural equation modelling), which
rarely have been utilized in software engineering research; (5) the study investi-
gates an emerging phenomenon, providing timely advice for organizations and
professionals; (6) the study incorporates research on emergency and disaster
management, which is rarely considered in software engineering studies.
We hope that this study inspires more research on how software develop-
ment is affected by crises, pandemics, lockdowns and other adverse conditions.
As the climate crisis unfolds, research intersecting crisis management and soft-
ware engineering will be increasingly needed.
Acknowledgements This project was supported by Dalhousie University. Thanks to Brett
Cannon, Alexander Serebrenik, Klaas Stol and all of our pilot participants for their sup-
port. Thanks also to all of media outlets who provided complementary advertising, including
DNU.nl, eksisozluk, InfoQ and Heise Online. Finaly, thanks to everyone at Empirical Soft-
ware Engineering for fast-tracking COVID-related research.
References
Amir B, Ralph P (2018) There is no random sampling in software engineering
research. In: Proceedings of the 40th International Conference on Software
Engineering: Companion Proceeedings, pp 344–345
Anderson RM, Heesterbeek H, Klinkenberg D, Hollingsworth TD (2020) How
will country-based mitigation measures influence the course of the covid-19
epidemic? The Lancet 395(10228):931–934
28 Paul Ralph et al.
ARI (2020) Worry, gratitude & boredom: As covid-19 affects mental, finan-
cial health, who fares better; who is worse? URL http://angusreid.org/
covid19-mental-health/
Bailey DE, Kurland NB (2002) A review of telework research: Findings, new
directions, and lessons for the study of modern work. Journal of Organiza-
tional Behavior: The International Journal of Industrial, Occupational and
Organizational Psychology and Behavior 23(4):383–400
Baker E, Avery GC, Crawford J (2007) Satisfaction and perceived produc-
tivity when professionals work from home. Research & Practice in Human
Resource Management
Baltes S, Diehl S (2018) Towards a theory of software development expertise.
In: Leavens GT, Garcia A, Pasareanu CS (eds) Proceedings of the 2018
ACM Joint Meeting on European Software Engineering Conference and
Symposium on the Foundations of Software Engineering, ESEC/SIGSOFT
FSE 2018, Lake Buena Vista, FL, USA, November 04-09, 2018, ACM, pp
187–200, DOI 10.1145/3236024.3236061, URL https://doi.org/10.1145/
3236024.3236061
Baltes S, Ralph P (2020) Sampling in software engineering research: A critical
review and guidelines. arXiv 2002.07764
Baruch Y (1996) Self performance appraisal vs direct-manager appraisal: A
case of congruence. Journal of Managerial Psychology 11(6):50–65, DOI
10.1108/02683949610129758
Baruch Y (2000) Teleworking: benefits and pitfalls as perceived by profession-
als and managers. New technology, work and employment 15(1):34–49
Blake KD, Blendon RJ, Viswanath K (2010) Employment and compliance
with pandemic influenza mitigation recommendations. Emerging infectious
diseases 16(2):212
Bracha HS, Burkle FM (2006) Utility of fear severity and individual resilience
scoring as a surge capacity, triage management tool during large-scale, bio-
event disasters. Prehospital and disaster medicine 21(5):290–296
Brown TA (2006) Confirmatory factor analysis for applied research. Method-
ology in the social sciences, Guilford Publications
Brug J, Aro AR, Oenema A, De Zwart O, Richardus JH, Bishop GD (2004)
Sars risk perception, knowledge, precautions, and information sources, the
netherlands. Emerging infectious diseases 10(8):1486
Cascio WF (2000) Managing a virtual workplace. Academy of Management
Perspectives 14(3):81–90
Chrystal KA, Mizen PD (2003) Goodhart’s law: Its origins, meaning and impli-
cations for monetary policy. Central banking, monetary theory and practice:
Essays in honour of Charles Goodhart 1:221–243
Dall’Ora C, Ball J, Recio-Saucedo A, Griffiths P (2016) Characteristics of shift
work and their impact on employee performance and wellbeing: A literature
review. International journal of nursing studies 57:12–27
Davenport TH, Pearlson K (1998) Two cheers for the virtual office. MIT Sloan
Management Review 39(4):51
Pandemic Programming 29
DiGiovanni C, Conley J, Chiu D, Zaborski J (2004) Factors influencing compli-
ance with quarantine in toronto during the 2003 sars outbreak. Biosecurity
and bioterrorism: biodefense strategy, practice, and science 2(4):265–272
Donnelly N, Proctor-Thomson SB (2015) Disrupted work: home-based tele-
working (hbtw) in the aftermath of a natural disaster. New Technology,
Work and Employment 30(1):47–61
Duffy C (2020) Big tech firms ramp up remote working orders to prevent coro-
navirus spread. URL https://www.cnn.com/2020/03/10/tech/google-
work-from-home-coronavirus/index.html
Duxbury L, Higgins C, Neufeld D (1998) Telework and the balance between
work and family: Is telework part of the problem or part of the solution?
the virtual workplace. eds. m. igbaria & m. tan. hershey
Evers KE, Castle PH, Prochaska JO, Prochaska JM (2014) Examining rela-
tionships between multiple health risk behaviors, well-being, and productiv-
ity. Psychological reports 114(3):843–853
Flora DB, Curran PJ (2004) An empirical evaluation of alternative methods of
estimation for confirmatory factor analysis with ordinal data. Psychological
methods 9(4):466
Furia CA, Feldt R, Torkar R (2019) Bayesian data analysis in empirical soft-
ware engineering research. IEEE Transactions on Software Engineering
Goodhart CAE (1984) Monetary Theory and Practice: The UK Experience.
Macmillan Press
Graziotin D, Fagerholm F (2019) Happiness and the productivity of software
engineers. In: Sadowski C, Zimmermann T (eds) Rethinking Productiv-
ity in Software Engineering, Apress open / Springer, pp 109–124, DOI
10.1007/978-1-4842-4221-6\10, URL https://doi.org/10.1007/978-1-
4842-4221-6\_10
Hanselman S (2020) Tweet. URL https://twitter.com/shanselman/
status/1252040170783641600
Harper C, Satchell L, Fido D, Latzman R (2020) Functional fear predicts
public health compliance in the covid-19 pandemic. International Journal of
Mental Health and Addiction pp 1–21
Hu L, Bentler PM (1999) Cutoff criteria for fit indexes in covariance struc-
ture analysis: Conventional criteria versus new alternatives. Structural
Equation Modeling: A Multidisciplinary Journal 6(1):1–55, DOI 10.1080/
10705519909540118
Inalhan G, Ng CF (2010) Teleworker’s home office: an extension of corporate
office? Facilities 28(3/4):137–155
Jaspan C, Sadowski C (2019) No single metric captures productivity. In: Sad-
owski C, Zimmermann T (eds) Rethinking Productivity in Software Engi-
neering, Apress open / Springer, pp 13–20, DOI 10.1007/978-1-4842-4221-
6\2, URL https://doi.org/10.1007/978-1-4842-4221- 6\_2
oreskog KG (1999) How large can a standardized coefficient be. URL http:
//www.statmodel.com/download/Joreskog.pdf
Kaplan J, Frias L, McFall-Johnsen M (2020) A third of the global population
is on coronavirus lockdown — here’s our constantly updated list of countries
30 Paul Ralph et al.
and restrictions. URL https://www.businessinsider.com/countries-
on-lockdown-coronavirus-italy- 2020-3?op=1
Kessler RC, Barber C, Beck A, Berglund P, Cleary PD, McKenas D, Pronk N,
Simon G, Stang P, Ustun TB, et al. (2003) The world health organization
health and work performance questionnaire (hpq). Journal of Occupational
and Environmental Medicine 45(2):156–174
Kim Y, Zhong W, Jehn M, Walsh L (2015) Public risk perceptions and preven-
tive behaviors during the 2009 h1n1 influenza pandemic. Disaster medicine
and public health preparedness 9(2):145–154
Ko AJ (2019) Why we should not measure productivity. In: Sadowski C,
Zimmermann T (eds) Rethinking Productivity in Software Engineering,
Apress open / Springer, pp 21–26, DOI 10.1007/978-1-4842-4221-6\3, URL
https://doi.org/10.1007/978-1-4842-4221- 6\_3
Lavelle J (2020) Gartner cfo survey reveals 74% intend to shift some
employees to remote work permanently. https://www.gartner.com/
en/newsroom/press-releases/2020-04-03- gartner-cfo-surey-
reveals-74-percent-of- organizations-to-shift-some-employees-
to-remote-work-permanently2
Loose VW, Vargas VN, Warren DE, Starks SJ, Brown TJ, Smith BJ
(2010) Economic and policy implications of pandemic influenza. Report
SAND2010-1910, Sandia National Laboratories, Albuquerque, New Mexico
March
Madhav N, Oppenheim B, Gallivan M, Mulembakani P, Rubin E, Wolfe N
(2017) Chapter 17. pandemics: risks, impacts, and mitigation. Disease con-
trol priorities, 9
McInerney CR (1999) Working in the virtual office: Providing information
and knowledge to remote workers. Library & information science research
21(1):69–89
Meyer AN, Murphy GC, Zimmermann T, Fritz T (2017) Retrospecting on
work and productivity: A study on self-monitoring software developers’
work. PACMHCI 1(CSCW):79:1–79:24, DOI 10.1145/3134714, URL https:
//doi.org/10.1145/3134714
North CS, Pfefferbaum B, Hong BA, Gordon MR, Kim YS, Lind L, Pollio
DE (2010) The business of healing: Focus group discussions of readjustment
to the post-9/11 work environment among employees of affected agencies.
Journal of occupational and environmental medicine 52(7):713–718
Paton D (2008) Community resilience: Integrating individual, community and
societal perspectives. In: Gow K, Paton D (eds) The phoenix of natural
disasters: Community resilience, Nova Science Publishers New York, pp 13–
31
erez MP, S´anchez AM, de Luis Carnicer P, Jim´enez MJV (2004) A technology
acceptance model of innovation adoption: the case of teleworking. European
Journal of innovation management
Perry SJ, Rubino C, Hunter EM (2018) Stress in remote work: two studies
testing the demand-control-person model. European Journal of Work and
Organizational Psychology 27(5):577–593
Pandemic Programming 31
Prati G, Pietrantoni L, Zani B (2011) A social-cognitive model of pandemic
influenza h1n1 risk perception and recommended behaviors in italy. Risk
Analysis: An International Journal 31(4):645–656
Ralph P, Tempero E (2018) Construct validity in software engineering research
and software metrics. In: Proceedings of the 22nd International Conference
on Evaluation and Assessment in Software Engineering 2018, pp 13–23
Rebmann T (2009) Infectious disease disasters: Bioterrorism, emerging infec-
tions, and pandemics. In: APIC text of infection control and epidemiology,
3rd edn, Association for Professionals in Infection Control and Epidemiol-
ogy, Inc., pp 120.1–120.22
Roberto AJ, Goodall CE, Witte K (2009) Raising the alarm and calming fears:
Perceived threat and efficacy during risk and crisis. In: Handbook of risk
and crisis communication, Routledge, pp 287–303
Ronan KR, Alisic E, Towers B, Johnson VA, Johnston DM (2015) Disaster
preparedness for children and families: a critical review. Current psychiatry
reports 17(7):58
onkk¨o M, Evermann J (2013) A critical examination of common beliefs
about partial least squares path modeling. Organizational Research Meth-
ods 16(3):425–448
Salda˜na J (2015) The coding manual for qualitative researchers. Sage
Shultz JM, Cooper JL, Baingana F, Oquendo MA, Espinel Z, Althouse BM,
Marcelin LH, Towers S, Espinola M, McCoy CB, et al. (2016) The role
of fear-related behaviors in the 2013–2016 west africa ebola virus disease
outbreak. Current psychiatry reports 18(11):104
Spataro J (2020) Helping our developers stay productive while work-
ing remotely. URL https://www.microsoft.com/en-us/microsoft-
365/blog/2020/03/20/helping-developers-stay-productive-
working-remotely/
Taha S, Matheson K, Cronin T, Anisman H (2014) Intolerance of uncertainty,
appraisals, coping, and anxiety: The case of the 2009 h 1 n 1 pandemic.
British journal of health psychology 19(3):592–605
Teasdale E, Yardley L, Schlotz W, Michie S (2012) The importance of coping
appraisal in behavioural responses to pandemic flu. British journal of health
psychology 17(1):44–59
Thommes E, Cojocaru M, Athar S (2016) Absenteeism impact on local econ-
omy during a pandemic via hybrid sir dynamics. In: International Confer-
ence on Dynamics of Disasters, Springer, pp 309–328
Topp CW, Østergaard SD, Søndergaard S, Bech P (2015) The who-5 well-being
index: a systematic review of the literature. Psychotherapy and psychoso-
matics 84(3):167–176
WHO (2020a) Coronavirus disease 2019 (covid-19): situation re-
port 70. URL https://www.who.int/docs/default-source/
coronaviruse/situation-reports/20200430-sitrep-101- covid-
19.pdf?sfvrsn=2ba4e093_2
WHO (2020b) Statement on the second meeting of the international health
regulations (2005) emergency committee regarding the outbreak of novel
32 Paul Ralph et al.
coronavirus (2019-ncov). URL \url{https://www.who.int/news-room/
detail/30-01-2020-statement- on-the-second-meeting-of- the-
international-health-regulations-(2005)- emergency-committee-
regarding-the-outbreak-of- novel-coronavirus-(2019-ncov)}
Xiang YT, Zhao YJ, Liu ZH, Li XH, Zhao N, Cheung T, Ng CH (2020) The
covid-19 outbreak and psychiatric hospitals in china: managing challenges
through mental health service reform. International journal of biological
sciences 16(10):1741
Yong AG, Lemyre L (2019) Getting canadians prepared for natural disasters:
a multi-method analysis of risk perception, behaviors, and the social envi-
ronment. Natural Hazards 98(1):319–341
Yong AG, Lemyre L, Pinsent C, Krewski D (2017) Risk perception and disaster
preparedness in immigrants and canadian-born adults: Analysis of a national
survey on similarities and differences. Risk analysis 37(12):2321–2333
ResearchGate has not been able to resolve any citations for this publication.
Article
Full-text available
In the current context of the global pandemic of coronavirus disease-2019 (COVID-19), health professionals are working with social scientists to inform government policy on how to slow the spread of the virus. An increasing amount of social scientific research has looked at the role of public message framing, for instance, but few studies have thus far examined the role of individual differences in emotional and personality-based variables in predicting virus-mitigating behaviors. In this study, we recruited a large international community sample (N = 324) to complete measures of self-perceived risk of contracting COVID-19, fear of the virus, moral foundations, political orientation, and behavior change in response to the pandemic. Consistently, the only predictor of positive behavior change (e.g., social distancing, improved hand hygiene) was fear of COVID-19, with no effect of politically relevant variables. We discuss these data in relation to the potentially functional nature of fear in global health crises.
Article
Full-text available
Recently, more than 300 Chinese patients with psychiatric disorders were diagnosed with the 2019 novel coronavirus disease (COVID-19). Possible reasons quoted in the report were the lack of caution regarding the COVID-19 outbreak in January and insufficient supplies of protective gear. We outlined major challenges for patients with psychiatric disorders and mental health professionals during the COVID-19 outbreak, and also discussed how to manage these challenges through further mental health service reform in China.
Article
Full-text available
As part of our broader research agenda on the psychology of risk communication and of risk management, we use a socio-ecological approach, inspired by Bronfenbrenner (Am Psychol 32:513–531, 1977. 10.1037/0003-066x.32.7.513), to better understand how Canadians perceive and prepare for risks originating from natural disasters. In this article, we present three empirical mixed-method studies as layers of analysis—a national survey, a social-spatial multi-level analysis, and a series of qualitative interviews—to bring a better understanding of how to engage Canadians in disaster preparedness and risk management. From our data, we examined how individuals perceived and understood natural disaster risks in Canada and how these fit in their social and life contexts. Given the increasingly diverse Canadian population, we used a cultural lens to contrast immigrants to Canadian-born individuals. After introducing the conceptual background and presenting our empirical studies, we conclude with a discussion on the implications for risk communication and management for natural disasters. Our findings suggest that Canadians could benefit from culturally targeted disaster risk reduction strategies that engage individuals–communities–government at all levels and are more attuned to the realities and specificities of life stressors.
Article
Full-text available
Statistics comes in two main flavors: frequentist and Bayesian. For historical and technical reasons, frequentist statistics have traditionally dominated empirical data analysis, and certainly remain prevalent in empirical software engineering. This situation is unfortunate because frequentist statistics suffer from a number of shortcomings---such as lack of flexibility and results that are unintuitive and hard to interpret---that curtail their effectiveness when dealing with the heterogeneous data that is increasingly available for empirical analysis of software engineering practice. In this paper, we pinpoint these shortcomings, and present Bayesian data analysis techniques that provide tangible benefits---as they can provide clearer results that are simultaneously robust and nuanced. After a short, high-level introduction to the basic tools of Bayesian statistics, we present the reanalysis of two empirical studies on the effectiveness of automatically generated tests and the performance of programming languages, respectively. By contrasting the original frequentist analyses with our new Bayesian analyses, we demonstrate the concrete advantages of the latter. To conclude we advocate a more prominent role for Bayesian statistical techniques in empirical software engineering research and practice.
Conference Paper
Full-text available
Representative sampling is considered crucial for predominately quantitative, positivist research. Researchers typically argue that a sample is representative when items are selected randomly from a population. However, random sampling is rare in empirical software engineering research because there are no credible sampling frames (population lists) for the units of analysis software engineering researchers study (e.g. software projects, code libraries, developers, projects). This means that most software engineering research does not support statistical generalization, but rejecting any particular study for lack of random sampling is capricious.
Article
The popularity of remote work continues to rise, but uncertainty remains about how it influences employee well-being. We extend the Demand-Control-Person (DCP) model to test both person and job factors as important considerations in remote work, suggesting that emotional stability influences the utility of autonomy as a job resource in protecting employees from strain. Additionally, we test self-determination theory (SDT), positioning need satisfaction for autonomy, relatedness, and competence as mechanisms explaining the relationship between remote work and strain. In two field studies, high–emotional stability employees reporting high levels of autonomy experienced the lowest levels of strain, with negative relationships between extent of remote work and strain. In contrast, low–emotional stability employees who also have high autonomy appear more susceptible to strain, and this may increase when they work remotely more often. Our multilevel structural equation modelling revealed that high–emotional stability employees with high autonomy appear best positioned to meet their needs for autonomy and relatedness, even when remote work is more frequent; these in turn reduced the likelihood of strain. Thus, our results support the DCP and SDT models, revealing theoretical and practical implications for designing and managing remote work arrangements.
Article
Research has documented that immigrants tend to experience more negative consequences from natural disasters compared to native-born individuals, although research on how immigrants perceive and respond to natural disaster risks is sparse. We investigated how risk perception and disaster preparedness for natural disasters in immigrants compared to Canadian-born individuals as justifications for culturally-adapted risk communication and management. To this end, we analyzed the ratings on natural disaster risk perception beliefs and preparedness behaviors from a nationally representative survey (N = 1,089). Factor analyses revealed three underlying psychological dimensions of risk perception: external responsibility for disaster management, self-preparedness responsibility, and illusiveness of preparedness. Although immigrants and Canadian-born individuals shared the three-factor structure, there were differences in the salience of five risk perception beliefs. Despite these differences, immigrants and Canadian-born individuals were similar in the level of risk perception dimensions and disaster preparedness. Regression analyses revealed self-preparedness responsibility and external responsibility for disaster management positively predicted disaster preparedness whereas illusiveness of preparedness negatively predicted disaster preparedness in both groups. Our results showed that immigrants’ risk perception and disaster preparedness were comparable to their Canadian-born counterparts. That is, immigrant status did not necessarily yield differences in risk perception and disaster preparedness. These social groups may benefit from a risk communication and management strategy that addresses these risk perception dimensions to increase disaster preparedness. Given the diversity of the immigrant population, the model remains to be tested by further population segmentation.