Conference PaperPDF Available

Is more always better? Simulating Feedback Exchange in Organizations

Authors:

Abstract and Figures

More and more employees request feedback from their organizations to develop and learn. This is reflected by a growing number of digital feedback apps which facilitate high-frequency feedback exchange. However, the effect of feedback has hardly been studied on an organizational level due to complexity. Therefore, we strive to analyze organizational feedback exchange with an agent-based simulation model. Concretely, we study the effect of feedback length and feedback frequency on the organizational return on investment (ROI) of feedback exchange. Our study shows that feedback length stays in an inverted U-shape relationship with ROI. Contrarily, feedback frequency is negatively correlated with ROI. When analyzed jointly, two sweet spots arise: one for medium-length, frequent feedback, and the other, for longer infrequent feedback.
Content may be subject to copyright.
Is more always better? Simulating Feedback Exchange in
Organizations
Sacha Fuchs1, Roman Rietsche1, Stephan Aier1, and Michael Rivera²
1University of St. Gallen, Institute of Information Management, St. Gallen, Switzerland
{sacha.fuchs, roman.rietsche, stephan.aier}@unisg.ch
²Department of Strategic Management, Temple University, Philadelphia, Pennsylvania, USA
{rivera}@temple.edu
Abstract. More and more employees request feedback from their organizations
to develop and learn. This is reflected by a growing number of digital feedback
apps which facilitate high-frequency feedback exchange. However, the effect of
feedback has hardly been studied on an organizational level due to complexity.
Therefore, we strive to analyze organizational feedback exchange with an agent-
based simulation model. Concretely, we study the effect of feedback length and
feedback frequency on the organizational return on investment (ROI) of feedback
exchange. Our study shows that feedback length stays in an inverted U-shape
relationship with ROI. Contrarily, feedback frequency is negatively correlated
with ROI. When analyzed jointly, two sweet spots arise: one for medium-length,
frequent feedback, and the other, for longer infrequent feedback.
Keywords: Organizational feedback exchange, feedback app, return on
investment, simulation, agent-based modeling
1 Introduction
Employees and the generation Y request more and more feedback from their managers
[1]. Additionally, they demand instant responses which they are used to from social
media platforms [1]. This call for new forms of feedback is clearly reflected by the
increasing number of digital feedback apps that facilitate more frequent feedback
exchange [2]. For example, workstream collaboration solutions like Slack, Skype, MS
Teams or standalone feedback apps like DevelapMe0F
1
, Lattice1F
2
, 15Five2F
3
, offer a wide
array of mechanisms that can be used to facilitate feedback in organizations [3].
1
https://www.develapme.com/
2
https://lattice.com/
3
https://www.15five.com/
2
But why are organizations concerned with providing feedback to their employees?
Building upon the insight that employees can be a key component of competitive
advantage [4], the improvement of existing work practices is of high relevance [5]. One
method for helping employees to improve their work practices, is constructive and
timely feedback. The existing body of knowledge highlights the strategic value of
feedback as an essential driver of employee motivation, learning and development [6,
7]. Feedback helps improve employees’ performance, when they anticipate, seek,
receive, process, react to, and finally use feedback to adjust their practices [8].
However, the effectiveness of feedback is dependent on its structure and content as
it determines the receivers reaction [9]. A feedback message comprises the content [10],
its timing [11] and the form of delivery [12]. The study at hand focuses on two of those
components in the given context of digital feedback apps. First, the feedback content
which is at the core of any feedback. Specific feedback helps employees improve, but
if the message is too long, employees might ignore it [11]. Therefore, in this study we
analyze the feedback length as a proxy for several content dimensions. Second, we
explore the effect of feedback frequency, which is a highly discussed topic in literature
and practice. In the past feedback was seen as an annual management process such as
managers provide feedback to their employees once-a-year [13]. However, this
approach has been criticized for a long time [2, 14, 15] as in “the world isn’t really on
an annual cycle anymore for anything” [16].
The trend of more and more feedback has hardly been challenged in the literature,
since measuring this effect on an organizational level is highly complex and
problematic as components of the feedback process are interdependent and depend on
organizational characteristics [9]. Hence, the question arises as how much feedback is
necessary and beneficial for organizations. Previous studies were predominantly
focused on an individual level of analysis to build a comprehensive understanding
around the concept of feedback. These efforts have led to an extensive body of literature
that explains the processes, components, and advantages of feedback. For example,
feedback characteristics [9], behavior reactions to feedback [6, 17, 18] and feedback
efficiency [14].
3
In fact, the effects of feedback on an organizational level could only be studied
within the constraints of empirical settings. However, the strong conceptual basis
allows us to overcome those constraints and to explore the organizational effects of
feedback through well-grounded computer simulation experiments. Specifically, agent-
based modeling can be used to model emergent phenomena stemming from interactions
among individuals [19]. This allows us to generate data on the organizational level from
empirical insights gathered on the individual level. For that purpose, we strive for
answering the following research question: What is the influence of feedback length
and feedback frequency on organizational return of investment (ROI)?
We contribute to theory in several ways. First, we provide descriptive knowledge by
shedding light on the aggregation logic of existing individual-level feedback concepts
on the organizational level. Second, we are, best to our knowledge, the first studying
the interrelationship of feedback length and feedback frequency on an organizational
level analysis. Third, we propose that there is a combined sweet spot of rather short and
frequent feedbacks, delivered via a feedback app, for maximizing the impact of
feedback on the organizational ROI.
We contribute to practice by providing insights for the development of feedback
trainings for managers. Furthermore, our study allows developers of feedback apps to
derive design features from our findings. For example, an app may help feedback givers
in achieving the optimal length for their message or send a reminder when the next
feedback is due. These efforts enable organizations to enhance the ROI of their
feedback exchange and ultimately build competitive advantage.
2 Conceptual Foundation
Our simulation model builds upon three research disciplines. First, feedback as a part
of organizational science. Second, the evaluation of the ROI of corporate projects builds
upon insights from accounting and finance. Third, research on socio-technical
interactions with digital artifacts like feedback apps belong to the realm of information
systems research.
4
2.1 Definition of Feedback
Feedback in the traditional world was conceptualized as information provided by an
agent (e.g., manager, colleague, book) regarding aspects of one’s performance or
understanding. Thus, feedback is a “consequence” of performance [20]. Hence, the
purpose of feedback is to assess a state and evaluate its strengths and weaknesses once
at the end of the carried-out task [21]. Feedback was not seen as something given along
the learning process to incrementally improve performance and support self-reflection
over time [22]. Thus, this definition does not explicitly contain the idea that feedback
can have multiple purposes, such as motivation, initiation of self-regulated processes
or provision of suggestions for improvement in the future. The conceptualization of the
purpose of feedback and how it should be provided has changed. Feedback is no longer
seen as a one-time event but rather as a process in which employees have an active role
to play [23]. Consequently, more recent definitions conceptualize feedback as a process
through which employees make sense of information from various sources and use it
to enhance their work or learning strategies. Hence, this conception goes beyond
notions that feedback is principally about managers or human resources informing
employees about strengths, weaknesses and how to improve, but it rather emphasizes
the centrality of the employee’s role in sense-making and processing the comments to
improve subsequent work.
There is a broad body of research around feedback characteristics. For example,
scholars distinguish between formal and informal feedback [9]. Furthermore, feedback
differs for tasks which require skill or effort [24] and creativity or diligence [25].
Moreover, performance depends on the amount of ambiguity and uncertainty
surrounding a particular task [26].
While feedback can be applied in many areas of life, we study it in the context of
organizations. Organizations can shape their employees feedback orientation by
fostering a feedback culture [8]. Furthermore, organizational feedback develops from a
task-based approach to an organizational practice [5]. Therefore, several authors argue
that feedback should be studied as a complex product of organizational culture [8, 9,
27]. One of the reasons organizations provide feedback to their employees to gain
competitive advantage [5]. While this shows that feedback can bring positive returns if
5
it is applied correctly, it still generates cost. Concretely, providing, reading, and
reflecting upon feedback requires time resources from employees which could be used
for other productive tasks. However, investments in human capital should be analyzed
like any other corporate investment [28]. For this, the measure of ROI can be used as a
widely accepted metric throughout business [28]. Phillips [29] proposes a calculation
which sets net returns in relation to total investment cost. However, while the value of
the investments in human resources can often be determined easily, the benefits are
sometimes hard to monetize [30].
2.2 Characteristics of a Feedback Message in the Context of Feedback Apps
Feedback can either be provided verbally or in written form. Verbal feedback is mostly
delivered face-to-face, which includes body language and intonation [31]. In contrast,
written feedback is rather delayed and emotions are often hidden between the lines [32].
To facilitate written feedback, organizations have increasingly adopted feedback apps
[33]. Feedback apps are digital work tools, enabling written feedback exchange [2, 33].
Such technological artifacts make it easier for organizations to provide the increased
feedback frequency demanded by employees [34].
The length of a feedback is highly correlated with its specificity [34]. Therefore,
insights about the relationship between specificity and performance can assumed to be
existent for feedback length. While high specificity leads to enhanced performance
[35], too lengthy feedbacks might not get read at all [11]. Especially, when feedback is
provided frequently, high specificity is not effective [35]. This implies a sweet spot
which optimizes specificity and makes sure that the message will be read.
Today’s working world is characterized by a dynamically changing environment.
Therefore, annual reviews do not fit in anymore [2]. Consequently, large international
organizations such as Accenture, Adobe, Goldman Sachs or SAP implement regular
check-ins and instant feedback tools [14, 15]. Similarly, scholars suggest that feedback
should be provided more often and in an informal way. In particular, the feedback
process should follow a continuous nature [36]. Frequent feedback is more effective in
improving employee performance than infrequent feedback [35]. However,
Holderness, Olsen and Thornock [37] claim that frequent feedback is only able to
6
improve performance when employees consent to receiving high-frequency feedback.
Hence, feedback frequency has a curvilinear, inverted-U relationship with task
performance [38]. But if feedback is provided less frequently, it has to be more detailed
to be effective [39]. Furthermore, the frequency base-rate depends on the underlying
task that is performed by employees [39].
3 Research Method
3.1 Simulation
The basic idea behind the methodology of computer simulation is mimicking real-word
constructs with software code [40]. To achieve this, researchers program connections
and interactions between simplified theoretical concepts. This allows them to run
experiments with various parameter settings and analyze different outcomes [41].
Agent-based modelling is one such simulation method, which enables quantitative
theory development. As the name suggests, it consists of agents, which act upon the
given situation by pre-defined behavior rules [19]. This method is particularly useful in
conducting ‘what-if’ analyses by modifying inputs or processes [42]. Consequently,
organizational science scholars have accepted the methodology and take advantage of
simulation models in their research [4345].
In developing the agent-based simulation model, this study follows the process
proposed by Sargent [41, 45, 46]. First, the theoretical foundation is synthesized from
the existing body of literature. These insights are used to build a conceptual model.
Based on this conceptual model, the simulation is being implemented. This step
includes the calibration and validation of the model. Lastly, experiments with the built
model are conducted and the resulting data is being analyzed.
3.2 Conceptual Model Development
Next to the theoretical foundations presented in chapter two, the organizational context
plays an important role for developing the simulation. Therefore, we collected and
analyzed data in a US-bank’s call center to build an empirical foundation for the
7
simulation model. For this, we introduced a designated feedback app which was built
into the agents’ workflow. Whenever a ticket was resolved, the manager provided
feedback. While it was not mandatory to use the app, the strong integration built a
favorable foundation. Our data contains 4076 feedbacks collected over the period of
one year. Feedback exchange happened between 131 unique givers and 181 unique
receivers.
This organizational setting makes sense, as the main task of call center agent is to
solve tickets. First, solving a ticket can easily be priced by multiplying the required
time with the hourly wage of a call center agent. This is often a hurdle in measuring
return in organizational settings. Second, this task can be measured and recorded easily.
Third, task outcomes are comparable among employees. This allows managers to
identify inefficiencies and build feedback recommendations upon these insights. In
conclusion, our organizational setting features a task which requires effort and
diligence, and managers give informal feedback on it.
Concretely, three simulation model parameters stem from this data. First, to evaluate
individual work performance, we use the daily number of solved tickets per call center
agent. Second, we have information about the length of feedback messages measured
in words. Third, the number of days between feedback interactions gives us the
feedback frequency. We analyzed the distribution of these three measures with a kernel
density estimation. From this, we derived a function that allows the simulation model
to sample data that follow the empirical distribution. By feeding empirical data into the
simulation model, our results can be grounded in a more realistic scenario, which
safeguards the validity of simulation results.
3.3 Simulation Development and Validation
To develop the simulation model, this study utilizes NetLogo [47], a software tool
specifically developed for agent-based modelling. This tool has been successfully
utilized in previous studies [42] and is able to simulate organizational behavior [48].
8
Agents: The simulation consists of two types of agents. First, managers who are
responsible of several subordinates and provide feedback to them. Second, call center
agents solving support tickets. In doing so, they receive feedback from their managers.
Interactions: In the beginning of the simulation, all agents are created and
configured according to model inputs. Managers are responsible to provide feedback to
their assigned employees. This happens after a certain time interval, which is sampled
from the empirical model described previously. For this job, they must perform two
tasks. First, they need to monitor an employee’s work. Second, they need to write the
feedback message. Both tasks require a time investment from managers. The
monitoring time is randomly drawn, and the writing time is calculated based on the
number of words of a feedback and the average duration to write a word. When an
employee receives a feedback, the model triggers three actions. First, the employee
reads the feedback. Second, she needs to reflect upon the content [18]. Third, she reacts
to the feedback [9]. The first two require a time investment by the employee, which
follow the same logic as the writing and monitoring of the manager. The reaction is
modeled according to the following logic. The first decision is whether the employee
accepts the feedback [9]. If she accepts it, she decides whether she is willing to change
or not [17]. The former leads to an improved performance in the form of an increased
ticket solving speed, the latter implies an unchanged working speed. However, if the
employee does not accept the feedback in the first place, she faces another decision.
She can either react negatively and reduce her performance or ignore the message and
stay at the same output level [17]. Employeesreactions are randomly assigned to them
at the beginning of a simulation run. Afterwards, they change it based on assigned
probabilities, which reflect different personalities and business events.
After the employee reacted to the feedback, the manager again reacts to the
employee’s behavior. If the manager recognizes that the employee is changing his or
her behavior (both negatively or positively), increases the frequency and length of the
feedback message. This implies a higher feedback perceived quality, which in turn
leads to improved outcomes [9]. Table 1 summarizes the most important model
parameters.
9
Table 1. Model Parameters
Parameter
Description
Default Value
Justification
Ticket
solving time
The amount of time it takes an
employee to solve a ticket
Dist. from feedback
app
Empirical
data
Probability
of behavior
change per
round
Determines the likelihood that
an employee changes his
behavior from the one a
feedback back.
Personality type:
1: 10%
2: 25%
3: 50%
Parameter
testing
Feedback
acceptance
Whether or not an employee
accepts a feedback
TRUE or FALSE
according to
behavior change
[9]
Willingness
to change
Whether or not an employee is
willing to change
TRUE or FALSE
according to
behavior change
[9]
Negative
reaction
Whether or not an employee
shows a negative reaction
TRUE or FALSE
according to
behavior change
[49]
Feedback
length
Length of the feedback
message in words
Dist. from feedback
app
Empirical
data
Feedback
frequency
The number of days between
consecutive feedback
Dist. from feedback
app
Empirical
data
Positive
change
Base rate of improvement
(scaled with length, frequency
and learning effect)
0.5
Empirical
data
Negative
change
Performance reduction
occurring when employee
reacts negatively
0.001
Parameter
testing
Learning
effect
Scales the improvement with a
learning effect
Learning speed
follows a sigmoid-
curve
[50]
Feedback
giver
reaction
How a feedback giver reacts to
recipient’s behavior after
receiving feedback
For positive and
negative reactions
increased frequency
and length
[51]
Writing time
per word
How long it takes to write a
word (seconds)
Random: 1.5 - 4
Based on
average of
adults
Monitoring
time
Time to check employee’s
work (minutes)
Random: 4 - 10
Parameter
tests
Reading
time per
word
How long it takes to read a
word (seconds)
Random: 0.4 - 1
Based on
average of
adults
Reflection
time
Reflect upon the feedback
content (minutes)
Random: 2 - 10
Parameter
tests
10
Organizational Setting: We set the number of employees in the simulation in such
a way, that they represent a call center team. This allows us to optimize simulation
speed while capturing sufficient interactions among workers. Furthermore, the obtained
results can be scaled for larger organizations. To control the time dimensions, we set
the number of working hours per day (8) and the working days per year (261) to US-
standards. As we run the simulation for three years, this translates to 783 ticks.
To account for differences in the value of time for managers and employees, we set
an individual hourly wage for each agent type. The validity of the simulation model
was analyzed by applying three techniques [46]. First, internal validity tests ensure the
consistency of results across different simulation runs with the same setting. The model
was calibrated until there was low enough variance in the results across multiple
simulation runs. However, some variance is expected, as the various random variables
lead to different starting points. Second, degeneracy tests allowed us to set ranges for
model parameters. For example, time ranges over more than five years do not produce
valid results as the mechanisms of the simulated organization are different in the long
run. Similarly, not all employees will ever be willing to change their behavior. Lastly,
through sensitivity analysis the effects of the independent variables could be validated.
We did this by changing one independent variable at the time ceteris paribus.
3.4 Simulation Experiments
All three experiments measure ROI of feedback exchange in the simulated
organization. For this, we analyze the simulation results as follows. The measure of
return is based on the additional ticket volume the agents solved thanks to the feedback
they received. This volume is multiplied with the average ticket solving time. To
calculate returns and investments in the same unit, the total time is multiplied by the
wage of call center agents. The organization’s feedback cost consists of the agent’s and
manager’s time investments as specified in the previous section multiplied with each
agent type’s wage. This allows us to calculate ROI by subtracting the total costs from
the total gains to receive the return and then dividing the result with the total costs.
11
Table 2. Simulation Experiments (each simulation run comprised 783 time steps)
Experiments
Setup
Feedback length
We shifted the distribution of the feedback length from 0 to
800 words in steps of 10 and ran the simulation 50 times per
setting. Thus, the analyses of individual effects were based
on n = 4,050 = 81 × 50 simulation runs. The feedback
frequency was set to the baseline of the empiric data.
Feedback frequency
We shifted the distribution of the feedback frequency from
0 to 125 in steps of 1 and ran the simulation 50 times per
setting. Thus, the analyses of individual effects were based
on n = 6,300 = 126 × 50 simulation runs. The feedback
length was set to the baseline of the empiric data.
Joint effects
We shifted the distribution of the feedback length and
feedback frequency simultaneously. The length from
0 to 800 in steps of 20 and the frequency from 0 to 125 in
steps of 5. Then, we ran the simulation 30 times per setting.
Thus, the analyses of individual effects were based on n =
31,980 = 41 × 26 × 30 simulation runs.
This measure represents a ratio that shows how many times a monetary unit invested in
feedback exchange is rising financial return from it.
The first experiment varies the independent variable feedback length. To do so, we
move the distribution of the kernel density estimation. Therefore, the average sample
will be either lower or higher than in the empirical distribution. This allows us to vary
the length of feedback messages from managers. Second, we vary the frequency of the
feedbacks by again moving the empirical distribution. Finally, we vary both variables
simultaneously to study combined effects. Table 2 presents an overview of our
experiments.
12
4 Simulation Results
To analyze the data generated by our simulation experiments, we conducted regression
analyses. Hereby, the analysis of our R2-values (Tables 3-5) revealed that non-linear
models were significantly better in explaining the relationship between our independent
variables and ROI. Therefore, we present the results of our polynomial regression
analysis. Due to the highly different magnitude of the independent variables and the
dependent variable, coefficients are rather small. While we could normalize
independent variables to scale the ratio, we prefer the intuitiveness of the
operationalization of feedback length through the number of words and feedback
frequency through the amount of days between feedbacks. Furthermore, even small
changes in ROI have a significant impact for large organizations.
4.1 Individual Effects of Feedback Length
Figure 1 reveals a relationship between feedback length and ROI, which follows an
inverted U-shape. Table 3 shows that the length of feedback messages has a significant
(all parameters p<0.001) impact on the ROI of feedback in organizations (R² = 0.215).
Very short feedbacks (0-150 words) provide less return than medium ones (150-450
words). But the longer a feedback message is written, the lower the ROI gets after a
tipping point. Hence, the ideal feedback length is medium.
Table 3. Regression Models for the Individual Effects of Feedback Length
Model
Linear
Quadratic
Cubic
Intercept
1.098***
-0.087***
-0.035
FB-Length
0.0001
0.016***
0.024***
FB-Length2
-2.279e-05***
-5.908e-05***
FB-Length3
3.456e-08***
R2
0.000
0.215
0.241
F-Statistic
4.627
4906
3807
AIC
1.830e+05
1.744e+05
1.731e+05
Notes: *p < 0.01; **p < 0.005; ***p < 0.001
13
Figure 1. Individual Effects of Feedback Length
4.2 Individual Effects of Feedback Frequency
While the cubic model provides a better fit for the feedback length, figure 2 reveals that
for feedback frequency, the quadratic and cubic model are very similar. Both show a
falling ROI for larger delays between feedbacks. Therefore, table 4 indicates that ROI
is highest when organizations provide frequent feedback (R² = 0.206).
Table 4. Regression Models for the Individual Effects of Feedback Frequency
Model
Linear
Quadratic
Cubic
Intercept
1.313***
2.124***
2.253***
FB-Frequency
-0.023***
-0.063***
-0.075***
FB-Frequency2
0.0003***
0.0006***
FB-Frequency3
-1.361e-06
R2
0.173
0.206
0.207
F-Statistic
1320
819.2
548.2
AIC
2.565e+04
2.540e+04
2.539e+04
Notes: *p < 0.01; **p < 0.005; ***p < 0.001
14
Figure 2. Individual Effects of Feedback Frequency
Table 5. Regression Models for the Joint Effects of Feedback Length & Frequency
Model
Linear
Quadratic
Cubic
Intercept
5.068***
5.938***
4.942***
FB-Length
-0.004***
0.003***
0.024***
FB-Length2
-1.333e-05***
-7.494e-05***
FB-Length3
4.631e-08***
FB-Frequency
-0.032***
-0.100***
-0.133***
FB-Frequency2
0.0004***
0.0009***
FB-Frequency3
-1.342e-06***
Length × Frequency
5.245e-05***
5.189e-05***
Length2 × Frequency
9.643e-08***
Length × Frequency2
-6.127e-07***
R2
0.249
0.343
0.375
F-Statistic
5299
3345
2135
AIC
1.532e+05
1.489e+05
1.473e+05
Notes: *p < 0.01; **p < 0.005; ***p < 0.001
4.3 Joint Effects of Feedback Length and Feedback Frequency
The quadratic model (table 5) suggests that frequent (0-21 days), medium-length
feedback boasts the highest ROI potential for organizations (R2 = 0.343). The effect of
feedback length is much stronger in shorter frequencies than for longer time-periods
15
between feedbacks. The cubic model reveals another level of complexity. While the
sweet spot is also for medium-length, frequent feedback, there is another high-point for
infrequent feedback which is long (figure 3). Furthermore, the frequency shows a
higher sensitivity than in the quadratic model. The low point is represented by long
feedbacks that are sent very frequently.
Figure 3. Joint Effects of Feedback Length & Frequency
5 Discussion
In the past feedback was seen as an annual management process in which managers
provide feedback to their employees for the entire year. However, the world isn’t really
on an annual cycle anymore for anything [16]. Employees and the generation Y request
to receive more and timely feedback from their managers. Nevertheless, the trend of
more and more feedback has hardly been challenged in the literature, since measuring
this effect on an organizational level is highly complex and problematic. Hence, the
question arises as how much feedback is necessary and beneficial for organizations.
Therefore, we explored the organizational effects of feedback through well-grounded
computer simulation experiments in a specific task setting. This limits the
generalizability of our results to a subset of tasks and organizations.
Our study presents several findings. First, we show that the feedback length has a
curve-linear relationship with ROI of feedback exchange that follows an inverse U-
shape. This implies that there is a sweet spot for feedback length when optimizing ROI.
16
While very short messages do not suffice in delivering enough specificity, too long
feedbacks might not get read or overwhelm recipients. This is consistent with previous
literature [39]. However, our findings extend the current knowledge as they measure
the impact not only on performance but on ROI. This is important because the
performance gain must be financially justified [28].
Second, feedback frequency has a negative relationship with ROI. The less frequent
employees get feedback, the lower is the return for the organization. While the cost is
low, the performance is not high either. This implies that if feedback cannot be provided
frequently, the resources might not be worthwhile, and the investment should be
rejected. Nevertheless, some of our findings are not consistent with literature. While
Casas-Arce et al. [39] describe an inverted U-shape relationship between frequency and
performance, we present a falling relationship. However, we analyze another dependent
variable which is conceptualized as ROI. Furthermore, Casas-Arce et al. [39]
acknowledged that the relationship may alter for different tasks.
Third, we analyzed the joint effects of feedback length and feedback frequency. Our
analysis shows that when the days between consecutive feedbacks are low, the length
of the feedback has a large effect. If the frequency is smaller, the impact of a change in
length is much lower. Moreover, Figure 3 shows two optimal points. One represents
frequent, medium-length feedback. The other less frequent, but long feedback. This
might seem contradictory as we have previously shown a negative relationship between
ROI and feedback frequency when analyzed isolated. However, this second optimum
can be understood as very resource effective. Because feedback is not given often, the
associated costs are low. Hence, smaller improvements still have a positive ROI.
This study’s findings have several theoretical and practical implications. First, it
extends existing literature by studying individual level effects of feedback exchange on
the organizational level. This allows us to challenge the assumption that more feedback
is always better. Our study shows that organizations must analyze their investments in
feedback apps to gain the expected benefits. Second, while past research revealed
interactions of feedback length and feedback frequency on the individual level, our
study is, best to our knowledge, the first to shed a light on joint effects of these two
feedback characteristics. Third, we revealed that for maximizing ROI, organizations
17
must motivate their employees to write rather short feedback and provide it frequently.
While it requires more time resources than annual feedbacks, the return makes the
investment worthwhile.
We contribute to practice in three ways. g Second, app developers can derive design
choices from our insights. For example, a feedback app can highlight whether a
feedback message contains enough words while it is written. Additionally, managers
could receive push messages when the next feedback for an employee is due. Third,
managers must closely analyze organizational feedback exchange and adjust their
strategy by analyzing the ROI of their efforts. Our insights provide them guidance in
doing so.
6 Conclusion and Limitations
No study comes without limitations. First, to operationalize the ROI we selected ticket
solving speed as measure of return. While this allows us to overcome the hurdle of
monetizing a cultural investment [30], we ignore other important factors. For example,
the quality of the solved ticket plays an equally important role for long-term success.
Further studies could analyze the ROI with a focus on quality. Second, as any model
we had to abstract from the conceptual foundations. For instance, our simulation model
assumes that all employees stay with the organization. This is not true in practice and
might have an impact on ROI as organizations invest in resources they will not possess
in the future and therefore, cannot profit from arising competitive advantages. Third,
our results are only valid for a certain type of task. Solving tickets is a relatively easy
and repetitive task. In contrast, tasks such as drug discovery, creative work or legal
counselling are more complex and do not follow the same logic. Therefore, further
studies need to analyze the impact of task type on the ROI of feedback.
In conclusion, our findings have significant implications for both theory and
practice. We show that organizations can optimize ROI from feedback exchange by
varying the feedback length and frequency. While feedback length shows an inverted
U-shape relationship with ROI, feedback frequency is negatively correlated. When
analyzed jointly, medium-length, frequent feedback and infrequent, longer feedback
represent ROI sweet spots.
18
References
1. Barth, L.: Wie Unternehmen eine attraktive, aber anspruchsvolle neue Generation von
Potenzialträgern begeistern und binden können,
https://www.egonzehnder.com/de/insight/werben-um-die-generation-y
2. Levy, P.E., Tseng, S.T., Rosen, C.C., Lueke, S.B.: Performance Management: A Marriage
between Practice and Science Just Say “I do”. In: Buckley, M.R., Wheeler, R.A.,
Halbesleben, R.B.J. (eds.) Research in Personnel and Human Resources Management, pp.
155213. Emerald Publishing Limited (2017)
3. Lechler, R., Stoeckli, E., Rietsche, R., Uebernickel, F.: Looking Beneath the Tip of the
Iceberg: The Two-Sided Nature of Chatbots and Their Roles for Digital Feedback
Exchange. In: Proceedings of the 27th European Conference on Information Systems
(ECIS 2019) (ed.) (2019)
4. Carmeli, A., Weisberg, J.: Exploring turnover intentions among three professional groups
of employees. Human Resource Development International 9, 191206 (2006)
5. Baker, N.: Employee feedback technologies in the human performance system. Human
Resource Development International 13, 477485 (2010)
6. Ilgen, D.R., Fisher, C.D., Taylor, M.S.: Consequences of individual feedback on behavior
in organizations. Journal of Applied Psychology 64, 349371 (1979)
7. Ilgen, D.R., Barnes-Farrell, J.L., McKellin, D.B.: Performance Appraisal Process Research
in the 1980s: What Has It Contributed to Appraisals in Use? Organizational Behavior and
Human Decision Processes 54, 321368 (1993)
8. London, M., Smither, J.W.: Feedback orientation, feedback culture, and the longitudinal
performance management process. Human Resource Management Review 12, 81100
(2002)
9. H. Mulder, R., D. Ellinger, A.: Perceptions of quality of feedback in organizations.
European Journal of Training and Development 37, 423 (2013)
10. Narciss, S.: Feedback Strategies for Interactive Learning Tasks. In: Spector, J.M., Merrill,
M.D., Elen, J., Bishop, M.J. (eds.) Handbook of Research on Educational Communications
and Technology, pp. 125144. Springer, New York (2008)
11. Shute, V.J.: Focus on Formative Feedback. Review of Educational Research 78, 153189
(2008)
12. Tata, J.: The Influence of Managerial Accounts on Employees’ Reactions to Negative
Feedback. Group & Organization Management 27, 480503 (2002)
13. Buckingham, M., Goodall, A.: Reinventing Performance Management. Harvard Business
Review 93, 4050 (2015)
14. Cappelli, P., Tavis, A.: The Performance Management Revolution. Harvard Business
Review 94, 5867 (2016)
15. Pulakos, E.D., Hanson, R.M., Arad, S., Moye, N.: Performance Management Can Be
Fixed: An On-the-Job Experiential Learning Approach for Complex Behavior Change.
Industrial and Organizational Psychology 8, 5176 (2015)
16. Nisen, M.: The management cliche you really can’t afford to ignore. Quartz (2015)
17. Raemdonck, I., Strijbos, J.-W.: Feedback perceptions and attribution by secretarial
employees. Euro J of Training and Dev 37, 2448 (2013)
18. Anseel, F., Lievens, F., Schollaert, E.: Reflection as a strategy to enhance task
performance after feedback. Organizational Behavior and Human Decision Processes 110,
2335 (2009)
19
19. Bonabeau, E.: Agent-based modeling: Methods and techniques for simulating human
systems. Proceedings of the National Academy of Sciences 99, 72807287 (2002)
20. Hattie, J., Timperley, H.: The Power of Feedback. Review of Educational Research 77,
81112 (2007)
21. Schleicher, D.J., Baumann, H.M., Sullivan, D.W., Levy, P.E., Hargrove, D.C., Barros-
Rivera, B.A.: Putting the System Into Performance Management Systems: A Review and
Agenda for Performance Management Research. Journal of Management 44, 22092245
(2018)
22. Ashford, S.J., Tsui, A.S.: Self-Regulation For Managerial Effectiveness: The Role Of
Active Feedback Seeking. Academy of Management Journal 34, 251280 (1991)
23. Dawson, P., Henderson, M., Mahoney, P., Phillips, M., Ryan, T., Boud, D., Molloy, E.:
What makes for effective feedback: staff and student perspectives. Assessment &
Evaluation in Higher Education 44, 2536 (2019)
24. Baumeister, R.F., Hutton, D.G., Cairns, K.J.: Negative Effects of Praise on Skilled
Performance. Basic and Applied Social Psychology 11, 131148 (1990)
25. Stobbeleir, K.E.M. de, Ashford, S.J., Buyens, D.: Self-Regulation of Creativity at Work:
The Role of Feedback-Seeking Behavior in Creative Performance. AMJ 54, 811831
(2011)
26. London, M., Mone, E.M.: Performance management: Processes that reflect and shape
organizational culture and climate. The Oxford Handbook of Organizational Climate and
Culture, 79 (2014)
27. Dahling, J.J., O’Malley, A.L.: Supportive Feedback Environments Can Mend Broken
Performance Management Systems. Ind. Organ. Psychol. 4, 201203 (2011)
28. Avolio, B.J., Avey, J.B., Quisenberry, D.: Estimating return on leadership development
investment. The Leadership Quarterly 21, 633644 (2010)
29. Phillips, J.J.: Return on investment in training and performance improvement programs.
Butterworth-Heinemann, Amsterdam, Boston (2003)
30. Murray, L.W., Efendioglu, A.M.: Valuing the investment in organizational training. Ind
and Commercial Training 39, 372379 (2007)
31. Kulik, J.A., Kulik, C.-L.C.: Timing of Feedback and Verbal Learning. Review of
Educational Research 58, 79 (1988)
32. Kulhavy, R.W.: Feedback in Written Instruction. Review of Educational Research 47, 211
(1977)
33. Stöckli, E., Uebernickel, F., Brenner, W., Weierich, A., Hess, S.: Digital Feedback for
Digital Work? Affordances and Constraints of a Feedback App at InsurCorp. In:
Proceedings of the International Conference on Wirtschaftsinformatik (WI) (ed.) (2019)
34. Miller, J.S.: High tech and high performance: Managing appraisal in the information age.
Journal of Labor Research 24, 409424 (2003)
35. Park, J.-A., Johnson, D.A., Moon, K., Lee, J.: The Interaction Effects of Frequency and
Specificity of Feedback on Work Performance. Journal of Organizational Behavior
Management 39, 164178 (2019)
36. Pulakos, E.D., O’Leary, R.S.: Why Is Performance Management Broken? Ind. Organ.
Psychol. 4, 146164 (2011)
37. Holderness, D.K., Olsen, K.J., Thornock, T.A.: Assigned versus Chosen Relative
Performance Information: The Effect of Feedback Frequency on Performance. Journal of
Management Accounting Research 32, 137158 (2020)
38. Lam, C.F., DeRue, D.S., Karam, E.P., Hollenbeck, J.R.: The impact of feedback frequency
on learning and task performance: Challenging the “more is better” assumption.
Organizational Behavior and Human Decision Processes 116, 217228 (2011)
20
39. Casas-Arce, P., Lourenço, S.M., Martínez-Jerez, F.A.: The Performance Effect of
Feedback Frequency and Detail: Evidence from a Field Experiment in Customer
Satisfaction. Journal of Accounting Research 55, 10511088 (2017)
40. Law, A.M., Kelton, W.D.: Simulation Modeling and Analysis. McGraw-Hill, New York
(1991)
41. Davis, J.P., Eisenhardt, K.M., Bingham, C.B.: Developing Theory Through Simulation
Methods. AMR 32, 480499 (2007)
42. Nan, N., Tanriverdi, H.: Unifying the Role of IT in Hyperturbulence and Competitive
Advantage Via a Multilevel Perspective of IS Strategy. MIS Quarterly 41, 937958 (2017)
43. Burton, R.M., Obel, B.: Computational Modeling for What-Is, What-Might-Be, and What-
Should-Be StudiesAnd Triangulation. Organization Science 22, 11951202 (2011)
44. Fioretti, G.: Agent-Based Simulation Models in Organization Science. Organizational
Research Methods 16, 227242 (2013)
45. Haki, K., Beese, J., Aier, S., Winter, R.: The Evolution of Information Systems
Architecture: An Agent-Based Simulation Model. MIS Quarterly 44, 155184 (2020)
46. Kuhl, M.E., Steiger, N.M., Armstrong, F.B., Jones, J.A. (eds.): Proceedings of the 2005
Winter Simulation Conference. IEEE, Piscataway, NJ (2005)
47. Wilensky, U.: NetLogo. Center for Connected Learning and Computer-Based Modeling,
Northwestern University, Evanston, IL (1999)
48. Abar, S., Theodoropoulos, G., Lemarinier, P., O'Hare, G.: Agent Based Modelling and
Simulation tools: A review of the state-of-art software. Computer Science Review, 1333
(2017)
49. Murphy, K.R.: Performance evaluation will not die, but it should. Human Resource
Management Journal 30, 1331 (2020)
50. Leibowitz, N., Baum, B., Enden, G., Karniel, A.: The exponential learning equation as a
function of successful trials results in sigmoid performance. Journal of Mathematical
Psychology 54, 338340 (2010)
51. Elicker, J.D., Levy, P.E., Hall, R.J.: The Role of Leader-Member Exchange in the
Performance Appraisal Process. Journal of Management 32, 531551 (2006)
ResearchGate has not been able to resolve any citations for this publication.
Article
Full-text available
A wide range of systems for evaluating performance have been used in organisations, ranging from traditional annual performance appraisals to performance management systems built around informal, real‐time evaluations, and these systems almost always fail. Rather than continuing to make cosmetic adjustments to this system, organisations should consider dropping the practice of regularly evaluating the performance of each of their employees, focusing rather on the small subset of situations in which evaluations of performance and performance feedback are actually useful. Four barriers to successful performance evaluation are reviewed: (a) the distribution of performance, (b) the continuing failure to devise reliable and valid methods for obtaining judgments about performance, (c) the limited utility of performance feedback to employees, and (d) the limited utility of performance evaluations to organisations. In this paper, I propose ways of managing performance without relying on regular performance evaluation, refocusing managers' activities from performance management to performance leadership.
Conference Paper
Full-text available
Enterprises are forecasted to spend more on chatbots than on mobile app development by 2021. Up to today little is known on the roles chatbots play in facilitating feedback exchange. However, digitization and automation put pressure on companies to setup digital work environments that enable reskilling of employees. Therefore, a structured analysis of feedback-related chatbots for Slack was conducted. Our results propose six archetypes that reveal the roles of chatbots in facilitating feedback exchange on performance, culture and ideas. We show that chatbots do not only consist of conversational agents integrated into instant messenger but are tightly linked to complementary front-end systems such as mobile and web apps. Like the upper part of an iceberg, the conversational agent is above water and visible within the chat, whereas many user interactions of feedback-related chatbots are only possible outside of the instant messenger. Further, we extract six design principles for chatbots as digital feedback systems. We do this by analyzing chatbots and linking empirically observed design features to (meta-)requirements derived from explanatory theory on feedback, self-determination and persuasive systems. The results suggest that chatbots benefit the social environment of conversation agents and the richness of the graphical user interface of external applications.
Article
Full-text available
Since the early 2010s the literature has shifted to view feedback as a process that students do where they make sense of information about work they have done, and use it to improve the quality of their subsequent work. In this view, effective feedback needs to demonstrate effects. However, it is unclear if educators and students share this understanding of feedback. This paper reports a qualitative investigation of what educators and students think the purpose of feedback is, and what they think makes feedback effective. We administered a survey on feedback that was completed by 406 staff and 4514 students from two Australian universities. Inductive thematic analysis was conducted on data from a sample of 323 staff with assessment responsibilities and 400 students. Staff and students largely thought the purpose of feedback was improvement. With respect to what makes feedback effective, staff mostly discussed feedback design matters like timing, modalities and connected tasks. In contrast, students mostly wrote that high-quality feedback comments make feedback effective – especially comments that are usable, detailed, considerate of affect and personalised to the student’s own work. This study may assist researchers, educators and academic developers in refocusing their efforts in improving feedback.
Article
Full-text available
It has been 13 years since the last comprehensive review of the performance management (PM) literature, and a lot has changed in both research and practice in that time. The current review updates (identifying new research directions post 2004) and extends this previous work by creating a systems-based taxonomy and conceptual model of PM. We then use this model to interpret and integrate the extensive work in this area and to identify fruitful and systems-based directions for future PM work. As input to our conclusions, we reviewed the last 36-plus years of PM research (1980–2017) and conducted a comprehensive coding of all empirical PM articles. We offer several specific directions for future PM research, with the ultimate goal of improving PM in practice.
Article
Understanding how information systems (IS) architecture evolves and what outcomes can be expected from the evolution of IS architecture presents a considerable challenge for both research and practice. The evolution of IS architecture is marked by management’s efforts to keep local and short-term IS investments in line with enterprise-wide and long-term objectives, so they often employ coercive mechanisms to enforce enterprise-wide considerations on local actors. However, an organization is shaped by a multitude of heterogeneous local actors’ actions that pursue their own, sometimes conflicting, goals, norms, and values. This study offers a theory-informed simulation model that explores how IS architecture evolves and with what outcomes in various types of organizations. The simulation model is informed by institutional theory to capture various types of organizations that are characterized by different combinations of coercive, normative, and mimetic pressures, and by complex adaptive systems theory to capture the emergent character of IS architecture’s evolution. First, we outline the insights from simulation experiments. Then, building on the simulation model and theoretical insights, we discuss implications for both research and practice.
Article
Technological advancements have greatly improved the ability to provide relative performance information (RPI), including information systems capable of providing feedback on demand. We use an experiment to examine the effect of RPI feedback frequency on task performance when RPI is assigned by managers and when RPI is solicited by employees. When RPI is assigned to employees, we hypothesize and find a non-linear relationship between RPI frequency and performance such that an increase in frequency first increases and then decreases performance. In contrast, we hypothesize and find that when RPI is chosen by employees, the negative effect of highly frequent RPI on performance is not only mitigated, but reversed, due to individuals placing greater weight on chosen feedback than assigned feedback. We also find that when individuals choose to view RPI feedback, strategic effort (i.e., expending short-term resources for long-term benefits) mediates the relation between viewing RPI and subsequent performance. JEL Classifications: M10; M40; M41. Data Availability: Data available upon request.
Article
This study investigated the interaction effect between specificity (specific vs. global) and frequency of feedback (frequent vs. infrequent) on the quality of work performance. Eighty participants were recruited and randomly assigned to one of the four groups: specific and frequent feedback, global and frequent feedback, specific and infrequent feedback, and global and infrequent feedback. A 2 × 2 factorial design was adopted. Participants were asked to work on a simulated order-fulfilling task and attended 24 sessions. The dependent variable was the error rate of the completed tasks. The results showed that more frequent feedback was more effective and specific feedback was more effective than global feedback in improving the quality of performance. Furthermore, an interactive effect between feedback frequency and specificity was found. Specific feedback was more effective than global feedback when the feedback was infrequent, but global feedback was comparable to specific feedback when it was frequent.
Article
While information technology (IT) serves as a new source of sustainable competitive advantage for firms, it also induces hyperturbulent environments (or hyperturbulence) that Erode that sustainable competitive advantage. In this paper, we posit that these contradictions might be due to cross-level nonlinear causality between firmlevel IT-based strategic actions and collective-level IT-induced hyperturbulence. We develop a multilevel perspective of IS strategy for theorizing this causality, and unifying novel with established research. Complex adaptive systems theory is employed as the overarching framework for its strength in formalizing cross-level nonlinear causal paths. Using literature-based theorization and agent-based modeling, we establish two bottom-up nonlinear causal paths by which IT drives hyperturbulence: IT can act as an external force (i.e., component IT innovation) to locally instigate firm strategic actions that aggregate to temporary hyperturbulence or as an internal force (i.e., architectural IT innovation) to drive pervasive firm strategic interactions that aggregate to persistent hyperturbulence. Each causal path produces varied amounts of reducible and irreducible uncertainties and thereby renders a top-down nonlinear effect that reshapes the opportunity for IT to contribute to competitive advantage. This multilevel theorization paves the way for new, IS-specific theory regarding IT's unique role in inducing nonlinear dynamics and in affording new business strategies in today's competitive environments.
Article
This paper presents the results from a field experiment that examines the effects of non-financial performance feedback on the behavior of professionals working for an insurance repair company. We vary the frequency (weekly and monthly) and the level of detail of the feedback that the 800 professionals receive. Contrary to what we would expect if these professionals conformed to the model of the Bayesian decision maker, more (and more frequent) information does not always help improve performance. In fact, we find that professionals achieve the best outcomes when they receive detailed but infrequent (monthly) feedback. The treatment groups with frequent feedback, regardless of how detailed it is, perform no better than the control group (with monthly and aggregate information). The results are consistent with the information in the latest feedback report being most salient, and professionals in the weekly treatments overweighting their most recent performance, hampering their ability to learn. This article is protected by copyright. All rights reserved