Gmunden Retreat on NeuroIS 2012
Neurophysiological Correlates of Information Systems Commonly Used Self-
Reported Measures: A Multitrait Multimethod Study
Ana Ortiz de Guinea, Ryad Titah, Pierre-Majorique Léger
Corresponding author: email@example.com
Given the importance and criticality of instrument
validation in IS research, the main objective of this
study is to provide a systematic assessment of IS
construct validity via a multitrait multimethod
(MTMM). To do so, the paper uses structurally
different methods – neurophysiological and self-
reported scales - to measure three commonly used IS
constructs: engagement, arousal and cognitive load in
two different experimental settings. The experiments
involved seventeen (17) and twenty four (24)
participants respectively and consisted in using
different IS to execute a set of both instrumental and
hedonic tasks. The neurophysiological measures were
taken while participants performed their tasks, while
the psychometric measures were taken immediately
after the tasks were completed. The study’s results
generally support MTMM expectations and shed light
on the complexity of detecting the nature of mono-
method bias. More specifically, the study’s results
show that primitive perceptual IS constructs such as
arousal seem to be less affected by mono-method bias,
whereas more complex perceptual constructs such as
engagement or cognitive load have higher within
method correlations. There are two alternative
explanations for the within method correlations: a) a
method bias, or b) a combination between trait and
Finally, the study’s results prompt a series of
potentially important questions for our field and for
the scientific research process in general: What are the
implications if no correlation is found between
neurophysiological and self-reported measures? Are
actual methods for detecting and/or controlling for
mono-method bias reliable? Are we under or over
estimating the impact of mono-method bias on
Bagozzi, R. P. 2011. “Measurement and meaning in Information
Systems and organizational research: Methodological and
philosophical foundations,” MIS Quarterly, (35:2), pp. 261-292.
Brannick, M. T., Chan, D., Conway, J. M., Lance, C. E. and Spector,
P. E. 2010. “What Is Method Variance and How Can We Cope With
It? A Panel Discussion,” Organizational Research Methods, (13:3),
Burton-Jones, A. 2009. “Minimizing Method Bias through
Programmatic Research,” MIS Quarterly, (33:3), pp. 445-471.
Cameron, A. F. 2007 “Juggling Multiple Conversations with
Communication Technology: Towards a Theory of Multi-
Communicating Impacts in the Workplace,” Queen's University,
Campbell, D. T. and Fiske, D. W. 1959. “Convergent and
Discriminant Validation by the Multitrait-Multimethod Matrix,”
Psychological Bulletin, (56:2), pp. 81-105.
Churchill, G. A. 1986. “A Paradigm for Developing Better Measures,”
Journal of Marketing Research, XVI, pp.64-73.
Dimoka, A. 2010 “What Does the Brain Tell Us About Trust and
Distrust? Evidence from a Functional Neuroimaging Study,” MIS
Quarterly, (34:2), pp. 373-396.
Dimoka, A., Banker, R. D., Benbasat, I., Davis, F. D., Dennis, A. R.,
Gefen, D., Gupta, A., Ischebeck, A., Kenning, P., Pavlou, P. A.,
Müller-Putz, G., Riedl, R., vom Brocke, J. and Weber, B. “On the use
of neurophysiological tools in IS research: Developing a research
agenda for NeuroIS,” MIS Quarterly forthcoming.
Doty, D. H. and Glick, W. H. 1998. “Common Method Bias: Does
Common Methods Variance Really Bias Results?” Organizational
Research Methods, (1:4), pp. 374-406.
Fiske, D. W. and Campbell, D. T. 1992. “Citations do not Solve
Problems,” Psychological Bulletin, (112:3), pp. 393-395.
Riedl, R., Banker, R. D., Benbasat, I., Davis, F. D., Dennis, A. R.,
Dimoka, A., Gefen, D., Gupta, A., Ischebeck, A., Kenning, P., Pavlou,
P. A., Müller-Putz, G., Straub, D., vom Brocke, J. and Weber, B.
2009. “On the foundations of NeuroIS: Reflections on the Gmundem
retreat 2009,” Communication of the Association for Information
Systems, (27:15), pp. 243-264.
Spector, P. E. 2006. “Method Variance in Organizational Research:
Truth or Urban Legend?” Organizational Research Methods, (9:2),