ArticlePDF Available

Prototyping Visual Learning Analytics Guided by an Educational Theory Informed Goal

Authors:

Abstract and Figures

Prototype work can support the creation of data visualizations throughout the research and development process through paper prototypes with sketching, designed prototypes with graphic design tools, and functional prototypes to explore how the implementation will work. One challenging aspect of data visualization work is coordinating the expertise of people from a variety of roles to produce data visualizations guided by an educational theory informed goal (ETIG) in order to better support research. When collaborating, concessions must be made: typically, everyone seeks to follow the best practices established within their own disciplines. This paper attempts to illustrate how to rethink this interdisciplinary approach to adhere more strictly to educational research goals and consider how we may need to, at times, break away from best practices with the intent to evaluate the novel decisions resulting from this approach. A case study of the creation of a self-reported emotional measure is used to illustrate this type of collaboration. By taking this approach, a clear departure from best practices occurs in the scale selection for the visualization in order to support the ETIG.
Content may be subject to copyright.
!
"#$%&'(!)*+,+,-./01!2/3456!675*0/01!5056-,/83!14/979!:-!50!79485,/+056!,;7+*-!/0<+*=79!1+56(!!"#$%&'(")(*+&$%,%-(.%&'/0,12>!3"?'>!%%@A%B#(!
;,,.CDD9E(9+/(+*1D%$(%F&$FDG65(#$%&(??(H!
IJJK!%L#LMHH@$!"+06/07'(!N;7!O+4*056!+<!P75*0/01!Q056-,/83!R+*S3!4097*!5!T*75,/27!T+==+03!P/87037>!Q,,*/:4,/+0!M!K+0T+==7*8/56MK+U7*/23!?($!V0.+*,79!"TT!WXMKTMKU!?($'!
!
445(
Prototyping Visual Learning Analytics Guided by an Educational
Theory Informed Goal
Garron&Hillaire&
TQJN>!I08(!VJQ!
N;7!Y.70!V0/27*3/,->!IZN>!V[!
\5**+0(]/665/*7^+.70(58(4S!
Gabrielle&Rappolt-Schlichtmann&
N;7!]5*25*9!\*5945,7!J8;++6!+<!Z9485,/+0!
Z9!N+17,;7*%>!VJQ!
Kim&Ducharme&
K+*,;753,7*0!V0/27*3/,-!
TQJN>!I08(!VJQ!
ABSTRACT:& )*+,+,-.7! R+*S! 850! 34..+*,! ,;7! 8*75,/+0! +<! 95,5! 2/3456/_5,/+03! ,;*+41;+4,! ,;7!
*7375*8;! 509! 97276+.=70,! .*+8733!,;*+41;! .5.7*! .*+,+,-.73! R/,;! 3S7,8;/01>! 973/1079!
.*+,+,-.73! R/,;! 1*5.;/8! 973/10! ,++63>! 509! <408,/+056! .*+,+,-.73! ,+! 7E.6+*7! ;+R! ,;7!
/=.67=70,5,/+0! R/66! R+*S(! Y07!8;566701/01! 53.78,! +<! 95,5! 2/3456/_5,/+0!R+*S!/3!8++*9/05,/01! ,;7!
7E.7*,/37! +<! .7+.67! <*+=! 5! 25*/7,-! +<! *+673! ,+! .*+9487! 95,5! 2/3456/_5,/+03! 14/979! :-! 50!
79485,/+056!,;7+*-!/0<+*=79!1+56!"ZNI\'!/0!+*97*!,+!:7,,7*!34..+*,!*7375*8;(!`;70!8+665:+*5,/01>!
8+08733/+03! =43,! :7! =597C! ,-./8566->! 727*-+07! 377S3! ,+! <+66+R! ,;7! :73,! .*58,/873! 73,5:6/3;79!
R/,;/0!,;7/*!+R0!9/38/.6/073(!N;/3!.5.7*!5,,7=.,3!,+!/6643,*5,7!;+R!,+!*7,;/0S!,;/3!/0,7*9/38/.6/05*-!
5..*+58;!,+! 59;7*7! =+*7!3,*/8,6-!,+!79485,/+056! *7375*8;!1+563!509!8+03/97*!;+R! R7! =5-! 0779!
,+>! 5,! ,/=73>! :*75S! 5R5-! <*+=! :73,! .*58,/873! R/,;! ,;7! /0,70,! ,+! 725645,7! ,;7! 0+276! 978/3/+03!
*7346,/01!<*+=!,;/3!5..*+58;(!Q!8537!3,49-!+<!,;7!8*75,/+0!+<!5!376<M*7.+*,79!7=+,/+056!=7534*7!/3!
4379!,+!/6643,*5,7!,;/3!,-.7!+<!8+665:+*5,/+0(!W-!,5S/01!,;/3!5..*+58;>!5! 8675*!97.5*,4*7!<*+=!:73,!
.*58,/873!+884*3!/0!,;7!38567!37678,/+0!<+*!,;7!2/3456/_5,/+0!/0!+*97*!,+!34..+*,!,;7!ZNI\(!
Keywords:&U5,5!2/3456/_5,/+0>!2/3456!675*0/01!5056-,/83>!40/27*356!973/10!<+*!675*0/01!"VUP'>!.5.7*!
.*+,+,-.7>!<+*=5,/27!725645,/+0>!34==5,/27!725645,/+0!
1 INTRODUCTION
Q,! ,;7! P75*0/01! Q056-,/83! J4==7*! I03,/,4,7! #$%B>! R7! ;769! 5! R+*S3;+.! +0! 43/01! ,;7! .5.7*! .*+,+,-.7!
.*+8733!,+! 8*75,7! 95,5! 2/3456/_5,/+03(!I0! ,;/3! R+*S3;+.>! R7! /6643,*5,79! 5! .*+8733! ,;5,! 3,5*,79! R/,;!
865*/<-/01!,;7!79485,/+056! ,;7+*-! ,+! :7!4379! <+*! ,;7!95,5! 2/3456/_5,/+0!509! R+*S79!R/,;! .5*,/8/.50,3! ,+!
.4*347! .*+,+,-./01! 95,5! 2/3456/_5,/+03! <+*! ,;7/*! +R0! .*+G78,3(! )*+948,/27! <779:58S! <766! /0,+! ,R+!
%!!RRR(79M,+17,;7*(+*1!
!
"#$%&'(!)*+,+,-./01!2/3456!675*0/01!5056-,/83!14/979!:-!50!79485,/+056!,;7+*-!/0<+*=79!1+56(!!"#$%&'(")(*+&$%,%-(.%&'/0,12>!3"?'>!%%@A%B#(!
;,,.CDD9E(9+/(+*1D%$(%F&$FDG65(#$%&(??(H!
IJJK!%L#LMHH@$!"+06/07'(!N;7!O+4*056!+<!P75*0/01!Q056-,/83!R+*S3!4097*!5!T*75,/27!T+==+03!P/87037>!Q,,*/:4,/+0!M!K+0T+==7*8/56MK+U7*/23!?($!V0.+*,79!"TT!WXMKTMKU!?($'!
!
446(
85,71+*/73(!N;7! </*3,! 85,71+*-! +<! <779:58S! R53! ,;5,! ,;7! R+*S3;+.! R+469! ;527! =+279! =48;! =+*7!
3=++,;6-!/<! R7!</*3,! .*+2/979! 5! R+*S79! 7E5=.67!,;5,! /6643,*5,79! ,;7! .*+8733! 3+!,;5,! .5*,/8/.50,3! 8+469!
;527! 5! .+3/,/27! =+976! +<! ,;7! 5..*+58;(!N;7! 378+09! R53! 5! 973/*7! ,+! 4097*3,509! =+*7! :7;/09! ,;7!
*5,/+0567!+<!3,5*,/01!R/,;! 79485,/+056!,;7+*-!</*3,!R;70! 8*75,/01! 2/3456!675*0/01!5056-,/83(!I0!,;/3! .5.7*>!
R7!R/66!599*733!:+,;!+<!,;737!8+087*03!*5/379!94*/01!+4*!R+*S3;+.(!a/*3,>!R7!R/66!5*,/8465,7!R;-!R7!,;/0S!
3,5*,/01! :-! 73,5:6/3;/01! 50! 79485,/+056! ,;7+*-! 850! :7! :707</8/56! 509! ,;70! 437! 5! R+*S79! 7E5=.67! 53! 5!
8537!3,49-! +<!;+R! ,+! 3,5*,! <*+=!79485,/+056! ,;7+*-!509! R+*S! ,+R5*9! 8*75,/01! 5! 95,5! 2/3456/_5,/+0(!Q3!
R/,;! 566! .*+873373>! ,;7! 3,7.3! 5*7! 5! 14/976/07!509b! 758;! .;537! +<! ,;7! R+*S! 850! :7! 8+0948,79! 53! :-! 50!
/09/2/9456! +*! 5! 1*+4.(! I0! ,;7! R+*S79! 7E5=.67>! R7! R/66! /6643,*5,7! ;+R! ,;*+41;+4,! ,;7! .*+8733! R7!
8+027*179!509!9/27*179!:7,R770!1*+4.!R+*S!509!/09/2/9456!R+*S(!
Y07! 1707*56! <*5=7R+*S! <+*!,;/0S/01! 5:+4,! ;+R! ,+! 8*75,7! 95,5! 2/3456/_5,/+03! /3! 8+=.4,5,/+056!
/0<+*=5,/+0!973/10!,;5,!341173,3!,;5,!/,!,5S73!5!=46,/9/38/.6/05*-!5..*+58;!,+!58c4/*7>!.5*37>!</6,7*>!=/07>!
*7.*7370,>!*7</07>!509!/0,7*58,!R/,;!95,5!"a*->!#$$B'(!\/270!,;7!/0,7*9/38/.6/05*-!05,4*7!+<!,;7!R+*S>!+07!
850!/=51/07!,;5,!8++*9/05,/01!,;+37!7<<+*,3!850!:7!8;566701/01(!Q,!,/=73>!,;/3!,-.7!+<!R+*S!<7763!6/S7!,;7!
25*/+43!7E.7*,3! 850! 5,,7=.,!,+! 14/97! ,;7!9/*78,/+0!+<! ,;7! 1*+4.!,+R5*9! ,;7/*! <5=/6/5*!,7**/,+*-!+<! :73,!
.*58,/873(!a*-!5*,/8465,73! ,;7!8;5667017! +<! ,;/3!,-.7! +<! /0,7*M9/38/.6/05*-!R+*S! :-! 8566/01!/,!d5!e,767.;+07!
15=7f! <+*! 8+0,7E,>! R;7*7! 758;! 3,7.! +<! ,;7! .*+8733! 9/=/0/3;73! 53.78,3! +<! ,;7! /0/,/56! c473,/+0! 4097*!
8+03/97*5,/+0g!"a*->!#$$B'(!I,! =5-! :7! ,;5,! /0! +*97*! ,+! 8+665:+*5,7!+4,3/97! +<! ,;7/*!8+=<+*,!_+073>!=50-!
7E.7*,3!R+469!0779!,+!;527!3+=7!*753+0!,+! 8;5667017!,;7/*!8+0270,/+056!,;/0S/01(!I0!,;7!8537!+<! h/3456!
P75*0/01! Q056-,/83>! /,! =5-! :7! .+33/:67! <+*! .7951+1-! ,+! 14/97! 978/3/+03! <*+=! 95,5! 8+6678,/+0! ,;*+41;!
/0,7*58,/27!95,5!2/3456/_5,/+03(!`;/67!,;7!:+4095*/73!+4,6/079!/0!a*-f3!=+976!850!:7!97:5,79>!R7!R/66!437!
,;7!*765,/+03;/.3!<+*!,;7!R+*S!+<!8+=.4,7*!38/7087>!=5,;7=5,/83>!3,5,/3,/83>!95,5!=/0/01>!1*5.;/8!973/10>!
/0<+2/3>! 509! ;4=50! 8+= .4,7*! /0,7*58,/+0! "]TI'! 53! 50! /6643,*5,/+0! +<! ;+R! .7951+1-! 850! 8++*9/05,7! ,;7!
7<<+*,3!<+*!,;7!.4*.+373!+<!8*75,/01!95,5!2/3456/_5,/+03!,;5,! 56/10!R/,;!79485,/+056!1+563(! N;7!/=.+*,50,!
c473,/+03!5*7! ,;737C!`;7*7!5*7!3,5095*9!.*58,/873!8;56670179!:-!,5S/01!,;/3! 5..*+58;i! ]+R!850! ,;+37!
/00+25,/27!978/3/+03!:7!725645,79i! \/270!,;5,!,;7! 79485,/+056!1+56!R/66!:7!,;7!9*/2/01!<+*87!:7;/09!,;7!
/00+25,/27! :7;52/+4*>! /,! 850!56R5-3! :7!4379! R;70!73,5:6/3;/01! 50!725645,/+0! 3,*5,71-(!I0! ,;7! <+66+R/01!
8537!3,49-!+<!50!/,7*5,/27!.*+8733>!R7!R/66!<+66+R!5!37*/73!+<!3,7.3!,;5,!50-!,75=!+*!/09/2/9456!R+*S/01!+0!
79485,/+056!95,5!2/3456/_5,/+0!8+469!59+.,!509!437(!
2 CONTEXT FOR THE WORKED EXAMPLE
I0! ,;7! V0/,79! J,5,73>! 5! 3/10/</850,! 8;5667017! .7*3/3,3! /0! 86+3/01! ,;7! 15.! <+*! 3,*4116/01! *7597*3! 5,! ,;7!
=/9967M38;++6!67276>! 73.78/566-!5=+01! ,;+37! R/,;!9/35:/6/,/73! "TTQQP>! #$%$b! P7354E!j!J/7176>! #$$?'(!N;7!
K5,/+056!Q33733=70,!+<!Z9485,/+056!)*+1*733!*7.+*,79! /0! #$%?! ,;5,! &$k! +<!7/1;,;!1*597!3,4970,3!R/,;!
9/35:/6/,/73!.7*<+*=79!:76+R!:53/8!67276! /0! *759/01!"0+,7!,;5,! ,;737! 3,5,/3,/83!*7<678,! +06-! 3,4970,3! R;+!
850! :7! 53373379! R/,;+4,! 588+==+95,/+03! "KQZ)>! #$%?'(! l+*7! 3.78/</8566->! 3,4970,3! R/,;! 675*0/01!
9/35:/6/,/73! *759! ?(B! 1*597! 672763! :76+R! ,;7/*! .77*3! R/,;+4,! 9/35:/6/,/73! "KOTPU>! #$$Fb! KPNJ! II>! #$$?b!
`5107*! 7,! 56(>! #$$?b! Z9=+093! 7,! 56(>! #$$Lb! \5G*/5>! O/,709*5>! J++9>! j! J58S3>!#$$H'(! I0! 599/,/+0>! =50-!
3,4970,3! R/,;! 675*0/01! 9/35:/6/,/73! 7E.7*/7087! 3/10/</850,! 3+8/56! 509! 7=+,/+056! 3,*5/0! /0! 38;++6>! ;+69!
!
"#$%&'(!)*+,+,-./01!2/3456!675*0/01!5056-,/83!14/979!:-!50!79485,/+056!,;7+*-!/0<+*=79!1+56(!!"#$%&'(")(*+&$%,%-(.%&'/0,12>!3"?'>!%%@A%B#(!
;,,.CDD9E(9+/(+*1D%$(%F&$FDG65(#$%&(??(H!
IJJK!%L#LMHH@$!"+06/07'(!N;7!O+4*056!+<!P75*0/01!Q056-,/83!R+*S3!4097*!5!T*75,/27!T+==+03!P/87037>!Q,,*/:4,/+0!M!K+0T+==7*8/56MK+U7*/23!?($!V0.+*,79!"TT!WXMKTMKU!?($'!
!
447(
0715,/27! 7=+,/+03! 5:+4,! *759/01>! 509! 7E;/:/,! 6+R! =+,/25,/+0! <+*! *759/01! /08649/01! ,7E,! 52+/950,!
:7;52/+4*! "W*-50! j! W*-50>! %LF?b! l551! j! W7;*703>! %LFLb! l5*156/,! j! m52/2>! %LFBb! J/97*/9/3>! l +4_5S/>!
J/=+3>! j! )*+,+.5.53>! #$$&b! J4,;7*6509! j! J/01;>! #$$B'(! N;737! </09/013! 5*7! 8+087*0/01! :785437>! :-!
=/9967!38;++6>!58597=/8! +4,8+=73! :78+=7!/07E+*5:6-!6/0S79! ,+!3,4970,3f! /097.70970,! 5:/6/,-! ,+!675*0!
<*+=!,7E,!509! ,;70! 7E.*733!R;5,!,;7-! S0+R! ,;*+41;!,7E,!"l4*0507! j!P72->!%LL&b!W/5085*+35!j!J0+R>!
#$$&b!)7*<7,,/>!P509/>!j!Y5S;/66>!#$$@'(!
N+!599*733!,;737!8;56670173>!,;7!T70,7*!+0!Z=7*170,!N78;0+6+1->!U/35:/6/,->!509!l/9967!J8;++6!m759/01!
"TZNb!87,(853,(+*1'!/3! *7375*8;/01!509! 97276+./01!5! ,78;0+6+1-M*/8;!675*0/01!702/*+0=70,!,;5,!R/66! =5S7!
/,! .+33/:67! <+*! 38;++63! ,+! .*+2/97! &''!3,4970,3! R/,;! 40/27*3566-! 973/1079! <+*! 675*0/01! "VUP'! 6/,7*58-!
7E.7*/70873! 53! ,;7-! *759! /097.70970,6-! 58*+33! ,;7! 84**/8464=(!VUP! +<<7*3! 5! =7503! ,+! .*+2/97!
+..+*,40/,/73! <+*! 977.! 675*0/01! ,;*+41;! ,;7! 973/10! +<! ;/1;6-! <67E/:67! =7,;+93>! =5,7*/563>! 509!
533733=70,3!"l7-7*>!m+37>!j!\+*9+0>!#$%?b!m+37!j!l7-7*>!#$$#b!m+37>!l7-7*>!j!]/,8;8+8S>!#$$@'(!N;7!
:53/8!.*7=/37!+<!VUP!/3!,;5,!:5**/7*3!,+!675*0/01!+884*!/0!,;7!/0,7*58,/+0!R/,;!84**/8464=!n!,;7-!5*7!0+,!
/0;7*70,! 3+676-! /0! ,;7! 85.5 8/,/73! +<! ,;7! 675*07*(!O43,! 53! 40/27*3566-! 973/1079! :4/69/013! .*+2 /97! +.,/+03!
,;5,! 588+==+95,7! 5! :*+59! 3.78,*4=! +<!437*3>! ,++63! 509!84**/8465! 973/1079! 43/01!,;7! VUP! <*5=7R+*S!
+<<7*!5!*5017!+<!+.,/+03!<+*!588733/01!509!70151/01!R/,;!675*0/01!=5,7*/563(!`;70!97276+.79!4097*!,;7!
VUP!<*5=7R+*S>!9/1/,56!702/*+0=70,3!.*+2/97!,;7!0787335*-!/0<*53,*48,4*7!509!<67E/:/6/,-!,+!566+R!<+*!,;7!
8*75,/+0!+<!588733/:67>!;/1;6-!7<<78,/27!5..*70,/873;/.!702/*+0=70,3!R;7*7!3,4970,3!5*7!58,/276-!14/979!
/0! ,;7! .*+8733! +<! 8+03,*4 8,/01! =750/01! ,;*+41;! ,;7! .*+2/3/+0! +<! G4 3,M/0M,/=7! <779:58S! 509! 8+0,7E,456!
34..+*,3! ,;5,! 850! :7! 1*594566-! R/,;9*5R0! 53! 3,4970,! 7E.7*,/37! /08*75373! "T+66/03>! W*+R0>! j! K7R=50>!
%LFLb!)56/0835*>!%LF&>!%LLFb!)56/0835*!j!W*+R0>!%LFB'(!
TZNf3! 1+563! 5*7! ,+! 6727*517! 7=7*170,! ,78;0+6+1-! /0! ,;7! 37*2/87! +<! *759/01! 8+=.*7;703/+0>! 3S/66!
97276+.=70,>!509!/=.*+279!=+,/25,/+0!<+*!*759/01>!3+!,;5,!R;70727*!3,4970,3!5*7!*759/01!,;7-!R/66!:7!
5:67! ,+! .*58,/87! 509! 1*+R! ,;7/*! *759/01! 8+=.*7;703/+0! 3S/663! ,;*+41;! dG43,M*/1;,g! *759/01! 8;56670173(!
l+*7! 3.78/</8566->! TZN!/3!6727*51/01! ,;7! 85.5:/6/,/73! +<! ,;7! `7:! /,376<! ,+! 8*75,7! 5!=+*7! 9/3,*/:4,79!
78+3-3,7=!<+*!59+673870,!6/,7*58->!,7*=79!dV9/+>g!R;7*7!,;7!1+563!5*7!,+!%'!<+3,7*!5!.533/+05,7!/0,7*73,!
509!/0273,=70,!/0!*759/01!<+*!3,4970,3!R;+!;527!,*59/,/+0566-!:770!40/0,7*73,79!/0>!+*!9/370<*508;/379!
:->!,*59/,/+056!86533*++=!6/,7*58-!.*58,/873b!509!#'!,+!34:3,50,/566-!/=.*+27!,;7!*759/01!8+=.*7;703/+0!
3S/663!+<!=/9967M38;++6!3,4970,3!R;+!;527!7E.7*/70879!*784**70,!<5/64*7!/0!,;7!9+=5/0!+<!*759/01(!
V9/+! /3! 5! 9-05=/8! R7:! +<! 54,;70,/8>! 84**70,>! 509! R/976-! 9/3,*/:4,79! VUP! *759/01! 702/*+0=70,3!
8+0078,79! :-! 5! 870,*56! ]4:(! m759/01! =5,7*/563! 8+0078,79! ,;*+41;! V9/+! *7.*7370,! 5! 8+665:+*5,/27!
07,R+*S! +<! .5*,07*3! R;+37! +06/07! 8+0,70,! /3! 56*759-! +0! ,;7! R7:! 509! +<! ;/1;! /0,7*73,! ,+! 3,*4116/01!
59+673870,!*7597*3! 509! ,;7/*! ,758;7*3(!T4**70,!07,R+*S!.5*,07*3!/086497!J8;+653,/8>!a58/01!]/3,+*-!509!
Y4*376273>!,;7!JZm)!I03,/,4,7D`+*9!\707*5,/+0>!,;7!)+373!a5=/6-!a+4095,/+0DJ,+*-!J;5*7>!N;7!T+4*517!
T4**/8464=>!J8/7087!a*/95->!509! 81,+%0,),1( .9+$,1&%(! K7,R+*S! .5*,07*3! 5*7! R+*S/01!R/,;! TZN! ,+!*7097*!
,;7/*!+06/07!8+0,70,!8+=.6/50,!R/,;!,;7!`7:!T+0,70,!Q88733/:/6/,-!\4/976/073!"`TQ\'!27*3/+0!#($!509!,+!
566+R!<+*!,;7!.*+2/3/+0!+<!VUP!34..+*,3!509!385<<+693!,+!<58/6/,5,7!*759/01!8+=.*7;703/+0!97276+.=70,(!
I0! ,;/3! R5->! *759/01! =5,7*/563!R/,;/0! V9/+! 5*7!d,4079g! ,+! 3,4970,3f! 3.78/</8!3,*701,;3! 509! R75S073373!
!
"#$%&'(!)*+,+,-./01!2/3456!675*0/01!5056-,/83!14/979!:-!50!79485,/+056!,;7+*-!/0<+*=79!1+56(!!"#$%&'(")(*+&$%,%-(.%&'/0,12>!3"?'>!%%@A%B#(!
;,,.CDD9E(9+/(+*1D%$(%F&$FDG65(#$%&(??(H!
IJJK!%L#LMHH@$!"+06/07'(!N;7!O+4*056!+<!P75*0/01!Q056-,/83!R+*S3!4097*!5!T*75,/27!T+==+03!P/87037>!Q,,*/:4,/+0!M!K+0T+==7*8/56MK+U7*/23!?($!V0.+*,79!"TT!WXMKTMKU!?($'!
!
44:(
5*+409! *759/01! 8+=.*7;703/+0>! 7<<78,/276-! ,4*0/01! ,;7! `7:! /0,+! 5! .6587! R;7*7! 50-+07! 850! 675*0! ,+!
8+=.*7;709!,7E,!7<<78,/276-(! Q,! ,;7! 8+*7! +<! ,;7! *759/01!07,R+*S>! V9/+f3!]4:!.*+2/973!5! ;+=7! :537!+*!
.65,<+*=! <*+=! R;/8;! 3,4970,3! 509! ,758;7*3! 850! 7E.6+*7! ,;7! R/97! 07,R+*S! +<! *759/01! 7E.7*/70873!
.*+2/979!509! 35-! +*! R*/,7! 3+=7,;/01!5:+4,! R;5,! ,;7-!;527! *759(!N;7! ]4:! .*+2/973! +..+*,40/,/73!<+*!
3,4970,3! ,+! =+0/,+*! ,;7/*! .*+1*733! 509! 1*+R,;! ,;*+41;! 95,5! 5056-,/83! 509! /3! 973/1079! 7E.6/8/,6-! ,+!
34..+*,!,;7!97276+.=70,!+<!=7,58+10/,/+0!509!.+3/,/27!<776/013!5:+4,!*759/01(
3 STEP ONE: DEFINE THE EDUCATIONAL GOAL
`7!341173,!3,5*,/01!:-!97</0/01!,;7!79485,/+056!1+56!+<!5!95,5!2/3456/_5,/+0!:785437!/,!850!:7!6727*5179!
,;*+41;!,;7!.*+8733!,+!7034*7!,;5,!566!7<<+*,3! 5*7! 56/1079! ,+R5*9!58;/72/01!,;5,!1+56(!Q3!-+4!*72/3/,!,;7!
1+56!<+*!14/95087!,;*+41;+4,!,;7!.*+8733>!/,!/3!8+==+0!,+!*7</07!-+4*!1+56(!a+*!,;7!R+*S79!7E5=.67>!R7!
;527!8+=.67,79!,;7!/0/,/56!/=.67=70,5,/+0!509!5*7!=+2/01!,+R5*9!./6+,/01!,;7!R+*S!509!175*/01!4.!<+*!
3+=7!<4,4*7!R+*S!3+!,;7!<+66+R/01!1+56!;53!:770!*7</079!=46,/.67!,/=73(!
N;7!79485,/+056!1+56!,;5,!R7!R/66!437!53!50!7E5=.67!+<!2/3456!675*0/01!5056-,/83!/0<+*=79!:-!79485,/+056!
,;7+*-!/3!,+!7E.6+*7!5!40/27*356!=78;50/3=!:-!R;/8;!3,4970,3!8+469!*7<678,!+0!,;7/*!7=+,/+056!*758,/+03!
,+!675*0/01!=5,7*/56(!N;/3!/3!<*5=79!<*+=!,;7!.7*3.78,/27!,;5,!,;7!*+67!+<!7=+,/+0!/0!675*0/01!/3!8+=.67E(!
W5379! /0! 3+8/56! /0,7*58,/+0>! 7=+,/+03! 5*7! :/+6+1/856! .*+873373! ,;5,! +*150/_7! ;4=50! :7;52/+4*! :-!
8+03,*5/0/01! ,;+41;,! 509! 58,/+0! 53! ,;7! 702/*+0=70,! /3! 5..*5/379! 53! 7/,;7*! :707</8/56! +*! 97,*/=70,56!
"a/38;7*! j! W/9766>! %LLFb! a*/G95>! %LF&b! P5_5*43 >!% LL%'(! I=.+*,50,6->!.+3/,/27!509! 0715,/27!7=+,/+03!850>!
758;! +0! ,;7/*! +R0! 588+*9>! 58,/25,7! +*! 9758,/25,7! =+,/25,/+0! 3,5,73>! 509! 70;5087! +*! 97,*58,! <*+=!
.*+948,/27! 675*0/01! "W+R7*>! %LF%b! Z/8;>! [/;63,*+=>! W+R7*>! a+*153>! j! K/7970,;56>! #$$$b! J7**766>! #$$&b!
J,7/96>! l+;/M499/0>! j! Q097*3+0>! #$$&'(! N;7! 1+56! /0! *7375*8;/01! ,;/3! ,+./8! R53! ,+! 7E.6+*7! /<! R7! 8+469!
.*+2/97!3,4970,3!R/,;!50!/0,7*58,/27!95,5!2/3456/_5,/+0!R;7*7!,;7-!8+469!*7<678,!/0!437<46!R5-3!+0! ,;7/*!
7E.7*/7087! +<! *759/01(!\/270! ,;7! 1+56!+<! 5! 40/27*356! =78;50/3=! *765,79! ,+! *7<678,/01! +0! 7=+,/+0>! R7!
978/979!,+!<+843!+0!,;7!8+087.,!+<!8+*7!5<<78,!<*+=!,;7!*7375*8;!6/,7*5,4*7(!
T+*7!5<<78,!/3!5!.*78+087.,456!.*+8733>!5!074*+.;-3/+6+1/856!3,5,7!,;5,!58,/276-!*7<678,3!+07f3!725645,/+0!
+<!,;7!702/*+0=70,!/0!,R+!9/=703/+03C!<776/01!1++9!+*!:59>!<776/01!67,;5*1/8!+*!707*1/_79!"m433766>!%LF$>!
#$$@>!#$$L'(!T+*7!5<<78,!/3! 8+0,/04+43!/0! ,;5,! .7+.67! 5*7! 725645,/01! ,;7! 702/*+0=70,!*765,/27!,+!,;737!
9/=703/+03!8+03,50,6-!509!34:8+038/+436-(!N;7!8+087.,!/3!3/=/65*!,+!=++9>!:4,!/097.70970,!/0!,;5,!8+*7!
5<<78,!/3!=+*7!25*/5:67!509!*73.+03/27!,+!3,/=46/!/0!,;7!702/*+0=70,>!6/S7!8+0,70,!+07!=/1;,!*759!+06/07(!
X+4!850! /=51/07!*759/01! 50! 5*,/867!,;5,! 9+73!0+,! 56/10! R/,;! -+4*! .+/0,!+<! 2/7R!+*! R;7*7! -+4! <776! ,;7!
54,;+*! /3! 0+,! .*+2/9/01! 597c45,7! 72/97087b! 53! -+4! *759>! -+4*!8+*7! 5<<78,! 8;50173! /0! *765,/+0! ,+! ,;7!
8+0,70,(!X+4!<776!=+*7!707*1/_79>!509!.7*;5.3!0715,/27(!
Y<,70!8+*7! 5<<78,!/3!<*77M<6+5,/01!n!-+4! =/1;,! <776! 9+R0>!36/1;,6-!0715,/27!509!67,;5*1/8>!:4,!0+,!S0+R!
R;-(!]+R727*>!,;7*7!/3!3/10/</850,!72/97087!,;5,!,;7!7E.7*/7087!+<!8+*7!5<<78,!850!:78+=7!8+038/+43!509!
/0,70,/+056! "J75*67>! %LL#b! m433766>! #$$L'(! Q3! +07! 8+03/97*3! +*! /3! .*+=.,79! ,+! 8+03/97*! ;+R! ,;7-! <776!
*765,/27!,+! ,;737!,R+! 9/=703/+03! +<! 8+*7!5<<78,! ".67534*7M9/3.67534*7>!58,/25,/+0M9758,/25,/+0'!+07! 850!
!
"#$%&'(!)*+,+,-./01!2/3456!675*0/01!5056-,/83!14/979!:-!50!79485,/+056!,;7+*-!/0<+*=79!1+56(!!"#$%&'(")(*+&$%,%-(.%&'/0,12>!3"?'>!%%@A%B#(!
;,,.CDD9E(9+/(+*1D%$(%F&$FDG65(#$%&(??(H!
IJJK!%L#LMHH@$!"+06/07'(!N;7!O+4*056!+<!P75*0/01!Q056-,/83!R+*S3!4097*!5!T*75,/27!T+==+03!P/87037>!Q,,*/:4,/+0!M!K+0T+==7*8/56MK+U7*/23!?($!V0.+*,79!"TT!WXMKTMKU!?($'!
!
44;(
5..6-! g<776/01g! R+*93! ,+! ,;7/*!7E.7*/70873! ";5..->! 359>! 500+-79>!/0,7*73,79>!:+*79'>!,;43! *5/3/01! 8+*7!
5<<78,!,+! 8+038/+43! 5R5*70733>! 566+R/01! -+4!,+!:78+=7!/0,70,/+056!5:+4,!-+4*!7E.7*/70873C!dN;/3!,+./8!
R53! *7566-! :+*/01! <+*! =7(! If=! 1+/01! ,+! ,*-! 50+,;7*! ,+./8! 5*75g! +*! dN;/3! 54,;+*f3!+./0/+0! R53! *7566-!
500+-/01(!I!5=!1+/01!,+!8*75,7!5!.*+G78,!509!8+0,73,!;/3!/9753!R/,;!+,;7*!72/97087(g
`;70! .*+2/9/01! ;/1;M/0,7*73,! 8+0,70,>! 348;! 53! /0! 702/*+0=70,3! 6/S7! V9/+>! R7! 0+,79! ,;5,! dP/S/01g! 50!
5*,/<58,!"7(1(>!5*,/867>!.+3,>!2/97+>!7,8('!.*+2/973!27*-!6/,,67!/0<+*=5,/+0!,+!+,;7*3!5:+4,!R;-! ,;5,! 5*,/867!
R53! 437<46>! /0,7*73,/01>! +*! ;+R! 9/<</846,! +*! <*43,*5,/01! ,;7! 5*,/<58,! R53! .7*! 37(!Z<<78,/276-! ,;7! d6/S7g!
<75,4*7!8500+,!.*+2/97!5!=78;50/3=!,+!*7<678,!+0!,;7!8+=.67E!*+67!,;5,!7=+,/+03!.65-!/0!675*0/01(!N;7*7!
/3!72/97087!,;5,!d6/S73g! +0! R7:!8+0,70,!5*7!.*/=5*/6-!5!*7<678,/+0!+<!.+.465*/,-!n!50!+06/07!5*,/867!,;5,!
;53!56*759-!:770!d6/S79g!/3!=+*7!6/S76-!,+!5,,*58,!599/,/+056!d6/S73g!*715*96733!+<!437<460733!+*!c456/,-(!
N+! 599*733! ,;7! R75S073373! +<! ,;7! d6/S7g! <75,4 *7>! +4*!,75=! 3+41;,! ,+! .*+2/97! 5! =78;50/3=! :-! R;/8;!
3,4970,3!R+469! :7!5:67! ,+!*7<678,!7=+,/+0566-! +0!,;7/*!7E.7*/7087! +<! 5*,/8673!509!*759/01! R/,;/0! V9/+(!
N;/3! =78;50/3=! R+469! 0779! ,+! .*+2/97! 437<46! /0<+*=5,/+0! ,+! +,;7*! *7597*3! 3/=46,507+436-!R;70!
:*+R3/01!8+0,70,!,+!/0<+*=!,;7/*!8;+/873!5:+4,!R;5,!,+!*759>!509!34..+*,!,;7!97276+.=70,!+<! 3,4970,!
=+,/25,/+0!<+*!*759/01!53!,;7-!</0/3;!+07!5*,/867!509!=+27!+0!,+!50+,;7*(!N+!9+!,;/3>!R7!,++S!,;7!/975!+<!
d6/S/01g!509>! 43/01!8+*7!5<<78,! 53! 5! 14/97>!R+*S79!,+!56/10!5!07R!973/10!+<!,;/3!8+087.,!,+!,;7!*7375*8;!
6/,7*5,4*7!9738*/:/01!7=+,/+0!/0!675*0/01(!
4 STEP TWO: DEFINE TARGET USERS
V9/+f3!.*/=5*-!,5*17,!549/7087!/3!=/9967M38;++6!3,4970,3!R/,;!675*0/01!9/35:/6/,/73(!N+!7<<78,/276-!*758;!
,;/3!,5*17,!549/7087>!;+R727*>!R7!*78+10/_7!,;5,!R7!8500+,!<4*,;7*!371*715,7!+*!3,/1=5,/_7!,;5,!1*+4.!
:-!8*75,/01!5!.*/=5*/6-!*7=79/56!+*!d3.78/56!07793g!.*+1*5=!,;5,!7<<78,/276-!37.5*5,73!,;7=>!8+10/,/276-!
+*!3+8/566->!<*+=!,;7/*!.77*3!R/,;+4,!9/35:/6/,/73(!Q,!,;/3! 73.78/566-!24607*5:67!517>! R;7*7!.77*!*765,/+03!
,5S7!+0!5!9+=/05,/01!/0<647087!509!R;7*7!,;7!97276+.=70,!+<!54,+0+=-!509!376<M*71465,/+0!5*7!8*/,/856!
97276+.=70,56! 8;56670173>! /,! R/66! :7! 2/,56! ,+! *758;! +4*! ,5*17,! 549/7087! ,;*+41;! 50! 5..*+58;! ,;5,! /3!
/08643/27!*5,;7*!,;50!7E8643/27>!973/1079!,+!8;5667017>!=+,/25,7>!34..+*,>!509!79485,7!0+,!+06-!3,4970,3!
R/,;!675*0/01!9/35:/6/,/73>!:4,!,;7/*!.77*3!"509!,758;7*3'!,++(!
a4*,;7*=+*7>!53!TZN!/3!*++,79!/0!,;7!40/27*356!973/10!=+27=70,!/,!/3!/=.+*,50,!<+*!43!,+!*758;!+4*!,5*17,!
549/7087! 43/01! 5..*+58;73! ,;5,! /086497! ,;7! <466! *5017! +<! ,;7/*! .77*3>! /08649/01!3,4970,3! R/,;! d+,;7*g!
9/35:/6/,/73(!a*+=! 5! VUP!.7*3.78,/27>!3,4970,3!R/,;! 566! S/093! +<! 9/35:/6/,/73!;527!5!40/c47!*+67!,+! .65-! /0!
973/10!:785437!,;7-!5*7!.5*,/8465*6-!24607*5:67!,+!/0<67E/:67>!d+07M3/_7M</,3M566g!3+64,/+03b!,;7-!*7.*7370,!
,;7! 7917! +<! 25*/5:/6/,-! R/,;/0! ,;7! .+.465,/+0! ,;5,! 14/973! 43! ,+R5*9! =+*7! 3488733<46>! 509! /08643/27>!
3+64,/+03(!W-! 5,,709/01! ,+! ,;7! 8;56670173! <5879! :-! /09/2/94563! R/,;! =50-! 9/35:/6/,/73! /0! ,;7! 973/10!
.*+8733>! 675*0/01! 702/*+0=70,3! 850! :7! =597! =+*7! 588733/:67>! 70151/01>! 509! 7<<78,/27! <+*! ,;7!R /973,!
.+3/:67!5**5-! +<! 675*07*3(!a+*! ,;737! *753+03>! R7!/086497! 3,*4116/01! *7597*3! R/,;+4,! /970,/</79! 675*0/01!
9/35:/6/,/73!509! 3,4970,3!R/,;! 6+RM/08/97087!9/35:/6/,/73! "3.78/</8566->!54,/3=! 509!/0,76678,456! 9/35:/6/,/73'!
!
"#$%&'(!)*+,+,-./01!2/3456!675*0/01!5056-,/83!14/979!:-!50!79485,/+056!,;7+*-!/0<+*=79!1+56(!!"#$%&'(")(*+&$%,%-(.%&'/0,12>!3"?'>!%%@A%B#(!
;,,.CDD9E(9+/(+*1D%$(%F&$FDG65(#$%&(??(H!
IJJK!%L#LMHH@$!"+06/07'(!N;7!O+4*056!+<!P75*0/01!Q056-,/83!R+*S3!4097*!5!T*75,/27!T+==+03!P/87037>!Q,,*/:4,/+0!M!K+0T+==7*8/56MK+U7*/23!?($!V0.+*,79!"TT!WXMKTMKU!?($'!
!
4<=(
/0!+4*!,5*17,!*7375*8;!.+.465,/+03!",;7!65,,7*!:785437!3+!6/,,67!/3!S0+R0!5:+4,!*759/01!8+=.*7;703/+0!5,!
,;7!=/9967M38;++6!67276!<+*!,;737!1*+4.3'(!
5 STEP THREE: INTERDISCIPLINARY PAPER PROTOTYPING PROCESS
`;70!8+03/97*/01!,;7!973/10!+<!2/3456!675*0/01!5056-,/83>!,;7!.5.7*!.*+,+,-./01!.*+8733!R/,;/0!5!973/10!
,;/0S/01!=7,;+9+6+1-!;53!=50-!:707</,3!"W*+R0>!#$$Lb!W*+R0!j![5,_!#$$Lb!W*+R0!j![5,_!#$%%'(!a/*3,>!/,!
34..+*,3!,;7!8+665:+*5,/+0!+<!5!25*/7,-!+<!9/38/.6/073!/0!,;7!8+M973/10!.*+8733(!`7!*78*4/,!/0.4,!<*+=!,;7!
,78;0+6+1->! 973/10>! 509! *7375*8;! 97.5*,=70,3! *7.*7370,/01! 3S/663! 58*+33! 8+=.4,7*! 38/7087>!
=5,;7=5,/83>! 3,5,/3,/83>! 1*5.;/8! 973/10>! 509! /0<+*=5,/+0! 2/3456/_5,/+0(! J78+09>! /,! /3! /9756! <+*!
8+==40/85,/01! 509! ,73,/01! /9753! 75*6-! +0! 509! /,7*5,/01! <53,7*(! a/0566->! /,! 566+R3! -+4! ,+! <+843! ,;7!
<779:58S!+0!8+*7!/0,70,/+0!509!/0,7*58,/+0!:7<+*7!.4,,/01!,++!=50-!*73+4*873!/0,+!;/1;7*!</976/,-!2/3456!
973/10!509!/=.67=70,5,/+0(!
N;7! :53/8! /975! /3! ,+! 701517! /0! 3727*56! *+4093! +<! /09/2/9456! 509! 1*+4.! /975,/+0! 509! 3-0,;73/3(! N;/3!
+.,/=/_73!,;7!.65-/01!</769!,+!566+R!<+*!25*/5:/6/,-!/0!8+0,*/:4,/+0!3,-673>!/09/2/9456!509!1*+4.!,;/0S>!509!
,+! +.70! ,;7! 8*75,/27!15,73! ,+!,;7! .+,70,/56! <+*!8*+33M.+66/05,/+0!509! =/E/01! +<!/9753(! I0! .*58,/87>!758;!
.;537!850!:7!9+07!53!5!1*+4.!+*!53!50!/09/2/9456(!a+*!+4*!,75=>!,;7!.*+8733!+<,70!6++S3!6/S7!5!25*/5,/+0!
+0! N/=! W*+R0f3! IUZY! 9/27*17D8+027*17! "W*+R0! j! [5,_>!#$$L'(! l50-! ,/=73! ,;7! 37678,/+0! +<! /09/2/9456!
27*343!1*+4.!R+*S!/3!5!=5,,7*!+<!8+0270/7087!509!525/65:/6/,-(!]+R727*>!,;7*7!5*7!,/=73!R;70!5!9/<</846,!
.*+:67=!;53!:770!/970,/</79!509!17,,/01!727*-+07!,+17,;7*!,+!7E.6+*7!8*75,/27!3+64,/+03!;76.3!,+!=+27!
,;7!R+*S!56+01(!
5.1 Defining the Brainstorm Problem
U7</0/01! ,;7! .*+:67=>! 509! S77./01! /,! ,+.M+<M=/09! R;/67! /975,/01! 3+64,/+03>! /3! 8*48/56(! N;/3! ,+.M67276!
.*+:67=! 3,5,7=70,! R53! 1707*5,79! :5379! +0! ,;7! ZNI\! ,+! 7E.6+*7! 5! 40/27*356! =78;50/3=! :-! R;/8;!
3,4970,3!8+469!*7<678,!+0!,;7/*!7=+,/+056!*758,/+03!,+!675*0/01!=5,7*/56(!`7!437!5!437<46!d)Yh!l596/:g!
<+*=5,! <*+=! 973/10! ,;/0S/01! =7,;+9+6+1-C! ")7*3+0'! 07793! 5!R5-! ,+! ".7*3+0f3! 0779'>! :785437! "/03/1;,'!
"W5**->! #$%$'(! I,! /3!27*-! /=.+*,50,! ,+! 5*,/8465,7!,;7! .*+:67=>! :4,! 0+,! -7,!/975,7! 3+64,/+03(!N;5,! =7503!
</66/01!/0!,;7!:650S3>! :4,! *73/3,/01!,;7!,7=.,5,/+0! ,+! 35-!d;+Rg!G43,! -7,(! Y4*!.*+:67=! 3,5,7=70,!R53! ,+!
17,!5,!;+R!R7!=/1;,!<58/6/,5,7!5!3,4970,f3!376<M*7.+*,/01!+<!7=+,/+056!=7,*/83C!80#>+%0?.(%++>2(&(@&/(0"(
+AB$+22(C"@(0C+/()++'(&D"#0(0C+(&$0,1'+(0C+/( C&E+(F#20( $+&>G(&%>(C&E+( 0C+(+AB+$,+%1+( 0C&0( 0C+,$("B,%,"%(
9&00+$2( &%>( ,2( D+,%-( 1"#%0+>H(N;7! /08643/+0! +<! ;52/01! ,;7! 7E.7*/7087! ,;5,! ,;7/*! +./0/+0! =5,,7*3! /3! 5!
*7</07=70,!+<! ,;7! ZNI\!,+! /086497! ,;5,!,;7!7E.7*/7087! 3;+469!:7! /0,*/03/8566-! *7R5*9/01(!W-!8+0078,/01!
+4*!ZNI\!,+!,;7!973/10!,;/0S/01!.*+8733>!R7!15/079!<4*,;7*!865*/</85,/+0!+<!,;7!1+56(!
Y087!R7!5*7!566!+0!,;7!35=7!.517!5:+4,!,;7!.*+:67=!,+!3+627>!R7! :*5/03,+*=!5*+409!5!37*/73!+<!d;+R!
=/1;,!R7og!3779!c473,/+03C!
]+R!=/1;,!R7!8*75,7!3+=7,;/01!,;5,!566+R3!,;7!3,4970,!,+!376<M*7.+*,!50!7=+,/+056!*758,/+0!,+!
50!5*,/867i!
!
"#$%&'(!)*+,+,-./01!2/3456!675*0/01!5056-,/83!14/979!:-!50!79485,/+056!,;7+*-!/0<+*=79!1+56(!!"#$%&'(")(*+&$%,%-(.%&'/0,12>!3"?'>!%%@A%B#(!
;,,.CDD9E(9+/(+*1D%$(%F&$FDG65(#$%&(??(H!
IJJK!%L#LMHH@$!"+06/07'(!N;7!O+4*056!+<!P75*0/01!Q056-,/83!R+*S3!4097*!5!T*75,/27!T+==+03!P/87037>!Q,,*/:4,/+0!M!K+0T+==7*8/56MK+U7*/23!?($!V0.+*,79!"TT!WXMKTMKU!?($'!
!
4<4(
]+R!=/1;,!R7!437!95,5!2/3456/_5,/+0!,+!34..+*,!509!708+4*517!,;7!3,4970,!,+!R7/1;!/0i!
`;5,!58,/2/,-!95,5!37,!=/1;,!R7!6727*517i!
`;5,!3+8/56!53.78,3!=/1;,!=5S7!5!<75,4*7D<408,/+056/,-!=+*7!5,,*58,/27!+*!8+=.766/01!,+!437i!
]+R!=/1;,!R7!972/37!5!=78;50/3=! ,;5,!566+R3!3,4970,3!,+!.*+2/97!437<46!/0<+*=5,/+0! ,+!+,;7*!
*7597*3i!
!
I09/2/94566->!.7+.67!R*/,7!9+R0!/9753>!+07!.7*!.+3,M/,>!509!,;70!.4,!,;7=!+0!,;7!R566(!I9753!850!,;70!:7!
2/7R79>! 9/3843379>! 59979! ,+>! 509! 1*+4.79(! N;/3! 566+R3! <+*! =46,/.67! =+973! +<! /0.4,! n! /09/2/9456! 509!
1*+4.(!N;7!</056!3,7.!;7*7!/3!,+!978/97!53!5!1*+4.!+0!,;7!%A#!:73,!/9753!"R7!3+=7,/=73!.466!+4,!,;7!9+,3!
<+*!,;/3>!2+,/01!515/03,!25*/+43!8*/,7*/5!,+!;76.!43!978/97'(!`;70!R+*S/01!R/,;!:/117*!1*+4.3>!.+3,M/,3!5*7!
.4,!4.!870,*566-!509!758;!1*+4.!*7.+*,3!+0!,;7/*!%A#!/9753>!/0!:*/7<>!,+!,;7!R;+67!*++=(!
5.2 Diverge: Sketch and Solutions
`/,;!50!ZNI\!,+!,73,!/0!;509>!/9753!<+*!8+**73.+09/01!95,5!2/3456/_5,/+03!,+!6727*517>!509!58,/2/,-!95,5!
37,!,+!437>!,;7*7!/3!34<</8/70,! 1*+409R+*S!/0! .6587!,+!34..+*,! 3+=7!*5./9! /975! 1707*5,/+0(!Q! ,78;0/c47!
*7<7**79! ,+! 53! d8*5_-! Ff3g! "Z250>! #$%%'! /3! 1++9!<+*! c4/8S6-! 1707*5,/01! =50-!1++9>! +*/1/056! /9753>! 509!
;76.3!52+/9!3,+../01!5,! ,;7!</*3,!1++9!/975(!I0!+*97*!,+!.5*,/8/.5,7!/0!5!8*5_-!F!58,/2/,->!758;!.5*,/8/.50,!
<+693! 5! .5.7*! /0! ;56<! ,;*77! ,/=73! ,+! 8*75,7! 7/1;,! 378,/+03(! N;7! .5*,/8/.50,3! ,;70! ,5S7! </27! =/04,73! ,+!
=5S7! 7/1;,!c4/8S! 3S7,8;73! +0! ;/1;M67276! /9753>! 509! 25*/5,/+03! +<! /9753(! Y07! ,78;0 /c47! ,+! 34..+*,! ,;7!
*5./9!05,4*7!+<!,;/3!=7,;+9!/3!,+!437!5!<5,!3;5*./7!53!/,!S77.3!+07!<*+=!17,,/01!,++!97,5/679(!T+03,*5/0/01!
,;7!,/=7!563+!;76.3!,+!S77.! .5*,/8/.50,3!<*+=!17,,/01!,++! 97,5/679(!a/14*7!%!/6643,*5,73!,R+! 7E5=.673!+<!
8*5_-!F!3S7,8;73(!
Figure&1.&Two&examples&of&crazy&8&sketches.!
!
Q!<58/6/,5,+*!"R;+!;53!:770!S77./01!,*58S!+<!,/=7'!850!,;70!53S!.7+.67!,+!3,+.!9*5R/01!509!7E.65/0!,;7/*!
/9753(!Q!378+09!*+409!+<!8*5_-!Ff3!34..+*,3!/09/2/94563!,+!/,7*5,7>!509!:+**+R!"3,756p'!<*+=!5!,75==5,7f3!
1++9!/9753>! =/E>!*7M=/E>!+*!,;/0S!4.!70,/*76-!07R!/9753(!Q<,7*!5!<7R!*+4093!+<!8*5_-!Ff3>!,;7!1*+4.!,;70!
37678,3!+07!+*!,R+!:73,!/9753!,+!*7</07(!
!
"#$%&'(!)*+,+,-./01!2/3456!675*0/01!5056-,/83!14/979!:-!50!79485,/+056!,;7+*-!/0<+*=79!1+56(!!"#$%&'(")(*+&$%,%-(.%&'/0,12>!3"?'>!%%@A%B#(!
;,,.CDD9E(9+/(+*1D%$(%F&$FDG65(#$%&(??(H!
IJJK!%L#LMHH@$!"+06/07'(!N;7!O+4*056!+<!P75*0/01!Q056-,/83!R+*S3!4097*!5!T*75,/27!T+==+03!P/87037>!Q,,*/:4,/+0!M!K+0T+==7*8/56MK+U7*/23!?($!V0.+*,79!"TT!WXMKTMKU!?($'!
!
4<<(
Q0+,;7*!5..*+58;! ,+! 58;/727!,;7!35=7! +4,8+=7!/3! ,+! 8+665:+*5,7!+0! 5! R;/,7:+5*9(! `;/67!,;7! 8*5_-! F!
3,*48,4*7!;76.3!,+!7034*7!=46,/.67!.7*3.78,/273!5*7!7E.*73379!/0!,;7!.*+,+,-./01!.;537>!/,!850!:7!;76.<46!
5,!,/=73!,+!<+66+R!4.!,;5,!7E7*8/37!R/,;!5!1*+4.!R;/,7:+5*9/01!7E7*8/37!,+!<4*,;7*!*7</07!3S7,8;73(! a+*!
,;7! R+*S79! 7E5=.67! +<! ,;7! m758,! /=.67=70,5,/+0>! R7! 3S/..79! ,;7! 8*5_-! F! 3,7.! 509! 3,5*,79! R/,;! 5!
8+6678,/27! R;/,7:+5*9! 7E7*8/37(! I0! a/14*7! #>! R7! ;+079! /0! +0! ,;7! /975! +<! *7.*7370,/01! 7=+,/+056!
9/=703/+03!+0!5E73!509!5!R/09*+37!53!5!0/876-!7E.*733/27!2/3456/_5,/+0!,-.7(!
!
Figure&2.&Example&of&a&whiteboard&outcome&of&converging&collaboration.
5.3 Converge: Refine
Y07!+<!,;7!</*3,!3,7.3!/0!*7</0/01!,;7!95,5!2/3456/_5,/+0!/3!,+!865*/<-!,;7!*756/3,/8!7E.78,5,/+03(!Q3!5!1*+4.>!
,75=3!850!=5S7!5!6/3,!+<!767=70,3!865*/<-/01!,;+37!7E.78,5,/+03(!a+*!7E5=.67C!
T+0,70,!*7c4/*7=70,3!"R;5,!-+4*!.*+,+,-.7!07793!,+!9+>!;+R!/,!<408,/+03>!R;5,!
/0<+*=5,/+0!/,!8+0,5/03>!509!,;7!05,4*7!+<!,;7!95,5!37,'!
)*/=5*-!/0<+D7E.7*/7087!"7(1(>!9/3.65-3D1*5.;3>!95,5!.+/0,3>!/0,7*58,/+03!R/,;>!7,8('!
J78+095*-!/0<+!"7(1(>!/0,*+!,7E,>!=7043>!95,73>!65:763>!6/0S3>!7,8('!
K52/15,/+0!n!R/,;/0>!509!16+:56!
l5S7!.*76/=/05*-!0+,73!+0!R;5,!-+4!R/66!,73,!+*!53S!+<!5!437*!
!
JS7,8;/01!R/,;!5!*756!95,5!37,!/3!/9756(!`;70!,;/3!/3!0+,!50!+.,/+0>!/,!850!R+*S!R766!,+!8*75,7!50!/=51/079!
+*!<5:*/85,79!95,5!37,>!7/,;7*!:7<+*7!/0/,/56!3S7,8;/01!+*!:7<+*7!,;7!*7</0/01!3,5173!"7(1(>!/03/97!+<!5!1/270!
3733/+0>! =-! 437*! R/66! 86/8S! /0,+! +07! 5*,/867! 509! 376<M*7.+*,! 50! 7=+,/+0 56! *758,/+0! ,+! ,;5,! 5*,/867'(!U5,5!
3S7,8;/01!n!,*-/01!+4,!25*/+43!2/3456!+4,.4,3!43/01!8;5*,/01!3+<,R5*7!n!/3!5!1*75,!R5-!,+!17,!5!37037!+<!
,;7!.+,70,/56!3;5.73!+<!-+4*!95,5!37,!509!,;7!0787335*-!*756!73,5,7!/,!=5-!*7c4/*7(!
I09/2/94566->! R7! 8*75,7! =+*7! 97,5/679! 3S7,8;73! +<! %A#! =5/0! 38*7703! +*! 9/3.65-3>! <466! 3/_7>! 599/01!
500+,5,/+03(!Q<,7*!:58S/01!4.!566!,;7!R+*S>!R7!.*7370,!*7</079!38*7703!<+*!9/38433/+0!509!3-0,;73/3>!,;70!
!
"#$%&'(!)*+,+,-./01!2/3456!675*0/01!5056-,/83!14/979!:-!50!79485,/+056!,;7+*-!/0<+*=79!1+56(!!"#$%&'(")(*+&$%,%-(.%&'/0,12>!3"?'>!%%@A%B#(!
;,,.CDD9E(9+/(+*1D%$(%F&$FDG65(#$%&(??(H!
IJJK!%L#LMHH@$!"+06/07'(!N;7!O+4*056!+<!P75*0/01!Q056-,/83!R+*S3!4097*!5!T*75,/27!T+==+03!P/87037>!Q,,*/:4,/+0!M!K+0T+==7*8/56MK+U7*/23!?($!V0.+*,79!"TT!WXMKTMKU!?($'!
!
4<3(
978/97!+0!5!3/0167!9/*78,/+0>!509!37,,67!+0!,;7!97,5/63(!N;/3!850!3,/66!/02+627!5!:/,!+<!=/E/01!509!=5,8;/01>!
:*/01/01!,+17,;7*!,;7!:73,!/9753!,+!R+*S!53!+07!3+64,/+0(!
N;/3!3,7.!/0864973!599/,/+056!38*7703(!Z58;!.7*3+0!,5S73!5!38*770!,+!*7</07>!/6643,*5,/01!509D+*!500+,5,/01!
<408,/+056/,->! /0,7*58,/+03>! 509! 052/15,/+0! R/,;/0! 509! :7,R770! 38*7703>! 509! 9/3.65-! 3,5,73! ,+! 3/=465,7!
50/=5,/+0!"8;5017!+27*!,/=7'(!`7!563+!437!.+3,M/,3!+*!,*58/01!.5.7*!,+!+27*65-!/0<+*=5,/+0(!
Q0+,;7*!.7*3+0!8*75,73!500+,5,79!3,+*-:+5*93! +<!,;7!<6+R!509!052/15,/+0!R/,;/0!509!:7,R770!38*7703!
509! 9/3.65-3>! 509! 17,3! =+*7! 3.78/</8! +0! 5! ,73,/01! .*+,+8+6(! I0! +4*! 8537>! R7! R50,79! ,+! ,73,! /0,7*<587!
<408,/+056/,-!509!5!<7R!9/<<7*70,!9/3.65-3!+<!,;7!95,5!37,(!
!
Figure&3.&Example&of&refined&sketch&of&“React”&data&visualization.!
m7</079! 3S7,8;73>! 53! /6643,*5,79! /0! a/14*7! ?>! 850! :7! 4379!53M/3!<+*! ,73,/01! +06-! R;70! ,;7-! ;527! :770!
8*75,79!R/,;!34<</8/70,!865*/,-(!Y,;7*R/37>!,;7-!850!37*27!53!5!:53/3!<+*!8*75,/01!.*76/=/05*-!2/3456!973/103!
+*! <408,/+056! R/*7<*5=73! <+*! ,73,/01(! `;70! R+*S/01! +0! ,;7! /=.67=70,5,/+0! +<! ,;7! /0,7*58,/+03! +<!
3,4970,3!37678,/01! ,;7/*! *758,/+03!509! 2/7R/01! +,;7*3f!*758,/+03>! R7! R50,79!,+! 588+=.6/3;!,R+!,;/013(!
a/*3,>!R7!R50,79!,+!=5S7!34*7!,;5,!,;7!/0,7*58,/27!973/10!34..+*,79!.5*,/8/.5,/+0b!378+09>!R7!R50,79!,+!
=5S7! 34*7! /,! R53! 753-! <+*! 5! 3,4970,! ,+! ,+1167! :7,R770! 377/01! ,;7/*! *758,/+03! 509! 377/01! +,;7*3f!
*758,/+03>!53!,;7!8+=.5*/3+0!R+469!8*75,7!50!+..+*,40/,-!<+*!3,4970,3!,+!*7<678,(!N;7!1+56!+<!34..+*,/01!
5!8+=.5*/3+0!:7,R770!50!/09/2/9456!*73.+037!509!,;7!*73.+0373!+<!,;7/*!.77*3!R53!/0,70979!,+!34..+*,!
,;7!ZNI\!5/=!+<!708+4*51/01!*7<678,/+0! +0! ,;7/*! 7=+,/+056!*758,/+0(!N;7!7E.6+*5,/+0!+<! 8+=.5*/01!-+4*!
*73.+037! ,+! ,;7! *73.+0373! +<! -+4*! . 77*3! /3! 50! 5,,7=.,! ,+! 58S0+R67917! ,;7! 3+8/56! 05,4*7!+<! 7=+,/+03!
+4,6/079! /0! ,;7! ZNI\(! N;7! 7E5=.67! +<! ,;7! :4,,+03! ;/1;6/1;,79! /0! *79! /6643,*5,79! 5! 973/10! 34..+*,! ,;5,!
566+R79!3,4970,3!,+!8+=.5*7!,;7/*!*73.+0373!,+!+,;7*3f!*758,/+03(!Y07!+<!,;7!</*3,!973/10!8;56670173!,;5,!
!
"#$%&'(!)*+,+,-./01!2/3456!675*0/01!5056-,/83!14/979!:-!50!79485,/+056!,;7+*-!/0<+*=79!1+56(!!"#$%&'(")(*+&$%,%-(.%&'/0,12>!3"?'>!%%@A%B#(!
;,,.CDD9E(9+/(+*1D%$(%F&$FDG65(#$%&(??(H!
IJJK!%L#LMHH@$!"+06/07'(!N;7!O+4*056!+<!P75*0/01!Q056-,/83!R+*S3!4097*!5!T*75,/27!T+==+03!P/87037>!Q,,*/:4,/+0!M!K+0T+==7*8/56MK+U7*/23!?($!V0.+*,79!"TT!WXMKTMKU!?($'!
!
4<I(
5*+37!<*+=!,;/3!R/*7<*5=7!R53!53S/01!,;7!c473,/+0>!J;+469!5!3,4970,!:7!566+R79!,+!377!+,;7*3f!*758,/+03!
:7<+*7! .*+2/9/01! ,;7/*! +R0i! `;/67! /,! 8+469! /0<647087! ,;7! 3,4970,f3! *73.+037>! /,! R+469! 34..+*,! ,;7!
3,4970,!:-! :707</,/01!<*+=!*7<678,/01!+0!+,;7*3f!*758,/+03!7270! R/,;+4,!*73.+09/01(!I<!/,!850!;76.!3.5*S!
<4*,;7*!c473,/+03!5:+4,!/=.67=70,5,/+0>!,;70!-+4!;527!5!3488733<46!3S7,8;/01!.*+8733(
6 FORMATIVE EVALUATION
`7! 8+0948,79! 5! <7R! 9/<<7*70,! *+4093! +<! ,73,/01(! N;*+41;! 75*6-! .5.7*! .*+,+,-./01>! R7! 15,;7*79! 5!
.*76/=/05*-!95,5! 37,! :-! 53S/01! 3,4970,3! ,+! =5,8;! 59G78,/273!,+! ;+R!,;7-!<76,!5:+4,!50!5*,/867!,;7-!;59!
G43,!*759(!
Figure&4.&Example&paper&prototype&for&formative&evaluation.!
N;7! ,73,7*! 3;+R79! 3,4970,3! 5! .*/0,+4,! +<! a/14*7! B! 509! 53S79! ,;7=! ,+! ,5.! ,;7/*! </017*!+0! ,;7! m758,!
:4,,+0! /0! ,;7! 38*770f3! .*/0,79! /0,7*<587(! N;70!,;7! ,73,7*! +27*65/9!5! 27*3/+0! +<! ,;7! 38*770! R/,;! ,;7!
*73.+037!59G78,/273(!N;7!,73,7*!*759!,;7!R+*93!53!,;7-!5..75*79!/0!,;7!/0,7*<587>!d]+R!R53!,;5,!5*,/867i!
T;++37!+07!+*!=+*7!":-!.+/0,/01!R/,;!-+4*!</017*'>g!,;70!53S79!,;7!3,4970,!,+!,5.!,;7!3527!:4,,+0(!N;7!
!
"#$%&'(!)*+,+,-./01!2/3456!675*0/01!5056-,/83!14/979!:-!50!79485,/+056!,;7+*-!/0<+*=79!1+56(!!"#$%&'(")(*+&$%,%-(.%&'/0,12>!3"?'>!%%@A%B#(!
;,,.CDD9E(9+/(+*1D%$(%F&$FDG65(#$%&(??(H!
IJJK!%L#LMHH@$!"+06/07'(!N;7!O+4*056!+<!P75*0/01!Q056-,/83!R+*S3!4097*!5!T*75,/27!T+==+03!P/87037>!Q,,*/:4,/+0!M!K+0T+==7*8/56MK+U7*/23!?($!V0.+*,79!"TT!WXMKTMKU!?($'!
!
4<5(
3,4970,! R53! ,;70! 53S79! R;5,! ,;7-! ,;+41;,! R+469! ;5..70! 07E,(!N;7! 7E.78,5,/+03! R7*7! *78+*979! <+*!
<4,4*7!5056-3/3!:4,!0+,!7E5=/079!/0!97.,;!5,!,;/3!3,517(!
!
Figure&5.&Example&paper&prototype&for&note&taking&during&formative&evaluation.!
N;7!,73,7*!=597!0+,73!509!,566/79!,;7!*73.+0373!R/,;!,/8S!=5*S3!+0!5!38+*7!3;77,>!53!/6643,*5,79!/0!a/14*7!
@>!3+! ,;5,! R7!8+469! 8+6678,! 5!*756!95,5! 37,! <*+=!+4*! 437*3(! a*+=!=46,/.67! *+4093!+<! ,73,/01>! ,;737!,566-!
=5*S3!R7*7!511*715,79!,+!1707*5,7!5! .*76/=/05*-!95,5! 37,(!`/,;!,;7!.*76/=/05*-!95,5! 37,!/0!;509!53!5!
*7<7*7087>!R7!8*75,79! 2/34566-! *7</079!38*7703!509! 95,5! 2/3456/_5,/+0!25*/5,/+03!,+! ,73,(! `7!.*/0,79!+4,!
,;7!2/3456!973/103!/0!a/14*7!&!<+*!50+,;7*!*+409!+<!.5.7*!,73,/01!/0!,;7!86533*++=(!Prototype evaluation is
typically done with several people. I<!-+4!R50,!5!3.78/</8!04=:7*>!K/76370!"#$%#'!;53!341173,79!,;5,!</27!
.7+.67!R+469!:7!34<</8/70,!<+*!5!435:/6/,-!3,49-(!
N;/3! 95,5! 8+6678,/+0! 563+! 566+R79! <+*! ,;7! /970,/</85,/+0! +<! R+*93! 40<5=/6/5*! ,+! ,;7! ,5*17,! .+.465,/+0(!
`;70!6++S/01!<+*!3;+*,8+=/013>!3=566!35=.67!3/_73!5*7!</07!3/087!5!3/0167!.7*3+0!850!/970,/<-!5!.*+:67=!
R/,;!,;7!973/10!3+!6+01!53!,;5,!/09/2/9456!;53!3/=/65*!8;5*58,7*/3,/83!,+!,;7!,5*17,!.+.465,/+0!"K/72770!j!
a+6=7*>!#$%?>!.(!%&?'(!I0!a/14*7!@>!-+4!850!377!5!c473,/+0!=5*S!07E,!,+!,;7!R+*9!3,567!:785437!5!3,4970,!
53S79! R;5,! ,;7! R+*9! =750,! 94*/01! ,;7! ,73,/01(! V0<5=/6/5*! R+*93! R7*7! *7=+279! <*+=! ,;7!
/=.67=70,5,/+0(!
`7!,73,79!,;7!/0,7*<587! 767=70,3!/0!5!3/0167! 86533*++=!R/,;!/09/2/9456!/0,7*2/7R3!+<! 3,4970,3!R/,;!,;7!
35=7!8;5*58,7*/3,/83!53!+4*!,5*17,!.+.465,/+0(!N;7!725645,/+0!435:/6/,-!,73,/01!<+84379!+0!5!<7R!9/<<7*70,!
!
"#$%&'(!)*+,+,-./01!2/3456!675*0/01!5056-,/83!14/979!:-!50!79485,/+056!,;7+*-!/0<+*=79!1+56(!!"#$%&'(")(*+&$%,%-(.%&'/0,12>!3"?'>!%%@A%B#(!
;,,.CDD9E(9+/(+*1D%$(%F&$FDG65(#$%&(??(H!
IJJK!%L#LMHH@$!"+06/07'(!N;7!O+4*056!+<!P75*0/01!Q056-,/83!R+*S3!4097*!5!T*75,/27!T+==+03!P/87037>!Q,,*/:4,/+0!M!K+0T+==7*8/56MK+U7*/23!?($!V0.+*,79!"TT!WXMKTMKU!?($'!
!
4<6(
9/3.65-3! +<! ,;7! 95,5! 37,! <+*! 865*/,-! 509! .*7<7*7087! 5*+409! 9/<<7*70,! 2/3456/_5,/+0! ,-.73! 509! 8+6+4*!
25*/5,/+03(!
Figure&6.&Paper&prototype&of&four&options&for&viewing&others
&reactions.!
Q15/0>!,;7!,73,7*3!R70,!,;*+41;!,;7!35=7!/0/,/56!3,7.3>!509!,;7!59G78,/273f!37678,79!3,5,73!R7*7!=5*S79!
R/,;!=/0/!.+3,M/,3(!Y087!,;7!3,4970,!,5..79!,;7!3527!:4,,+0>!5!95,5!2/3456/_5,/+0!R53!+27*65/9!R;/67!,;7!
,73,7*!*759!,;7!R+*93!53!,;7-!5..75*79!/0!,;7!/0,7*<587!dN;50S!-+4p!]7*7f3!R;5,!+,;7*3!,;+41;,(g!
N;7!<+66+R/01!,73,/01! .*+,+8+6!+4,6/073!5!37*/73! +<! c473,/+03! ,;5,! 3,4970,3!R7*7!53S79!R;/67!6++S/01!5,!
,;7!+.,/+03!+4,6/079!/0!a/14*7!&C!
J,5*,!R/,;!+07!2/3456/_5,/+0!,-.7>!509!53S!dT50!-+4!9738*/:7!R;5,!-+4!377ig!509!d`;5,!9+73!
/,!=750ig!
\/27!,;7!437*!3+=7!,/=7>!509!,;70!/<!077979>!.+/0,!,+!,;7!;76.!,7E,!509!*759!dW/117*!8/*8673!
=750!=+*7!.7+.67!8;+37!,;+37!R+*93g!+*!dW/117*!3;5.73!=750!=+*7!.7+.67!8;+37!,;+37!
R+*93(g!N;70>!.*+=.,!,;7=!,+!,*-!,+!9738*/:7!R;5,!,;7-!377!515/0(!
J;+R!,;7!+,;7*!25*/5,/+03!,+!17,!5,!.*7<7*70873!5*+409!2/3456/_5,/+0!,-.73!509!8+6+4*!
25*/5,/+0>!53S/01!c473,/+03!348;!53>!dI3!,;/3!+07!753/7*!,+!4097*3,509>g!dU+!-+4!6/S7!,;7!
3;5.73!/0!9/<<7*70,!8+6+4*3>!+*!566!/0!,;7!35=7!8+6+4*(g!d`;-ig!Q6R5-3!53S!R;-p!
]+R!R+469!-+4!17,!:58S!,+!8;5017!-+4*!503R7*i!
]+R!R+469!-+4!=+27!+0!,+!*759!50+,;7*!5*,/867i!
!
`;70!*400/01!,;7!38*7703!:-!437*3>!3,4970,3!5..75*79!,+!4097*3,509!,;7!8+087.,(!N;7-!R7*7!566!5:67!,+!
/0,7*.*7,! ,;7! /0<+*=5,/+0! 8+**78,6-(! l+*7+27*>!R;/67! ,;7-! 4097*3,++9! ,;7!/0<+*=5,/+0!533+8/5,79!R/,;!
,;7! 59G78,/273! R;70! .*7370,79! 53! 9/<<7*70,6-! 3/_79! :4::673>! ,;7-! .*7<7**79! ,;7! R/09*+37! 95,5!
*7.*7370,5,/+0(! N;7-! <+409! /,! =+*7! 7E.*733/27! R;/67! 3,/66! :7/01! /0<+*=5,/27! <+*!*7.*7370,/01! ,;7!
8;5*58,7*!+<!,;7!5*,/867(!Y4*!3=566!<+843!1*+4.!+<!</27!3,4970,3!1527!43!,;7!8+0</97087!,;5,!R7!8+469!.4,!
,+17,;7*!50!/=.67=70,5,/+0!<+*!5!./6+,(!
!
"#$%&'(!)*+,+,-./01!2/3456!675*0/01!5056-,/83!14/979!:-!50!79485,/+056!,;7+*-!/0<+*=79!1+56(!!"#$%&'(")(*+&$%,%-(.%&'/0,12>!3"?'>!%%@A%B#(!
;,,.CDD9E(9+/(+*1D%$(%F&$FDG65(#$%&(??(H!
IJJK!%L#LMHH@$!"+06/07'(!N;7!O+4*056!+<!P75*0/01!Q056-,/83!R+*S3!4097*!5!T*75,/27!T+==+03!P/87037>!Q,,*/:4,/+0!M!K+0T+==7*8/56MK+U7*/23!?($!V0.+*,79!"TT!WXMKTMKU!?($'!
!
4<7(
7 MOCK DATA
`;70!R+*S/01!+0!5! 95,5!2/3456/_5,/+0>! =+8S!95,5! 850! ;76.! /0! 8+03/97*/01! /<! -+4*! /=.67=70,5,/+0! 850!
*756/3,/8566-!9/3.65-!,;7!.+33/:67!*5017!+<!256473(!a+*!,;7!d-+4*!*73.+0373g!2/7R>!,;7!</*3,!,;/01!R7!97</07!
/3!,;7!*5017!+<!.+,70,/56!256473b!/0!,;/3!8537!/,!/3!$!+*!%>!R/,;!$!*7.*7370,/01!0+,!37678,/01!,;7!+.,/+0!R;/67!
%!*7.*7370,3!37678,/01!,;7!+.,/+0(!a*+=!,;/3!*5017>!R7!8+03/97*!5!25*/7,-!+<!.+,70,/56!37,3!+<!256473!<+*!5!
1/270!3,4970,(!I,! /3!.+33/:67! ,;5,!5! 3,4970,!;53!0+,!*73.+0979! 5,! 566>!;53!37678,79!+07!*73.+037>!!+*!;53!
37678,79!5! <7R! 9/<<7*70,! *73.+0373(!I,! ;76.3!,+!.4,!5!,5:67!+<!95,5!,+17,;7*! 53! 5!*7<7*7087(!J77!N5:67!%>!
R;/8;!/6643,*5,73! ,;5,! J,4970,!%! ;53! 0+!*758,/+0>!J,4970,! #! ;53!37678,79!Z0151/01!53! ;7*! *758,/+0>!509!
J,4970,!?!;53!37678,79!Z0151/01>!T4*/+43>!509!T+0<43/01!53!;/3!*758,/+0(!
Table&1.&Mock&Data&for&Three&Students!
!
J,4970,!%!
J,4970,!#!
J,4970,!?!
Z0151/01!
$!
%!
%!
I0,7*73,/01!
$!
$!
$!
T;566701/01!
$!
$!
$!
T4*/+43!
$!
$!
%!
T56=/01!
$!
$!
$!
\++9!
$!
$!
$!
U466!
$!
$!
$!
W+*/01!
$!
$!
$!
J59!
$!
$!
$!
T+0<43/01!
$!
$!
%!
a*43,*5,/01!
$!
$!
$!
Q00+-/01!
$!
$!
$!
!
I0! 1707*5,/01! =+8S!95,5>! R7!978/979!,+!437!,;*77! 86533*++=3!"377! N5:67!#'(! T6533*++=! %! R53! 8*75,79!
R/,;!566!m758,!256473!37,!,+!$!/0!+*97*!,+!3/=465,7!5!86533*++=!R;7*7!0+07!+<!-+4*!.77*3!;53!.*+2/979!5!
*758,/+0!,+! ,;7!5*,/867(!a+*!T6533*++=!#>! R7!*509+=6-!8;+37!04=:7*3!*501/01!<*+=!%!,+!@!<+*!</27!m758,!
R+*93!,+!3/=465,7!5!3/,45,/+0!R;7*7!3+=7!.77*3!;527!.*+2/979!5!*758,/+0!,+!,;/3!5*,/867(!a+*!T6533*++=!
?>!R7!8;+37!*509+=!04=:7*3!:7,R770!%!509!#@!<+*!0/07!m758,!R+*93!,+!3/=465,7!5!86533*++=!R;7*7!566!
+<!-+4*!86533=5,73!;527!.*+2/979!5!*758,/+0!,+!50!5*,/867(!
W-!*7<7**/01!,+! N5:673! %!509! #>!R7! 850! 8*75,7!0/07! .+,70,/56!8+=:/05,/+03!+<!*73.+0373(! a+*!7E5=.67>!
J,4970,!?D!8+09/,/+0!#!R+469!8*75,7!,;7!<+66+R/01!95,5!37,>!53!/6643,*5,79!/0!N5:67!?(!
!
!
!
"#$%&'(!)*+,+,-./01!2/3456!675*0/01!5056-,/83!14/979!:-!50!79485,/+056!,;7+*-!/0<+*=79!1+56(!!"#$%&'(")(*+&$%,%-(.%&'/0,12>!3"?'>!%%@A%B#(!
;,,.CDD9E(9+/(+*1D%$(%F&$FDG65(#$%&(??(H!
IJJK!%L#LMHH@$!"+06/07'(!N;7!O+4*056!+<!P75*0/01!Q056-,/83!R+*S3!4097*!5!T*75,/27!T+==+03!P/87037>!Q,,*/:4,/+0!M!K+0T+==7*8/56MK+U7*/23!?($!V0.+*,79!"TT!WXMKTMKU!?($'!
!
4<:(
Table&2.&Mock&Data&for&Three&Classrooms!
!
T6533*++=!%!
T6533*++=!#!
T6533*++=!?!
Z0151/01!
$!
$!
%!
I0,7*73,/01!
$!
$!
$!
T;566701/01!
$!
%!
%#!
T4*/+43!
$!
$!
@!
T56=/01!
$!
@!
$!
\++9!
$!
$!
?!
U466!
$!
?!
&!
W+*/01!
$!
$!
$!
J59!
$!
$!
#@!
T+0<43/01!
$!
%!
#$!
a*43,*5,/01!
$!
$!
%F!
Q00+-/01!
$!
$!
B!
!
Table&3.&Mock&Data&for&the&Two&States&of&React&(your&response&and&others
&response)!
J,4970,!?!
T6533*++=!#!
N+,56!m758,/+03!
%!
$!
%!
$!
$!
$!
$!
%!
%!
%!
$!
%!
$!
@!
@!
$!
$!
$!
$!
?!
?!
$!
$!
$!
$!
$!
$!
%!
%!
#!
$!
$!
$!
$!
$!
$!
8 IMPLEMENTATION&
K+R!,;5,!R7!;527!5!8675*!79485,/+056!1+56>!5!.5.7*!.*+,+,-.7>!509!3+=7!=+8S!95,5>!R7!8;78S!/0!515/0!
R/,;!+4*!,5*17,!437*3!<+*!=+*7!/03/1;,!/0,+!3+=7!.5*,/8465*!/=.67=70,5,/+0!c473,/+03!5*+409!8;5*,!,-.7>!
385673!509!5E/3>!8+6+4*>!/0,7*58,/27!973/10>!509!588733/:/6/,-(!N;737!85,71+*/73!/6643,*5,7!R+*S79!7E5=.673!
+<!5!3.78/</8!8537!/=.67=70,5,/+0!:4,!R7!:76/727!,;5,!,;7!85,71+*/73!5*7!437<46!<+*!5!25*/7,-!+<!8+0,7E,3(!
Z58;!378,/+0!R/66!:71/0!R/,;!,;7!S7-!c473,/+0!509!,;70!/6643,*5,7!;+R!R7!503R7*79!,;5,!c473,/+0!<+*!,;7!
R+*S79!7E5=.67(!
!
"#$%&'(!)*+,+,-./01!2/3456!675*0/01!5056-,/83!14/979!:-!50!79485,/+056!,;7+*-!/0<+*=79!1+56(!!"#$%&'(")(*+&$%,%-(.%&'/0,12>!3"?'>!%%@A%B#(!
;,,.CDD9E(9+/(+*1D%$(%F&$FDG65(#$%&(??(H!
IJJK!%L#LMHH@$!"+06/07'(!N;7!O+4*056!+<!P75*0/01!Q056-,/83!R+*S3!4097*!5!T*75,/27!T+==+03!P/87037>!Q,,*/:4,/+0!M!K+0T+==7*8/56MK+U7*/23!?($!V0.+*,79!"TT!WXMKTMKU!?($'!
!
4<;(
8.1 Research the Chart Type
Key&Question:& `;5,! 5*7! ,;7! S0+R0!3,*701,;3!509!R75S073373!,;5,! +,;7*3!;527! /970,/</79!5:+4,! -+4*!
8;5*,!,-.7!+*!3/=/65*!8;5*,!,-.73i!
I0!+4*! .*+,+,-.7>!R7!978/979!,+!:4/69!<*+=!5!R/09*+37!8;5*,!,-.7(!P7+07!P765007!/3!8+==+06-!8*79/,79!
R/,;!/0,*+948/01!,;7!R/09*+37!9/51*5=!/0!%FB?!,;+41;!/,!R53!</*3,!/0,*+94879!:-!\47**-!/0!%F#L>!509!5!
*765,79!8;5*,!,-.7>!/0,*+94879!:-!a6+*7087!K/1;,/01567!/0!%F@F>!/3!856679! ,;7!.+65*! 5*75! 8;5*,!"a*/7096->!
#$$F'(!K/1;,/01567f3!8;5*,!R53!3/=/65*!:+,;!/0!973/10!509!1+56>!53!K/1;,/01567!7E.65/079!,;5,!,;7!9/51*5=!
3;+469!d5<<78,!,;*+f!,;7!7-73!R;5,!R7!<5/6!,+!8+027-!,+!,;7!.4:6/8!,;*+41;!,;7/*!R+*9M.*++<!75*3g!"Z;-97!
j!J707,5>!#$$%'(!I0!5!*7870,!J1"%"9,20(5*,/867!"#$$H'>!K/1;,/01567f3!.+65*!5*75!8;5*,!R53!05=79!+07!+<!
,;7! ,;*77! :73,! 8;5*,3! +<! 566! ,/=7(! N4<,7! "#$$%'! 341173,3! ,;5,! <+*! 3=5667*! 95,5! 37,3>! ,5:673! R+469!
+4,.7*<+*=!,;737!1*5.;/83!509!d,;7!+06-!R+*37!973/10!,;50!5!./7!8;5*,!/3!3727*56!+<!,;7=(g!a+*!,;7!8537!
3,49-! +<! m758,>!R7!8+=.5*7! ,;7! /=.67=70,5,/+0! ,+! 5!=+*7! ,*59/,/+056!95,5! 2/3456/_5,/+0(! J495! "#$%$'!
8*/,/8/_79!8;5*,3!6/S7!,;7!R/09*+37!:785437! /,!/3!;5*9!,+!9*5R!50!/=51/05*-!5*8!+*!8/*867!R;70!8+=.5*/01!
256473!R;7*753! /,!/3!753-!,+!8+=.5*7!256473!/0!5! :5*!8;5*,!R/,;!50!/=51/05*-!6/07(!`;70!2/3456!675*0/01!
5056-,/83!97.5*,!<*+=!3,5095*9/_79!8;5*,!,-.73!509!,;+37!8;5*,3!;527!S0+R0!R75S073373>!/,!/3!/=.+*,50,!
,+!4097*3,509!,;7!.+,70,/56!3,*701,;!+<!,;7!978/3/+0!:5379!+0!,;7!ZNI\!3+!,;5,!,;7!,75=!850!97,7*=/07!
/<!,;7!:707</,3!+4,R7/1;!,;7!8+3,3(!
I0! ,;/3! 8537! 3,49->! /,! /3! .+33/:67! ,;5,! ,;7! 8+3,! +<! =5S/01! .*78/37!/0,7*.*7,5,/+03! +<! ,;7! 8;5*,! 58,4566-!
34..+*,3!,;7!ZNI\(!Q!*7870,!81,+%1+(K+@2!5*,/867!.+/0,3!+4,!,;5,!R;70!43/01!,;7!3c45*7!*++,!+<!,;7!25647!
<+*! ,;7! *59/43! /0! +*97*! ,+! =5/0,5/0! ,;7! 5 ..*+.*/5,7! *765,/+03;/.! :7,R770! 5*75! 509! 25647! ,;*+41;! ,;7!
38567>!/,!563+!97M7=.;53/_73!,;7!8+0,*53,!:7,R770!,;7!3=566!5*753!509!,;7!65*17!+073!"m7;=7-7*>!#$$F'(!
`;/67! ,;/3! .+/0,! /3! /970,/</79! 53! 5! R75S0733! <+*! K/1;,/01567f3! +*/1/056! 437! +<! ,;7! 8;5*,>! 53! /,!8+469!
.+,70,/566-!9/=/0/3;!,;7!/=.58,!+<!65*17*!04=:7*3!"m7;=7-7*>!#$$Fb!`/667*3>!#$%%b!P7R/>!#$$&'>!,;7!97M
7=.;53/3! /0! 50+,;7*! 8+0,7E,! 8+469! :7! 8+03/97*79! 5! .+,70,/56! 3,*701,;! 1/270!,;5,! /,! /3! ,;7+*7,/8566-!
.+33/:67! ,+! 437! 5*75! 708+9/01! ,+! .4*.+376-! 97M7=.;53/_7! 65*17*! 95,5! "`/667*3>! #$%%'(! I0! ,;7! 8537! +<!
m758,>!,;/3! 8+469! 34..+*,! ,;7! 1+56! ,;5,! ,;7! 3,4970,! 3;+469! dC&E+( 0C+( +AB+$,+%1+( 0C&0( 0C+,$( "B,%,"%(
9&00+$2(&%>(,2(D+,%-(1"#%0+>Hg!
N;7!+*/1/056! 9/51*5=!:-!K/1;,/01567!3;+R79!,R+!-75*3!+<! 95,5b! :-! 43/01!,R+!3/97M:-M3/97!8;5*,3>!/,!R53!
753-! ,+! 8+=.5*7! ,;7! ,R+! -75*3! 53! R766! 53! 6++S! <+*! 3753+056! /=.58,! +0! ,;7! 95,5! "P7R/>! #$$&'(! N;/3!
341173,3! ,;5,! 5! .+,70,/56! 3,*701,;! +<! ,;/3! 8;5*,! ,-.7! =/1;,! :7! /0!43/01! 3=566! =46,/.673(! \/270! ,;/3!
/0<+*=5,/+0>!R7!850!7E5=/07!,;7!973/10!978/3/+03!R/,;!3+=7!599/,/+056!/0<+*=5,/+0!5:+4,!,;7!.+,70,/56!
3,*701,;3!509! R75S073373!+,;7*3! ;527! 8+03/97*79!<+*! ,;/3! 8;5*,(!`;/67!,;/3!/6643,*5,73!*7375*8;!+0!,;7!
8;5*,!,-.7!<+*!,;7!8537!3,49->!56=+3,!566!8;5*,!,-.73!;527!97:5,5:67!3,*701,;3!509!R75S073373(!J.709/01!
,/=7!17,,/01!<5=/6/5*!R/,;!,;737!5,,*/:4,73!;76.3!,+!725645,7!R;7,;7*!+*!0+,!,;7!2/3456!675*0/01!5056-,/83!
5*7!58;/72/01!,;7!ZNI\(!
!
"#$%&'(!)*+,+,-./01!2/3456!675*0/01!5056-,/83!14/979!:-!50!79485,/+056!,;7+*-!/0<+*=79!1+56(!!"#$%&'(")(*+&$%,%-(.%&'/0,12>!3"?'>!%%@A%B#(!
;,,.CDD9E(9+/(+*1D%$(%F&$FDG65(#$%&(??(H!
IJJK!%L#LMHH@$!"+06/07'(!N;7!O+4*056!+<!P75*0/01!Q056-,/83!R+*S3!4097*!5!T*75,/27!T+==+03!P/87037>!Q,,*/:4,/+0!M!K+0T+==7*8/56MK+U7*/23!?($!V0.+*,79!"TT!WXMKTMKU!?($'!
!
43=(
8.2 Scale and Axis
Key&Question:!`;5,!38567!8675*6-!8+==40/85,73!,;7!1+56!+<!-+4*!2/3456/_5,/+0i
`;70!73,5:6/3;/01!,;7!385673!509!5E/3!+<!5!1*5.;>!,;7*7!/3!14/95087!+0!;+R!,+!8+==40/85,7!/0<+*=5,/+0!
8675*6-(!`;/67!R7!S0+R!,;5,!,;7! 37678,/+0! +<!385673! 509!5E/3! 3;5.73!/0<+*=5,/+0! "J,72703>! %LB&b! ]509>!
%LL&b!h7667=50!j!`/6S/03+0>!%LL?'>! R7!=43,!8+03/97*! ,;7! 8+0,7E,!4097*!R;/8;! ,;7! 95,5!5*7!8+6678,79!
509! 437! ,;7! 34:3,50,/27! 38/70,/</8! S0+R67917! 525/65:67! ,+! 37678,! 5..*+.*/5,7! 385673! <+*! +4*! 8;5*,3!
"`/6S/03+0>!#$$@'(!`;5,!9+73!/,!=750!R;70!675*0/01!38/7087!/3!R;5,!/0<+*=3!,;7!37678,/+0!+<!38567i!Q09!
R;5,!*+67!3;+469!c450,/,5,/27!9/3.65-3!.65-!/0!7=+,/+056!=7534*73i!
`;70!:4/69/01!5!<408,/+056!.*+,+,-.7>!R7!;59!,+!978/97!;+R! ,+! 533/10! 256473!,+!,;7! 8;5*,(! \/270!,;5,!
,;7!527*517!86533!3/_7!/0!#$$HA$F!R53!#$($!.4./63!<+*!.4:6/8!767=70,5*-!38;++63!"J0-97*!j!U/66+R>!#$%?'!
:-!;52/01! ,;7! 4..7*!6/=/,!+<! ,;7!*5017! +<! 256473!5,! 5*+409! B$>! ,;7! 973/10! R+469! ;527!5=.67! *++=! ,+!
9/3.65-! *758,/+03! 5..*+.*/5,76-(!I0! +*97*! ,+! 588+==+95,7! ,;+37! 4..7*! 6/=/,3>! R7! 4379! 5! a/:+0588/!
37c47087! +<! ,R7627! 04=:7*3! %>! %>! #>! ?>! @>! F>! %?>! #%>! ?B>! @@>! FL>! %BB! <+*! ,;7! *73.78,/27! *758,/+03C!
70151/01>! /0,7*73,/01>!8;566701/01>! 84*/+43>! 856=/01>! 1++9>! 9466>! :+*/01>!359>! 8+0<43/01>! <*43,*5,/01>! 509!
500+-/01(!`7!1707*5,79!5!6/075*!38567!"377!a/14*7!H'!,;5,!R+469!566+R!43!,+!437!W+*/01!5,!#%>!J59!5,!?B>!
509!T+0<43/01!5,!@@!,+!15417!,;7!4..7*!6/=/,!+<!*73.+0373!R;/67!Z0151/01!5,!%!509!I0,7*73,/01!5,!%!R+469!
*7.*7370,!R;5,! /,! R+469!6++S! 6/S7! ,+! 5!3,4970,!2/7R/01! +06-!,;7/*!+R0! *758,/+03(!I0!8+0078,/01! :58S!,+!
+4*!79485,/+056! ,;7+*-! /0<+*=79!1+56>! R7! 0+,79! ,;5,!,;7!3/0167!*73.+037!/0! 8+=.5*/3+0! ,+! ,;7!4..7*!
:+409!6++S79!/03/10/</850,(!N;/3!9/9!0+,!35,/3<-!,;7!65,,7*!;56<!+<!,;7!1+56!,;5,!,;7!3,4970,!3;+469(LC&E+(
0C+(+AB+$,+%1+(0C&0(0C+,$("B,%,"%(9&00+$2(&%>(,2(D+,%-(1"#%0+>HM!
&
&
&
&
&
&
&
&
&
Figure&7.&Linear&scale&for&React.!
!
"#$%&'(!)*+,+,-./01!2/3456!675*0/01!5056-,/83!14/979!:-!50!79485,/+056!,;7+*-!/0<+*=79!1+56(!!"#$%&'(")(*+&$%,%-(.%&'/0,12>!3"?'>!%%@A%B#(!
;,,.CDD9E(9+/(+*1D%$(%F&$FDG65(#$%&(??(H!
IJJK!%L#LMHH@$!"+06/07'(!N;7!O+4*056!+<!P75*0/01!Q056-,/83!R+*S3!4097*!5!T*75,/27!T+==+03!P/87037>!Q,,*/:4,/+0!M!K+0T+==7*8/56MK+U7*/23!?($!V0.+*,79!"TT!WXMKTMKU!?($'!
!
434(
Q<,7*!1707*5,/01!5!<408,/+056!.*+,+,-.7>!/,!:785=7!+:2/+43!,;5,!3+=7!,*503<+*=5,/+0!+<!,;7!95,5!R+469!
:7! 0787335*-(!`7! 53S79! ,;7! 973/107*! ,+! 972/37! 5! 1*5.;/856! 27*3/+0! ,;5,! 3;+R79! *+41;6-! ;+R! +07!
*758,/+0! R+469! 6++S! R;70! 727*-! R+*9! R53!37678,79! 509! R;7*7!@>! #$>!509! B$!*758,/+03! R+469! 6++S!:-!
+4,6/0/01!,;7! 3/_7! +<!,;7! 8/*867! R/,;!1*7-! +4,6/073(!N;7! .4*.+37! +<! ,;/3! /6643,*5,/+0!R53! ,+! 97=+03,*5,7!
2/34566-!,+!,;7!709!437*!,;5,!,;7/*!+./0/+0!=5,,7*3!509!/3!:7/01!8+40,79(!N;7!*7346,/01!3S7,8;!/0!a/14*7!F!
R53!4379!53!5!14/97!/0!8*75,/01!50!5..*+E/=5,7!38567(!`7!37678,79!B$!53!,;7!=5E/=4=!53!+4*!1+56!R53!
,+!3;+R!+06-!,;7!*758,/+03!+<!+,;7*!3,4970,3!/0!,;7!35=7!86533*++=(!N;7!1707*56!37037!<*+=!,;7!973/10!
97.5*,=70,! R53! ,;5,! #$! R+469! G43,! ,+48;! ,;7! 65:763! R;/67! B$! R+469! 7E,709! :7075,;! ,;7! 65:763(!Y07!
*73.+037!6++S79!3+=7R;5,!/0<65,79!509!</27!*73.+0373!6++S79!,+!:7!5:+4,!,R/87!,;7!*59/43!+<!+07(!
Figure&8.&Target&scale&for&React.!
a/14*7!F!563+!+4,6/073!;+R!R7!978/979!,+!,*75,!,;7!5E/3!+<!,;7!8;5*,(!`7!.65879!5!=/043!3/10!+0!,;7!67<,>!5!
.643! 3/10! +0! ,;7! */1;,!5!,/697! +0! ,;7! :+,,+=>! 509! 5! 6/1;,0/01! :+6,! +0! ,;7! ,+.(!N;737! 3-=:+63! R7*7!
/0,70979!,+!8+027-!6+R!707*1-!+0!,;7!:+,,+=>!;/1;!707*1-!+0!,;7!,+.>!0715,/27!+0!,;7!67<,>!509!.+3/,/27!
+0!,;7!*/1;,(!`;/67!,;7!8+0870,*/8!8/*8673!R7*7!4379!,+!;76.!73,5:6/3;!,;7!38567>!,;7-!R7*7!0+,!/0,70979!
,+!:7!/0864979!/0!,;7!</056!+4,8+=7(!
N;7!1+563! +<! ,;7!973/10!3S7,8;! R7*7! ,+!=5S7! 34*7! ,;5,!,;7!/09/2/9456! *758,/+0! 3;+469! ,5S7!4.! 70+41;!
38*770!*756! 73,5,7!,+!/09/85,7!,+!,;7!437*! ,;5,! ,;7/*!*758,/+03!5*7!/=.+*,50,(!`7!R50,79!,+!8+0,/047!,+!
!
"#$%&'(!)*+,+,-./01!2/3456!675*0/01!5056-,/83!14/979!:-!50!79485,/+056!,;7+*-!/0<+*=79!1+56(!!"#$%&'(")(*+&$%,%-(.%&'/0,12>!3"?'>!%%@A%B#(!
;,,.CDD9E(9+/(+*1D%$(%F&$FDG65(#$%&(??(H!
IJJK!%L#LMHH@$!"+06/07'(!N;7!O+4*056!+<!P75*0/01!Q056-,/83!R+*S3!4097*!5!T*75,/27!T+==+03!P/87037>!Q,,*/:4,/+0!M!K+0T+==7*8/56MK+U7*/23!?($!V0.+*,79!"TT!WXMKTMKU!?($'!
!
43<(
8+==40/85,7! ,;5,! 758;! *758,/+0! R53! 5! .5*,! +<! ,;7! +27*566! *758,/+03! <*+=!86533=5,73(! W5379! +0! ,;7!
973/10!3S7,8;>!R7!7E.6+*79!5!25*/7,-!+<!,*503<+*=5,/+03!+0!,;7!6/075*!38567(!V3/01!,;7!35=7!%#!04=:7*3!
53!,;7!/0/,/56!6/075*!38567>!R7!<408,/+0566-!.*+,+,-.79!,;*77!,*503<+*=5,/+03!+0!,;7!6/075*!38567C!3c45*/01!
,;7!25647>!17,,/01!,;7!3c45*7!*++,!+<!256473>!509!,;7!6+1!,*503<+*=5,/+0!"377!a/14*7!L'(!
Figure&9.&Three&options&of&transformations.!
W-!*7<7*708/01!W+*/01>!J59>!509!T+0<43/01!53!,;7!.+,70,/56!65*173,!256473>!,;7!6+1!38567!6++S79!,;7!=+3,!
.*+=/3/01! 509! 86+373,! ,+! ,;7! /0,70979! 973/10! +0! ,;7! 4..7*! 709! +<! ,;7! 38567! R;/67! ,;7! 3c45*7! *++,!
5..75*79! 86+37*! ,+! ,;7! /0,70979! 973/10! +0! ,;7! 6+R7*! 709! +<! ,;7! 38567(!N;7! =5G+*! <65R! R/,;! ,;7! P+1!
,*503<+*=5,/+0! +0!,;7! 6/075*!38567! R53!,;5,! ,;7!3/0167! *73.+037! R53! 0+,! 2/3/:67>!3+! R7!978/979! ,+!377!
R;5,!/,!R+469! 6++S! 6/S7! /<! R7!37,!,;7! =/0/=4=! 25647! :5379!+0!3+=7! <*58,/+0!+<! ,;7! ,+,56! 1*5.;/8! *756!
73,5,7(!`;/67!,;7!3c45*7!*++,!,*503<+*=5,/+0!.*+94879!,;7!=+3,!=5,;7=5,/8566-!5884*5,7!9/3.65-!+<!,;7!
95,5!37,>!R7! R50,79!,+! 7E.6+*7!+4*! +.,/+03!+0! ;+R!R7! 8+469!3;5.7! ,;7!9/3.65-! ,+! .*+2/97! ,;7! ,5*17,!
7E.7*/7087!973/*79!:-!,;7!973/10!,75=(!
Q15/0>!R7!:4/6,!5!<408,/+056!.*+,+,-.7!+<!R;5,!m758,!R+469!6++S!6/S7!R/,;!727*-!*758,/+0!;52/01!5!25647!
+<! %! 43/01! 5! <*58,/+0! +<! ,;7! 1*5.;/8! *756! 73,5,7(!a/14*7! %$! 3;+R3! ,;7! 40/,! 8/*867! 5,! ,;*77! +.,/+03C! 50!
7/1;,;>!5!c45*,7*>!509!;56<!+<!,;7!1*5.;/8!*756!73,5,7!+<!m758,(!
Figure&10.&Three&options&to&illustrate&radius&of&”Your&Reactions.”!
!
"#$%&'(!)*+,+,-./01!2/3456!675*0/01!5056-,/83!14/979!:-!50!79485,/+056!,;7+*-!/0<+*=79!1+56(!!"#$%&'(")(*+&$%,%-(.%&'/0,12>!3"?'>!%%@A%B#(!
;,,.CDD9E(9+/(+*1D%$(%F&$FDG65(#$%&(??(H!
IJJK!%L#LMHH@$!"+06/07'(!N;7!O+4*056!+<!P75*0/01!Q056-,/83!R+*S3!4097*!5!T*75,/27!T+==+03!P/87037>!Q,,*/:4,/+0!M!K+0T+==7*8/56MK+U7*/23!?($!V0.+*,79!"TT!WXMKTMKU!?($'!
!
433(
`;70!9/38433/01!R;5,!,+!437>!R7!R50,79!,+!37678,!5!*7.*7370,5,/+0!+<!50!/09/2/9456!*758,/+0!,;5,!6++S79!
=750/01<46>! R7! 978/979! ,;5,! ,;7! c45*,7*! +.,/+0! R+469! =5S7! 34*7! ,;5,! ,;7! /09/2/9456!9/9! 0+,! <776!
=5*1/056/_79! :-! ,;7! .+,70,/56! 38567!+<! 95,5(!N;7! *5,/+0567! :7;/09! 9+/01! 3+! R53! ,;5,>! 1/270! +4*! ,5*17,!
.+.465,/+0! +<! 3,*4116/01! 3,4970,3>! 50! 7E.7*/7087! ,;5,! <76,! 6/S7! ,;7/*! 8+0,*/:4,/+0! R53! /=.+*,50,! 509!
=750/01<46!R53!8*/,/856(!
`;70!:*/01/01!,;7!=/0/=4=!25647!+<!5!c45*,7*!,+17,;7*!R/,;!,;7!6+1!,*503<+*=5,/+0!+0!,;7!6/075*!38567>!
R7! .*+94879! R;5,! 6++S79! 6/S7! 5! 35,/3<58,+*-! +4,8+=7! 43/01! ,;7! %>%>#>?>@>F>%?>#%>?B>@@>FL>%BB!
37c47087>!53!/6643,*5,79!/0!a/14*7!%%(!
!
Figure&11.&Log&with&minimum&value&of&a&quarter.!
N;/3!=597!,;7!3/0167!*73.+037!5!86+37*!5..*+E/=5,/+0!+<!,;7!,5*17,!/6643,*5,79!/0!a/14*7!%%!509!6/079!,;7!
,5*17,! 256473! 86+37*! ,+! ,;7! 1+56(! N;/3!8675*! 8+0,*53,! +0! 37678,/+0! +<! 38567!/3! 5! 97.5*,4*7! <*+=! 8675*6-!
8+==40/85,/01! ,;7! 95,5>! R;/8;! *5/373! ,;7! 8+087*0! +<! 6/=/,/01! 9/38+27*-! :-! ,;7! 437*! "h7667=50! j!
`/6S/03+0>!%LL?'(! ]+R727*>!/,! R53! /0<+*=79! :-! 976/:7*5,7!8;+/87! 509>! =+*7!/=.+*,50,6->! *7.*7370,3! 5!
,;7+*-!,;5,!R7! 5*7!,73,/01!/0!5,,7=.,/01!,+!8+027-!3+=7,;/01!,;5,!R+469!1/27!,;7!437*!5!1707*56!37037!
+<! ,;7! 7=+,/+056! 6509385.7! +<! *758,/+03!.7+.67! ;527! ;59! ,+R5*9! ,;7! *759/01! R/,;+4,! =5*1/056/_/01! 5!
3/0167!2+/87(!I0!7<<78,>! R7! 5*7!377S/01!,+! ,5S7! ,;7! R75S0733! +<! 97M7=.;53/_/01! ,;7! 9/<<7*7087! :7,R770!
,;7!3=566!5*753!509!,;7!65*17*!5*753!509!.43;/01!,;5,!97M7=.;53/3!<4*,;7*!/0!+*97*!,+!.*7270,!,;7!37037!
+<! 5! 3/0167! 2+/87! :7/01! =5*1/056/_79! /0! ,;7! 8+0,7E,! +<! 95,5! <+*! ,;7! 70,/*7! 86533*++=(! \4/979! :-! +4*!
!
"#$%&'(!)*+,+,-./01!2/3456!675*0/01!5056-,/83!14/979!:-!50!79485,/+056!,;7+*-!/0<+*=79!1+56(!!"#$%&'(")(*+&$%,%-(.%&'/0,12>!3"?'>!%%@A%B#(!
;,,.CDD9E(9+/(+*1D%$(%F&$FDG65(#$%&(??(H!
IJJK!%L#LMHH@$!"+06/07'(!N;7!O+4*056!+<!P75*0/01!Q056-,/83!R+*S3!4097*!5!T*75,/27!T+==+03!P/87037>!Q,,*/:4,/+0!M!K+0T+==7*8/56MK+U7*/23!?($!V0.+*,79!"TT!WXMKTMKU!?($'!
!
43I(
79485,/+056! ,;7+*-! /0<+*=79! 1+563>! ,;/3!/6643,*5,73! ;+R! 348;! 1+563! 850! 8++*9/05,7! 50! /0,7*9/38/.6/05*-!
,75=(!
8.3 Interactive Design
Key&Question:!U+73!,;7!/0,7*58,/27!973/10!34..+*,!,;7!1+56!+<!-+4*!2/3456/_5,/+0i
Q<,7*!73,5:6/3;/01!+4*!38567!509!5E/3>!R7!8+469!437!+4*!=+8S!95,5!37,!+<!,;*77!437*3!509!,;*77!5*,/8673!,+!
1707*5,7! 0/07!9/<<7*70,! 27*3/+03! +<! ,;7! 8;5*,! ,+! 17,! 5! 1707*56! 37037! +<! R;5,! ,;/3! R+469! 6++S! 6/S7!
*756/3,/8566-!:5379!+0!+4*!=+8S!95,5!37,(!I0!599/,/+0>!1/270!,;5,!+07!+<!,;7!3,*701,;3!+<!,;/3!8;5*,!,-.7!/3!
,;5,!3+=7! 8+03/97*!/,! 1++9!<+*! 3=566!=46,/.673>! a/14*7! %#! /6643,*5,73!0/07!3=566! =46,/.673!:5379! +0!+4*!
=+8S!95,5!37,(!
!
!
!
!
!
!
!
!
!
!
!
Figure&12.&Small&multiples&of&React&from&mock&data.!
N;7!.4*.+37!+<!43/01!,;7!=+8S!95,5!37,!,+!1707*5,7!3+=7!3=566!=46,/.673!/3!,;5,!/,! 1/273!-+4!5!37037!+<!
R;5,!3,4970,3!R/66!7E.7*/7087(!J=566!=46,/.673!566+R!-+4!,+!377!;+R!-+4*!2/3456/_5,/+0!6++S3!/0!=46,/.67!
95,5!37,3!3/97!:-!3/97(!N;/3!8675*6-!97./8,3!,;7!95,5!3.78/</8566->!:4,!=+*7+27*!3;+R3!=50-!.+33/:67!2/7R3!
+<!,;7!2/3456/_5,/+0!6++S!6/S7!3/97!:-!3/97(!I0!,;7!3=566!=46,/.67!<+*=>!/,!/3!8675*!,;5,!,;7*7!/3!0+!1*5.;/856!
847! /09/85,/01! ,;5,! -+4! 5*7! 2/7R/01! 50! 56,7*05,7! 9/3.65-(!`;/67! -+4! 8500+,! 377! ,;7! R+*93! 8;+/873! +<!
m758,!;7*7! 8675*6->! -+4! 850! 377! ,;5,! 0+! R+*93! 5*7! 37678,79! <+*! J,4970,! %>! +07! R+*9! /3! 37678,79! <+*!
!
"#$%&'(!)*+,+,-./01!2/3456!675*0/01!5056-,/83!14/979!:-!50!79485,/+056!,;7+*-!/0<+*=79!1+56(!!"#$%&'(")(*+&$%,%-(.%&'/0,12>!3"?'>!%%@A%B#(!
;,,.CDD9E(9+/(+*1D%$(%F&$FDG65(#$%&(??(H!
IJJK!%L#LMHH@$!"+06/07'(!N;7!O+4*056!+<!P75*0/01!Q056-,/83!R+*S3!4097*!5!T*75,/27!T+==+03!P/87037>!Q,,*/:4,/+0!M!K+0T+==7*8/56MK+U7*/23!?($!V0.+*,79!"TT!WXMKTMKU!?($'!
!
435(
J,4970,!#>!509!,;*77!R+*93!5*7!37678,79!<+*!J,4970,!?>!53!/09/85,79!:-!,;7!*79!:+*97*!5*+409!,;7!R+*9!
8;+/87>!53! R766! 53! ,;7!5*75!8;5*,!,;5,!/09/85,73! ,;5,! ,;7!437*!=597! 37678,/+0(! Q3!-+4!3850!,+!,;7! */1;,>!
-+4!850!377!R;5,!,;7!3,4970,!R+469!377!/<!,;7-!86/8S79!+0!d377!+,;7*3f!*758,/+03(g!N;7!,;*77!.+,70,/56!
+4,8+=73! 5*7! 1707*5,79! <*+=! +4*! =+8S! 95,5! <+*! T6533*++=!%>! T6533*++=! #>! 509! T6533*++=! ?(! I0! ,;5,!
8+=.5*/3+0>! -+4! 850!56*759-! 377! 3+=7!7E5=.673! R;7*7!,;7! 3,4970,! ;59!3/=/65*! 2/7R3! ,+!,;+37! +<!,;7!
+,;7*! 3,4970,3!509! R;7*7! ,;7-! ;59! 9/27*170,! 2/7R3(! N;/3! 5..75*79! ,+! 34..+*,! ,;7! 1+56! +<! 3,4970,!
*7<678,/+0>!53!5!3,4970,!8+469!8+=.5*7!,;7/*!*73.+037!,+!,;+37!+<!,;7/*!.77*3(!
I0! +*97*! ,+! 7E.6+*7! ,;7 ! 1+56! +<! 43/01! ,;7! 95,5! 2/3456/_5,/+0! ,+! ; 76.! .77*3! R;70! 7E.6+*/01! 8+0,70 ,>!R7!
7E.6+*79!;+R!/,!8+469!:7!/08+*.+*5,79!/0,+!,;7!7E.6+*7!.517!R;7*7!3,4970,3!37678,79!50!5*,/867!,+!*759(!
N;7!<+66+R/01!=+8SM4.3!"a/14*73!%?!509!%B'!;527!:770!1707*5,79!,+!.5.7*!,73,!,;7!.+,70,/56!<75,4*7!+0!
,;7!7E.6+*7!.517!+<!,+116/01!:7,R770!,;7!8+27*!5*,!+<!5*,/8673!509!,;7!m758,!2/3456/_5,/+0!+<!,;7!5*,/8673!
9/3.65-/01!86533!*758,/+03(!a/14*7!%B!/6643,*5,73!,;7!7E.6+*7!.517!R;7*7!5!3,4970,!850!37678,!3+=7,;/01!,+!
*759!<*+=!=46,/.67!5*,/8673(!!
!
!
!
!
!
!
!
!
!
!
!
!
!
!
!
!
!
!
!
Figure&13.&Explore&page.&
!
"#$%&'(!)*+,+,-./01!2/3456!675*0/01!5056-,/83!14/979!:-!50!79485,/+056!,;7+*-!/0<+*=79!1+56(!!"#$%&'(")(*+&$%,%-(.%&'/0,12>!3"?'>!%%@A%B#(!
;,,.CDD9E(9+/(+*1D%$(%F&$FDG65(#$%&(??(H!
IJJK!%L#LMHH@$!"+06/07'(!N;7!O+4*056!+<!P75*0/01!Q056-,/83!R+*S3!4097*!5!T*75,/27!T+==+03!P/87037>!Q,,*/:4,/+0!M!K+0T+==7*8/56MK+U7*/23!?($!V0.+*,79!"TT!WXMKTMKU!?($'!
!
436(
a/14*7!%B! /3! 5! 97./8,/+0!+<!R;5,!/,!=/1;,!6++S!6/S7!+0!,;7! 7E.6+*7! .517! /<!5!3,4970,!8+469!377!,;7!6/3,!+<!
5*,/8673!R;7*7!,;7!8+27*! 5*,! /3! *7.65879!R/,;!m758,!95,5!2/3456/_5,/+0!,;5,!3;+R3!,;7! *758,/+03! +<! ,;7/*!
86533=5,73!,+!,;7!*759/01(!
Figure&14.&Explore&page&with&React&view.!
`;70!,5S/01!,;/3!+4,!<+*!<4*,;7*!<+*=5,/27!725645,/+0!R7!850!8;78S!R/,;!+4*!437*!.+.465,/+0!,+!377!/<!
,;7!<75,4*7!R+469!34..+*,!3,4970,3!/0!37678,/01!50!5*,/867!,+!*759(!Q,!,;/3!.;537!+<!,;7!R+*S>!,;/3!<75,4*7!
;53!0+,!:770!,73,79!:7-+09!,;7!1707*5,/+0!+<!,;7!.5.7*!.*+,+,-.7(!
8.4 Colour
Key&Question:!]+R!9+73!-+4*!8;+/87!+<!8+6+4*!34..+*,!-+4*!1+56i!
!
"#$%&'(!)*+,+,-./01!2/3456!675*0/01!5056-,/83!14/979!:-!50!79485,/+056!,;7+*-!/0<+*=79!1+56(!!"#$%&'(")(*+&$%,%-(.%&'/0,12>!3"?'>!%%@A%B#(!
;,,.CDD9E(9+/(+*1D%$(%F&$FDG65(#$%&(??(H!
IJJK!%L#LMHH@$!"+06/07'(!N;7!O+4*056!+<!P75*0/01!Q056-,/83!R+*S3!4097*!5!T*75,/27!T+==+03!P/87037>!Q,,*/:4,/+0!M!K+0T+==7*8/56MK+U7*/23!?($!V0.+*,79!"TT!WXMKTMKU!?($'!
!
437(
I0!75*6-!973/103>!R7!;59!,73,79!5!*5/0:+R!+<!8+6+4*3!509!5!=+0+8;*+=5,/8!,*75,=70,!R;7*7!3/_7!R53!,;7!
+06-! /09/85,+*(! `;/67! ,;737! 75*6/7*! 973/103!R7*7! 4097*3,5095:67>! 599/01! /0! ,;7! 9/=703/+03! +<! 8+6+4*!
,;7+*-! 509! 35,4*5,/+0! =597! ,;7! 2/34 56! 9/3.65-!=+*7!=750/01<46(!`7! *7/0<+*879! ,;7!*765,/27! 7=+,/+056!
707*1/73!+<!,;7!R+*93!:-! 8;++3/01! 5! 3.78,*4=! +<!35,4*5,/+03!+<!8+6+4*!/0!8+0G408,/+0!R/,;!,;7!*765,/27!
/0,703/,/73!+<!,;7!5E73(!
8.5 Accessibility
Key&Question:&]+R!9+73!,;7!588733/:/6/,-!+<!-+4*!95,5!2/3456/_5,/+0!865*/<-!+*!8;5667017!-+4*!1+56i
8.5.1 Keyboard Accessibility
Q88733/:/6/,-! R53! 7E5=/079! 43/01! \++167! T;*+=7! 27*3/+0! B$($(##%B(%%@! "&BM:/,'(!a4*,;7*! ,73,/01! /3!
077979! 58*+33! :*+R37*3(! [7-:+5*9! 588733/:/6/,-! R53! 58;/7279! :-! =5S/01! 34*7! ,;5,! ,;7! U?! O525J8*/.,!
/=.67=70,5,/+0!+<!,;7!:4,,+03!,;5,!566+R79!,;7!437*!,+!37678,!R+*93!<+*!,;7/*!*758,/+0!R+469!*73.+09!,+!
:+,;!N+/>"@%!509!1',1N!7270,!;50967*3!3+!,;5,!:+,;!,;7!S7-:+5*9!509!,;7!=+437!8+469!/0,7*58,!R/,;!,;7!
95,5!2/3456/_5,/+0(!N;7!Jh\!6/0S3! R7*7! 4379!<+*!,;7! :4,,+03! 3+!,;5,! ,;7-! R+469!:7! /0864979!/0! ,;7!,5:!
+*97*(![7-:+5*9!588733/:/6/,-!9/9!0+,!865*/<-!+*!8;5667017!+4*!1+56(!
8.5.2 Colour-blindness
N;7! 1+56! /3! <+*! 727*-+07! ,+! :7! 5:67! ,+! 4097*3,509! 509! 588733! ,;7! 2/3456/_5,/+0(!I0! 1707*56>! /,! /3! 1++9!
.*58,/87!0+,!,+!;527!8+6+4*!53!,;7!+06-!/09/85,+*!+<!,;7!/0<+*=5,/+0(!N;7!*7346,3!=43,!:7!978/.;7*5:67!<+*!
,;+37! R/,;! 8+6+4*M:6/090733(! J/=465,/+0! ,++63! 566+R! -+4! ,+! 8;78S! <+*! 8+== +0! 8+6+4*M:6/09!8+09/,/+03!
348;! 53! )*+,+0+./5! 509! U74,7*50+./5(!);+,+3;+.! +<<7*3! 5! 3/=465,/+0! ,++6! ,;*+41;! ,;7! .*++<! 8+6+4*!
<75,4*7(!N;7!*5,/+0567!5,!,;/3!3,517!/3!,;5,!R7!R+469!6/S7! ,+!,73,!,;7!%#! 8+6+4*3!R7!;527!509!377!/<!,;7-!
;76.!34..+*,!,;+37!R/,;!8+6+4*!2/3/+0!,+!/0,7*.*7,!,;7!7=+,/+056!85,71+*/73!+<! R+*93(!a/14*7! %@! 3;+R3!
R;5,!m758,!6++S3!6/S7!/0!,;+37!,R+!3/=465,/+03(!
Figure&15.&React&in&Photoshop&colour-blindness&simulation.!
N;737!3/=465,/+03!/6643,*5,79! /0! a/14*7!%@!5*7!8675*6-!0+!6+017*! /0! %#!9/3,/08,!8+6+4*3(!N;/3!9/9!8;5667017!
+4*!4097*3,509/01!+<!,;7!*+67!+<!8+6+4*!/0!,;7!95,5!2/3456/_5,/+0!53!+4,6/079!/0!378,/+0!F(B(!`;/67!8+6+4*!
509! 35,4*5,/+0! R7*7! /0,70979! ,+! ;76.! 8+027-! ,;7! /0,703/,/73! +<! ,;7! 5E/3>! ,;7-! R+469! 0+,! 0787335*/6-!
!
"#$%&'(!)*+,+,-./01!2/3456!675*0/01!5056-,/83!14/979!:-!50!79485,/+056!,;7+*-!/0<+*=79!1+56(!!"#$%&'(")(*+&$%,%-(.%&'/0,12>!3"?'>!%%@A%B#(!
;,,.CDD9E(9+/(+*1D%$(%F&$FDG65(#$%&(??(H!
IJJK!%L#LMHH@$!"+06/07'(!N;7!O+4*056!+<!P75*0/01!Q056-,/83!R+*S3!4097*!5!T*75,/27!T+==+03!P/87037>!Q,,*/:4,/+0!M!K+0T+==7*8/56MK+U7*/23!?($!V0.+*,79!"TT!WXMKTMKU!?($'!
!
43:(
8+027-! ,;7! 35=7! /9753! ,+! ,;+37! R/,;! 8+6+4*! :6/090733(! `;70! 978/9/01! R;7,;7*! +*! 0+,! R7! 077979! ,+!
8;5017!,;7! 8+6+4*!37678,/+0!R7!7678,79!,+!S77.! ,;7!/0/,/56!37678,/+0(!\/270!,;5,>!/0!378,/+0!F(#>!,;7!5E73!
R7*7!/0,70,/+0566-!65:76679!,+!8+027-!,;7!35=7!/0<+*=5,/+0!,;/3!R53!34<</8/70,!,*75,=70,!,+!34..+*,!,;7!
1+56!+<!;52/01!5!40/27*356!=78;50/3=!+<!376<M*7.+*,/01(!N;7!56,7*05,/27!978/3/+0!R+469!;527!:770!,+!437!
,;7! 8+6+4*M:6/09! :5**/7*M<*77! .567,,7(! YS5:7!509![7/!"#$$#'!341173,!43/01!,;7!<+66+R/01!7/1;,!8+6+4*3C&
W658S!*1:!"$>$>$'>!Y*5017!*1:!"#?$>%@L>$'>!JS-!W647!*1:!"F&>%F$>#??'>!W64/3;!\*770!*1:!"$>%@F>%%@'>!X766+R!
*1:!"#B$>##F>&&'>!W647!*1:!"$>%%B>%HF'>!h7*=/66/+0!*1:!"#%B>LB>$'>!509!!m799/3;!)4*.67!*1:!"#$B>%#%>%&H'(!
N;/3!3/=465,/+0!3;+R3!R;-!=46,/.67!*7.*7370,5,/+03!5*7!8*/,/856(!
8.5.3 Screen Reader
Q88733/:/6/,-! R53! 7E5=/079! 43/01! \++167! T;*+=7! 27*3/+0! B$($(##%B(%%@! "&BM:/,'! 509! l58! h+/87Y27*!
V,/6/,-! 27*3/+0!&($(! I0! ,;7! /=.67=70,5,/+0>! ,;7! :4,,+03! R7*7! 59G43,79! :-! 8;501/01! ,; 7! 86533! 5,,*/:4,7>!
R;/8;!;/1;6/1;,79!,;7!:4,,+03!/0!*79!R;70!,;7-!R7*7!37678,79(!Q0!7c4/25670,!7E.7*/7087!<+*!,;7!38*770!
*7597*!R53!58;/7279!,;*+41;!QmIQ!3,5095*93!:-! /09/85,/01!,;5,!758;!R53! 5!,+1167!:4,,+0!509! 4.95,/01!
,;7! 5,,*/:4,73! 3+! ,;5,! ,;7! 38*770! *7597*! 8+469! ,766!/<! ,;7! :4,,+0! R53! 37678,79(! `;/67! R7!;59! ;+.73! +<!
/0,71*5,/01!QmIQMP/27!,513!/0,+!,;/3!/=.67=70,5,/+0!/0!+4*!/0/,/56!702/*+0=70,>!R7!R7*7!405:67!,+!17,!,;7!
:4,,+03!,+!6727*517!,;7!QmIQMT+0,*+6!<75,4*7!3+!,;5,!,;7!38*770!*7597*!9/9!0+,!6/27!4.95,7!,;7!,*503/,/+0!
,+!,;7!J77!Y,;7*3f!m73.+037!,+1167(!Q!37.5*5,7!.*+.+356!R53!.4,!,+17,;7*!,+!/0273,/15,7!588733/:67!U?!
95,5!,*503/,/+03(!
N;7!=+3,!/0,7*73,/01!53.78,!+<!7E.6+*/01!;+R!,+!=5S7!50!7c4/25670,!7E.7*/7087!R/,;!,;7!38*770!*7597*!
*765,79!,+!+4*! /=.67=70,5,/+0!5*+409!,;7!1*5.;/8!767=70,3!+<! ,;7!2/3456/_5,/+0(!`7!73,5:6/3;79!5!0+0M
3,5095*9! 38567! 509! 8;+37! ,+! +=/,! 04=7*/8! 65:766/01! +0 ! ,;7! 5E/3(!I,! R53! 9/<</846,! ,+! 97,7*=/07! R; 5,! 50!
7c4/25670,! 7E.7*/7087! R+469! :7! <*+=! ,;7! 38*770! *7597*! 53! ,;/3! 8;5*,! *76/79! +0! ,;7! 2/7R7*f3!
/0,7*.*7,5,/+0! R/,;+4,! .*+2/9/01! 7E.6/8/,! 04=7*/8! /09/85,+*3(!N;7! 753/73,! 56,7*05,/27! ,7E,! ,+! .*+2/97!
R+469! :7! ,;7! 8+**73.+09/01! 04=7*/8! 256473! <+*! 758;! *758,/+0! +.,/+0>! :4,! ,;5,! 377=79! =+*7! 6/S7! 50!
7c4/25670,!7E.7*/7087!+<!5!:5*!8;5*,!9/3.65-/01!,;7!/0<+*=5,/+0>!0+,!,;/3!R/09*+37(!Q6,7*05,/273!/0864979!
,;7!+.,/+0!+<!9738*/:/01!*73.+0373!/0!85,71+*/856!R5-3!348;!53!d3+=7!.7+.67!*758,79!Z0151/01!509!5!6+,!
+<!.7+.67!*758,79! W+*/01>g!:4,! ,;/3! R53! 65-7*/01!/0,7*.*7,5,/+0!+0,+!,;7! 8;5*,!3/087! ,;7! 38*770! *7597*!
R+469!0+,! 566+R!,;7=!,+!9*5R!,;7/*!+R0!8+08643/+03>!53!,;7-!=/1;,!0+,!437!,;7!.;*5373!d3+=7g!+*! d5!
6+,g! ,+! 9738*/:7! ,;7! 95,5! 2/3456/_5,/+0(!V6,/=5,76->! R7! 978/979! ,+! 3,5*,! R/,;! 04=7*/8! 256473! <+*!
56,7*05,/27!,7E,!:785437!R7!R7*7!0+,!-7,!34*7!/<!+4*!5E/3!R53!58;/72/01!,;7!/0,70979!1+56(!N;7!07E,!3,7.3!
R/66! :7! 8+0948,/01!3+=7! <4*,;7*! <+843! 1*+4.3! 509! ,73,/01! ,+! :7,,7*! 4097*3,509! ;+R! +4*! 437*3! 5*7!
/0,7*.*7,/01!,;7!1*5.;!,+!377!/<!5!.+33/:67!56,7*05,/27!,7E,!8+469!.*+2/97!50!7c4/25670,!7E.7*/7087(!
9 FUTURE WORK
Y07!+<!,;7!:707</,3!+<!+*150/_/01!-+4*!R+*S!5*+409!50!ZNI\!/3!,;5,!/,!850!;76.!73,5:6/3;!;+R!,+!725645,7!
,;7!978/3/+03!=597(!N;/3!R+*S79!7E5=.67!R53!37678,79!:785437!+<!,;7!<*7c470,!/0,7*58,/+03!R/,;!<+843!
1*+4.3!509!,;7!/,7*5,/27!05,4*7!+<!,;7!.4*34/,!+<!50!ZNI\b!;+R727*>!978/3/+03!5*7!3,/66!=597!R/,;+4,!437*!
<779:58S(!
!
"#$%&'(!)*+,+,-./01!2/3456!675*0/01!5056-,/83!14/979!:-!50!79485,/+056!,;7+*-!/0<+*=79!1+56(!!"#$%&'(")(*+&$%,%-(.%&'/0,12>!3"?'>!%%@A%B#(!
;,,.CDD9E(9+/(+*1D%$(%F&$FDG65(#$%&(??(H!
IJJK!%L#LMHH@$!"+06/07'(!N;7!O+4*056!+<!P75*0/01!Q056-,/83!R+*S3!4097*!5!T*75,/27!T+==+03!P/87037>!Q,,*/:4,/+0!M!K+0T+==7*8/56MK+U7*/23!?($!V0.+*,79!"TT!WXMKTMKU!?($'!
!
43;(
Y07!973/10!767=70,!,;5,!*7c4/*73!<4*,;7*!725645,/+0!/3!+4*!8;+/87!+<!38567!509!5E/3(!a/*3,>!R7!0779!,+!377!
/<!/,!1707*5,79! ,;7!7<<78,!R7!R7*7!377S/01(!\/270!,;5,! +4*! ,5*17,! 437*!/3!,;7!3,*4116/01! *7597*!R;+! ;53!
+<,70!=7,!R/,;!<5/64*7!/0!,;7!86533*++=!56*759->!R7!;+.79!+4*!973/10!R+469!/08*7537!.5*,/8/.5,/+0!R/,;!
m758,!:-!=5S/01!34*7!,;5,!,;7/*!*73.+0373!9/9!0+,!5..75*!,+!;527!=5*1/056!25647(!`7!R50,79!,+!/0<65,7!
,;7!25647!+<!50!/09/2/9456!2/34566-!/0!,;7!;+.7!,;5,!437*3!R+469!.*+2/97!,;7/*!*758,/+03!+27*!,;7!8+4*37!+<!
,;7/*!43517!+<!V9/+(!N;/3!=7503!,;5,!R7!R+469!R50,!,+!377!:+,;!5!;/1;!67276!+<!437!509!343,5/079!43517!
+27*!,/=7(!Y07! +<!,;7!</*3,!8+=.5*/3+03!R7!8+469!=5S7!R/,;+4,!<4*,;7*!97276+.=70,!/3!;+R!,;7!m758,!
<75,4*7!6++S3!/0!8+=.5*/3+0!,+!+,;7*!34..+*,3!/0!,;7!3-3,7=(!]+R727*>!,;5,!R+469!:7!/0<6470879!:-!,;7!
0+276,-! +<! ,;7! 95,5! 2/3456/_5,/+0(!I0! <4,4*7! ./6+,3>! /,! R+469! :7! /0,7*73,/01! ,+! ,*-! 50! QAW! ,73,! 3/=.6-!
59G43,/01!,;7!38567!+<!m758,(!Y07!8+=.5*/3+0!8+469!:7!,;7!3c45*7!*++,!38567!53!+..+379!,+!+4*!/0<65,79!
6+1!38567!+*!.+33/:6-!3+=7!6/075*!38567(!`7!8+469!563+!59G43,!,;7!6+R73,!25647(!a+*!/03,5087>!S77.!,;7!6+1!
38567!:4,!8+=.5*7!,;7!/=.58,!+<!37,,/01!5!3/0167!*73.+037!,+!50!7/1;,;!+<!,;7!R/9,;!+<!m758,!53!+..+379!
,+!,;7! 84**70,! 37,,/01! +<!5!c45*,7*(!`;/67!,;737!3,*5,71/73!8+469!:7!7=.6+-79!,+!377!/<! +4*!2/3456/_5,/+0!
;59!,;7!/0,70979!/=.58,>!R7!3;+469!563+!,73,!<+*!3+=7!.+,70,/56!40/0,70979!8+037c470873(!
W-!972/5,/01!<*+=!,;7!1+56!+<!8675*6-!8+==40/85,/01!,;7!95,5!37,>!R7!S0+R!,;5,!+07!.+,70,/56!./,<566!/3!/0!
.*7270,/01!437*3!<*+=!9/38+27*/01!,;7/*!+R0!/03/1;,3(!\/270!,;5,!R7!9/9!0+,!.4,!65:763!+0!,;7!5E/3!+<!,;/3!
8;5*,>!/,!R+469!:7!/0,7*73,/01!,+!S0+R!/<!3,4970,3!8+469!/0,7*.*7,!,;7!8;5*,!<+*!04=7*/8! 25647!5884*5,76-!
509! /<! ,;7-! ;527! ,;7! 973/*7! ,+! S0+R! ,;7! 3.78/</8! 04=7*/8! 256473! +<! ,;7! 8;5*,! +*! /<! ,;7! 2/3456!
*7.*7370,5,/+0! /3! 25645:67! R/,;+4,! S0+ R/01! ,;7! 3.78/</8! 04=:7*3(!a+*! ,;/3! .4*.+37>! R7!8+469! 8+0948,!
/0,7*2/7R3!R/,;! 3,4970,3!509!437! 5! 25*/7,-! +<!3,5,/8!/=5173! +<! ,;7! m758,!2/3456/_5,/+0! ,+! 377!/<!,;7-!850!
/0,7*.*7,!04=7*/8!256473!509!/<!,;7-!850!9738*/:7!c456/,5,/276-!,;7!8;5*58,7*/3,/83!+<!,;7!5*,/867!:5379!+0!
,;7!m758,!2/3456/_5,/+0(!
REFERENCES
W5**->! T(! "#$%$>! l5*8;! %$'(! )Yh! l596/:3!q`7:6+1! .+3,r(! m7,*/7279! <*+=!
;,,.3CDD938;++6(3,50<+*9(794D1*+4.3DS%#DR/S/D##7?LD)Yhsl596/:3(;,=6!
W/5085*+35>!\(>! j! J0+R>!T(!Z(!"#$$&'(!O+&>,%-(%+A0P(.(E,2,"%()"$(&10,"%( &%>( $+2+&$1C(,%(9,>>'+(&%>(C,-C(
21C""'( ',0+$&1/P( .( O+B"$0( 0"( Q&$%+-,+( Q"$B"$&0,"%( ")( K+@( R"$N! "#09!79/,/+0'(! `53;/01,+0>! UTC!
Q66/5087!<+*!ZE876670,!Z9485,/+0(!
W+R7*>!\(!"%LF%'(!l++9!509!=7=+*-(!.9+$,1&%(S2/1C"'"-,20>!36"#'>!%#LA%BF(!
W*+R0>! N(! "#$$L>! J7.,7=:7*'(! T;5017! :-! U73/10(! IUZY(! m7,*/7279! <*+=! ;,,.3CDDRRR(/97+(8+=D:-M
/97+D8;5017M:-M973/10!
W*+R0>!N(>!j! [5,_>! W(!"#$$L).& QC&%-+( D/( >+2,-%P( T"@(>+2,-%( 0C,%N,%-( 1&%(0$&%2)"$9( "$-&%,U&0,"%2( &%>(
,%2B,$+(,%%"E&0,"n(!K7R!X+*SC!]5*.7*T+66/03(!
W*+R0>! N(>! j! [5,_>! W(! "#$%%'>!T;5017! :-! 973/10(! !"#$%&'( ")( S$" >#10( V%%"E&0,"%( W&%&-+9+%0G( <:>!?F%A
?F?(!;,,.CDD9E(9+/(+*1D%$(%%%%DG(%@B$M@FF@(#$%%($$F$&(E!
W*-50>!O(>!j! W*-50>!N(! "%LF?'(! N;7! 3+8/56!6/<7! +<! ,;7!675*0/01! 9/35:679! -+4013,7*(! I0!O(! U(! l8[/007-!j! P(!
a751503!"Z93('>!Q#$$+%0(0"B,12(,%('+&$%,%-(>,2&D,',0,+2("h+6(!I'>!"..(@HAF@'(!K7R!X+*SC!Q:67E(!
!
"#$%&'(!)*+,+,-./01!2/3456!675*0/01!5056-,/83!14/979!:-!50!79485,/+056!,;7+*-!/0<+*=79!1+56(!!"#$%&'(")(*+&$%,%-(.%&'/0,12>!3"?'>!%%@A%B#(!
;,,.CDD9E(9+/(+*1D%$(%F&$FDG65(#$%&(??(H!
IJJK!%L#LMHH@$!"+06/07'(!N;7!O+4*056!+<!P75*0/01!Q056-,/83!R+*S3!4097*!5!T*75,/27!T+==+03!P/87037>!Q,,*/:4,/+0!M!K+0T+==7*8/56MK+U7*/23!?($!V0.+*,79!"TT!WXMKTMKU!?($'!
!
4I=(
TTQQP! "T5*071/7! T+408/6! +0! Q92508/01! Q9+673870,! P/,7*58-'(! "#$%$'(! X,9+( 0"( &10P( .%( &-+%>&( )"$(
&>E&%1,%-(&>"'+21+%0(',0+$&1/()"$(1"''+-+(&%>(1&$++$(2#11+22H!K7R!X+*SC!T5*071/7!T+*.+*5,/+0!+<!
K7R!X+*S(!
T+66/03>!Q(>! W*+R0>! O(!J(>!j!K7R=50>! J(! "%LFL'(! T+10/,/27! 5..*70,/873;/.C!N758;/01!,;7! 8*5<,! +<! *759/01>!
R*/,/01>!509!=5,;7=5,/83(!P(!m730/8S!"Z9('>!Y%"@,%-G('+&$%,%-(&%>(,%20$#10,"%P(J22&/2(,%(C"%"$(")(
O"D+$0(Z'&2+$("..(!B@?ABLB'(!]/6639567>!KOC!Z*6:54=(!
J1"%"9,20(!"#$$H>!U787=:7*!%L'(!`+*,;!5!,;+43509!R+*93C!Q!1++9!1*5.;/8!850!,766!5!3,+*->!:*/01!5!64=.!
,+!,;7! ,;*+5,>! 7270!8;5017! .+6/8/73(!]7*7! 5*7! ,;*77!+<! ;/3,+*-f3! :73,(!XC+(J1"%"9,20(! m7,*/7279!
<*+=!;,,.CDDRRR(78+0+=/3,(8+=D0+97D%$#HF&B?!
Z9=+093>!l(! J(>! h541;0>! J(>!`7E67*>!O(>!m74,7:48;>!T(>!T5:67>! Q(>! N58S7,,>! [(! [(>! j!J8;05S70:7*1>!O(!`(!
"#$$L'(!Q!3-0,;73/3!+<!*759/01!/0,7*270,/+03!509!7<<78,3!+0!*759/01!8+=.*7;703/+0!+4,8+=73!+0!
+697*! 3,*4116/01! *7597*3(! O+E,+@( ")( J>#1&0,"%&'( O+2+&$1C>! 7;"%'>! #&#A#FH(!
;,,.CDD9E(9+/(+*1D%$(?%$#D$$?B&@B?$F?#@LLF!
Z;-97>!T(!T(>!j!J707,5>!Z(!"#$$%'(!80&0,20,1,&%2(")(0C+(1+%0#$,+2(!K7R!X+*SC!J.*/017*!
Z/8;>!Z(>! [/;63,*+=>! O(>!W+R7*>!\(>!a+*153>!O(>!j!K/7970,;56>!)(! "#$$$'(! Q"-%,0,"%( &%>( +9"0,"%(!K7R! X+*SC!
YE<+*9!V0/27*3/,-!)*733(!
Z250>! W(! "#$%%>! l5-! %H'(! d&MFM@(g! Z&9+20"$9,%-(!m7,*/7279! <*+=! RRR(15=73,+*=/01(8+=D15=73M<+*M
<*73;M,;/0S/01M509M/9753D&MFM@3!
a/38;7*>! [(! `(>! j! W/9766>! N(! m(! "%LLF'(! U-05=/8! 97276+.=70,! +<! .3-8;+6+1/856! 3,*48,4*73! /0! 58,/+0! 509!
,;+41;,(!I0!m(!l(!P7*07*!"Z9('>!T&%>D""N(")(1C,'>(B2/1C"'"-/H(["'(4P(XC+"$+0,1&'(9">+'2(")(C#9&%(
>+E+'"B9+%0("@,;!Z9(b!..(!B&HA@&%'(!K7R!X+*SC!`/67-(!
a*/7096->! l(! "#$$F'(! Q! :*/7<! ;/3,+*-! +<! 95,5! 2/3456/_5,/+0(! I0! T(! T;70>! `(! ]t*967>! j!Q(! V0R/0!"Z93('>!
T&%>D""N(")(1"9B#0&0,"%&'(20&0,20,12P(\&0&(E,2#&',U&0,"%(VVV("..(!%A?B'(!]7/976:7*1C!J.*/017*(!
a*/G95>!K(!"%LF&'(!XC+(+9"0,"%2(!T5=:*/917>!V[C!T5=:*/917!V0/27*3/,-!)*733(!
a*->!W(!"#$$B'(!Q"9B#0&0,"%&'(,%)"$9&0,"%(>+2,-%(!);U!9/337*,5,/+0>!lIN>!T5=:*/917>!lQ(!
\5G*/5>!l(>! O/,709*5>!Q(! [(>! J++9>! J(>! j! J58S3>!\(! "#$$H'(! I=.*+2/01! 8+=.*7;703/+0!+<!7E.+3/,+*-!,7E,!/0!
3,4970,3! R/,;! PUC! Q! *7375*8;! 3-0,;73/3(! !"#$%&'( ")( *+&$%,%-( \,2&D,',0,+2>!I="?'>! #%$A##@(!
;,,.CDD9E(9+/(+*1D%$(%%HHD$$###%LB$H$B$$$?$?$%!
]509>! U(! O(! "%LL&'(! J,5,/3,/83! 509! ,;7! ,;7+*-! +<! =7534*7=70,(! O+4*056! +<! ,;7! m+-56! J,5,/3,/856! J+8/7,->!
J7*/73!Q>!%@L>!BB@ABL#(!
P5_5*43>!m(!"%LL%'(!J9"0,"%(&%>(&>&B0&0,"%(!K7R!X+*SC!YE<+*9!V0/27*3/,-!)*733(!
P7354E>! K(! [>! j! J/7176>! P(! J(! "#$$?'(! N;7! 97276+.=70,! +<! *759/01! /0! 8;/69*70! R;+! 3.75S! Z016/3;! 53! 5!
378+09! 65014517! "ZJP'(! \+E+'"B9+%0&'( S2/1C"'"-/G( 3;"&'>! %$$@A%$%L(!
;,,.CDD9E(9+/(+*1D%$(%$?HD$$%#M%&BL(?L(&(%$$@!
P7R/>! )(! O(! "#$$&'(! a6+*7087! K/1;,/01567! 509! .+65*! 5*75! 9/51*5=3(! J.75S/01! +<! \*5.;/83C! Q0! Z335-! +0!
\*5.;/858-! /0! J8/7087>! N78;0+6+1-! 509! W43/0733(!m7,*/7279! <*+=!
;,,.CDDRRR(95,538+.7(:7D3+1(;,=!
l551>!O(!`(>!j!W7;*703>!O(!N(!"%LFL'(!U7.*733/+0!509!8+10/,/27!376<M3,5,7=70,3!+<!675*0/01!9/35:679!509!
37*/+436-! 7=+,/+0566-! 9/3,4*:79! 59+673870,3(!XC+( !"#$%&'( ")( 8B+1,&'( J>#1&0,"%>!<3"%'>! %HA#H(!
;,,.CDD9E(9+/(+*1D%$(%%HHD$$##B&&LFL$#?$$%$?!
!
"#$%&'(!)*+,+,-./01!2/3456!675*0/01!5056-,/83!14/979!:-!50!79485,/+056!,;7+*-!/0<+*=79!1+56(!!"#$%&'(")(*+&$%,%-(.%&'/0,12>!3"?'>!%%@A%B#(!
;,,.CDD9E(9+/(+*1D%$(%F&$FDG65(#$%&(??(H!
IJJK!%L#LMHH@$!"+06/07'(!N;7!O+4*056!+<!P75*0/01!Q056-,/83!R+*S3!4097*!5!T*75,/27!T+==+03!P/87037>!Q,,*/:4,/+0!M!K+0T+==7*8/56MK+U7*/23!?($!V0.+*,79!"TT!WXMKTMKU!?($'!
!
4I4(
l5*156/,>! l(>! j! m52/2>! Q(! "%LFB'(! PU3f! 7E.*733/+03! +<! 50E/7,-! /0! ,7*=3! +<! =/0+*! 3+=5,/8!
8+=.65/0,3(!!"#$%&'( ")( *+&$%,%-( \,2&D,',0,+2>!47"B'>! ##&A##F(!
;,,.CDD9E(9+/(+*1D%$(%%HHD$$###%LBFB$%H$$B$F!
l7-7*>! Q(>! m+37>! U(! ](>! j! \+*9+0>! U(! "#$%?'(! ]%,E+$2&'( >+2,-%( )"$( '+&$%,%-P( XC+"$/( &%>( B$&10,1+(!
`5S7</769>!lQC!K5,/+056!T70,7*!+0!V0/27*356!U73/10!<+*!P75*0/01(!
l4*0507>!m(!O(>!j!P72->!a(!"%LL&'(!X+&1C,%-(0C+(%+@(D&2,1(2N,''2P(S$,%1,B'+2()"$(+>#1&0,%-(1C,'>$+%(0"(0C$,E+(
,%(&(1C&%-,%-(+1"%"9/(!K7R!X+*SC!a*77!)*733(!
K.JS& (V(J(! U7.5*,=70,! +<! Z9485,/+0,!K5,/+056! T70,7*! <+*! Z9485,/+0! J,5,/3,/83'.&m7,*/7279&from&
;,,.CDDRRR(05,/+03*7.+*,85*9(1+2D*759/01s=5,;s#$%?D</673Dm7346,3sQ..709/Esm759/01(.9<(
K/76370>!O(! "#$%#>! O407! B'(! ]+R! =50-! ,73,!437*3! /0!5! 435:/6/,-! 3,49-i! K+*=50!K/76370!\*+4.(!m7,*/7279!
<*+=!!RRR(001*+4.(8+=D5*,/8673D;+RM=50-M,73,M437*3!
K/72770>! K(>! j! a+6=7*>! Z(! "#$%?'(! a+*=5,/27! 725645,/+0! /0! 79485,/+056! 973/10! *7375*8;(! J>#1&0,"%&'(
\+2,-%(O+2+&$1C>! ..(! %@#A%&L(! K7,;7*65093! I03,/,4,7! <+*! T4**/8464=! U7276+.=70,(! m7,*/7279!
<*+=!;,,.CDD9+R06+593(36+(06DU+84=70,70D79485,/+056M973/10M*7375*8;M.5*,M5(.9<!
KOTPU! "K5,/+056! O+/0,! T+==/,,77! <+*! P75*0/01! U/35:/6/,/73'(! "#$$F'(! J.78/</8! 675*0/01! 9/35:/6/,/73C! Q!
05,/+056!*72/7R(!K7R!X+*SC!K5,/+056!O+/0,!T+==/,,77!<+*!P75*0/01!U/35:/6/,/73(!
KPNJ!II!"K5,/+056!P+01/,49/056!N*503/,/+0!J,49-!II'(!"#$$?'(!K5,/+056!T70,7*!<+*!J.78/56!Z9485,/+0!m7375*8;!
5,!,;7!I03,/,4,7!+<!Z9485,/+0!J8/70873(!`53;/01,+0>!UTC!V(J(!U7.5*,=70,!+<!Z9485,/+0(!
YS5:7>!l(>!j![7/>!I(!"#$$#'(!Q"'"$(#%,E+$2&'(>+2,-%(^Q]\_P!T"@(0"(9&N+(),-#$+2(&%>(B$+2+%0&0,"%2(0C&0(&$+(
)$,+%>'/(0"(1"'"$D',%>(B+"B'+(!q`7:!.517r!m7,*/7279!<*+=!;,,.CDDG<6-(/5=(4M,+S-+(58(G.D8+6+*!
)56/0835*>!Q(!J(!"%LF&'(!l7,58+10/,/27!3,*5,71-!/03,*48,/+0(!JA1+B0,"%&'(QC,'>$+%>!53>!%%FA%#B(!
)56/0835*>! Q(! J(! "%LLF'(! J+8/56! 8+03,*48,/2/3,! .7*3.78,/273! +0! ,758;/01! 509! 675*0/01(! .%%#&'( O+E,+@( ")(
S2/1C"'"-/G(I;>!?B@A?H@(!
)56/0835*>!Q(>! j! W*+R0>! Q(!"%LFB'(!m78/.*+856!,758;/01!+<!8+=.*7;703/+0M<+3,7*/01!509!8+=.*7;703/+0M
=+0/,+*/01!58,/2/,/73(!Q"-%,0,"%(&%>(V%20$#10,"%>!4>!%%HA%H@(!
)7*<7,,/>! T(! Q(>! P509/>! K(>! j! Y5S;/66>! O(! "#$$@'(! N;7! 58c4/3/,/+0! +<! *759/01! 8+=.*7;703/+0! 3S/66(! I0! l(! O(!
J0+R6/01! j! T(! ]46=7! "Z93('>! XC+( 21,+%1+( ")( $+&>,%-P( .( C&%>D""N!"..(! ##HA#BH'(! YE<+*9>! V[C!
W658SR766(!
m7;=7-7*>! O(! "#$$F'(! a6+*7087! K/1;,/01567C! N;7! .533/+05,7! 3,5,/3,/8/50(! 81,+%1+( K+@2H( m7,*/7279! <*+=!
;,,.3CDDRRR(38/708707R3(+*1D5*,/867D<6+*7087M0/1;,/01567M.533/+05,7M3,5,/3,/8/50!
m+37>!U(!](>!j!l7-7*>!Q(!"#$$#'(!N758;/01!727*-!3,4970,!/0!,;7!9/1/,56!517C!V0/27*356!973/10!<+*!675*0/01(!
Q67E509*/5>!hQC!Q33+8/5,/+0!<+*!J4.7*2/3/+0!509!T4**/8464=!U7276+.=70,(!
m+37>!U(!](>!l7-7*>!Q(>!j!]/,8;8+8S>!T(!"#$$@'(!XC+(#%,E+$2&''/(>+2,-%+>(1'&22$""9P(.11+22,D'+(1#$$,1#'#9(
&%>(>,-,0&'(0+1C%"'"-,+2H(T5=:*/917>!lQC!]5*25*9!Z9485,/+0!)*733(!
m433766>! O(! Q(! "%LF$'(! Q! 8/*84=.67E! =+976! +<! 5<<78,(! !"#$%&'( ")( S+$2"%&',0/( &%>( 8"1,&'( S2/1C"'"-/G( 3;>!
%%&%A%%HF(!
m433766>!O(!Q(! "#$$@'(! Z=+,/+0! /0!;4=50! 8+038/+430733! /3!:4/6,!+0! 8+*7! 5<<78,(!!"#$%&'( ")(Q"%21,"#2%+22(
80#>,+2G(4<>!#&AB#(!
m433766>!O(!Q(!"#$$L'(!Z=+,/+0>!8+*7!5<<78,!509!.3-8;+6+1/856!8+03,*48,/+0(!Q"-%,0,"%(&%>(J9"0,"%G(<3"H'>!
%#@LA%#F?(!;,,.CDD9E(9+/(+*1D%$(%$F$D$#&LLL?$L$#F$L?H@!
!
"#$%&'(!)*+,+,-./01!2/3456!675*0/01!5056-,/83!14/979!:-!50!79485,/+056!,;7+*-!/0<+*=79!1+56(!!"#$%&'(")(*+&$%,%-(.%&'/0,12>!3"?'>!%%@A%B#(!
;,,.CDD9E(9+/(+*1D%$(%F&$FDG65(#$%&(??(H!
IJJK!%L#LMHH@$!"+06/07'(!N;7!O+4*056!+<!P75*0/01!Q056-,/83!R+*S3!4097*!5!T*75,/27!T+==+03!P/87037>!Q,,*/:4,/+0!M!K+0T+==7*8/56MK+U7*/23!?($!V0.+*,79!"TT!WXMKTMKU!?($'!
!
4I<(
J75*67>!O(!m(!"%LL#'(!XC+($+>,21"E+$/(")(0C+(9,%>(!T5=:*/917>!lQC!lIN!)*733(!
J7**766>! W(! "#$$&'(! !#>-,%-( +AC,D,0,"%2P( .( )$&9+@"$N( )"$( &22+22,%-( +A1+''+%1+(! `5604,! T*77S>! TQC! P7<,!
T+53,!)*733(!
J/97*/9/3>!\(! U(>! l+4_5S/>! Q(>! J/=+3>! )(>! j! )*+,+.5.53>! Q(! "#$$&'(!T6533/</85,/+0!+<!3,4970,3!R/,;!*759/01!
8+=.*7;703/+0! 9/<</846,/73C! N;7! *+673! +<! =+,/25,/+0>! 5<<78,>! 509! .3-8;+.5,;+6+1-(!*+&$%,%-(
\,2&D,',0/(`#&$0+$'/>!<;"?'!%@LA%F$(!;,,.CDD9E(9+/(+*1D%$(#?$HD?$$?@@$@!
J0-97*>!N(!U(>!j!U/66+R>!J(!Q(!"#$%?'(!U/173,!+<!Z9485,/+0!J,5,/3,/83!#$%#!"KTZJ!#$%BM$%@'(!K5,/+056!T70,7*!
<+*! Z9485,/+0! J,5,/3,/83>! I03,/,4,7! +<! Z9485,/+0! J8/70873>! `53;/01,+0>! UTC! V(J(! U7.5*,=70,! +<!
Z9485,/+0(!
J,7/96>! J(>! l+;/MV99/0>! J(>! j! Q097*3+0>! Q(! "#$$&'(! Z<<78,3! +<! 7=+,/+056! 5*+4356! +0! =46,/.67! =7=+*-!
3-3,7=3C! Z2/97087! <*+=! 97865*5,/27! 509! .*+8794*56! 675*0/01(! *+&$%,%-( a( W+9"$/>! 43"@'>! &@$A
&@F(!;,,.CDD9E(9+/(+*1D%$(%%$%D6=(?#BB$&!
J,72703>!J(!J(!"%LB&'(!Y0!,;7!,;7+*-!+<!385673!+<!=7534*7=70,(!81,+%1+G(4=3>!&HHA&F$(!
J495>!W(!"#$%$'(!.(B$&10,1&'(-#,>+(0"(>+2,-%,%-(@,0C(>&0&(!P+09+0C!a/27!J/=.67!J,7.3(!
J4,;7*6509>![(!J(>!j!J/01;>!K(!K(!"#$$B'(!P75*079!;76.67330733!509!3,4970,3!R/,;!7=+,/+056!+*!:7;52/+*56!
9/3+*97*3C!U7.*/25,/+0!/0!,;7!86533*++=(!b+C&E,"$&'(\,2"$>+$2>!<;"#'>!%&LA%F%(!
N4<,7>!Z(!m(!"#$$%'(!XC+(E,2#&'(>,2B'&/(")(c#&%0,0&0,E+(,%)"$9&0,"%>!#09!79(!T;73;/*7>!V[C!\*5.;/83!)*733(!
h7667=50>!)(!a(>!j!`/6S/03+0>!)(!a(!"%LL?'(!K+=/056>!+*9/056>!/0,7*256>!509!*5,/+!,-.+6+1/73!5*7!=/36759/01(!
.9+$,1&%(80&0,20,1,&%G(I7>!&@AH#(!
d&-%+$G( WHG( W&$>+$G( QHG( b'&1N"$D/G( !HG( Q&9+0"G( OHG( K+@9&%G( *HG( *+E,%+G( SHG( e( \&E,+2?W+$1+$G( JH(
"#$$?'(!XC+(&1C,+E+9+%02(")( /"#0C( @,0C(>,2&D,',0,+2(>#$,%-(2+1"%>&$/( 21C""'P( .($+B"$0( )$"9( 0C+(
K&0,"%&'( *"%-,0#>,%&'( X$&%2,0,"%( 80#>/?<( ^K*X8<).!l706+! )5*S>!TQC! JmI! I0,7*05,/+056(! m7,*/7279!
<*+=!;,,.CDDRRR(06,3#(+*1D*7.+*,3D#$$?s%%D06,3#s*7.+*,s#$$?s%%s8+=.67,7(.9<!
`/6S/03+0>!P(!"#$$@'(!XC+(-$&99&$(")(-$&BC,12>!#09!79(!K7R!X+*SC!J.*/017*(!
`/667*3>! W(! "#$%%'(! K4=:7*3! 509! 5*75(! m7,*/7279! <*+=!
;,,.CDDRRR(:70R/667*3(8+=D5337,3DR/667*3s04=:7*3s509s5*75(.9<!
... Consequently, these four elements form the basic structure of our review. We also made the adaptation of changing knowledge to goal to better incorporate the goal-relevant considerations of visual analysis approaches in a learning context, such as target users, target problems, and theoretical background (e.g., Hillaire et al., 2016;Vieira et al., 2018). Theoretical considerations largely inform and justify the knowledge that users desire to gain from visual analytics approaches. ...
... The development of learning tools must be guided by specific learning theories (Hillaire et al., 2016;Shaffer & Ruis, 2017;Wise & Schaffer, 2015). Therefore, we further examined whether the goal of VRCD involved any theoretical considerations. ...
... Innovative approaches have started to achieve this goal. For example, Hillaire et al. (2016) proposed a six-step model to guide the development of visual learning analytics tools. In this model, the first step is to define an educational goal informed by educational theories, and subsequent steps involve the definition of the target users, an interdisciplinary paper prototyping process, a formative evaluation, mock data, and implementation. ...
Article
Visual analytics combines automated data analysis and human intelligence through visualisation techniques to address the complexity of current real-world problems. This review uses the lens of visual analytics to examine four dimensions of visual representations of collaborative discourse: goals, data sources, visualisation designs, and analytical techniques. We found visual analysis approaches to be suitable and advantageous for decomposing the temporality of collaborative discourse. However, it has been challenging for current research to simultaneously consider learning theories and follow visualisation design principles when adopting visualisations to analyse collaborative discourse. At the same time, existing visual analysis approaches have mainly targeted learners or researchers and mainly focused on mirroring collaborative discourse rather than providing advanced affordances such as alerting or advising. Informed by these findings, we propose a possible future research agenda and offer suggestions for the features of successful collaboration to guide the design of advanced affordances.
... In recent years, visual learning analytics has become an important technical field to support the online learning process [8]. Visual learning analytics emphasizes the guiding and regulating role of learning theory in data exploration, and analysis rather than guiding and learning based on simple personal experience [9]. Visual learning analytics can gain insight into students' learning process, help students understand learning progress, and promote self-reflection [10,11]. ...
Article
Full-text available
Online Judge (OJ) is an important aid for programming learning that can help students evaluate learning effects in real-time, while teachers can adjust practice tasks in time according to the records of the tool. With these advantages, OJ shows great value for promoting teaching and learning in programming. The existing OJ system usually only provides information such as problem status list and recent rank list. However, it is unable to provide teachers with more fine-grained analysis information, such as the distribution of students’ incorrect responses and level of knowledge mastery. And it also cannot provide students with effective comparative information on their learning status. This research developed a visual learning analytics dashboard named VisOJ for the OJ system, which includes two types of user interfaces: teacher and student. The teacher interface presents students' learning status and ranking trends, which help teachers monitor and give feedback on their learning activities. The student interface provides views such as error type analysis and evaluation, which promote students' self-reflection and self-regulation. Preliminary case studies and expert interviews prove the usability of the dashboard. In the end, we summarize our main work and suggest future research directions.
... The digital literacy platform used in the current study was built on the principles of UDL, with a particular focus on providing multiple ways for students to engage with text. Called Udio, the intervention was developed through a process of design-based research and targeted to middleschool students identified by their schools as in need of additional support in reading comprehension (see Hillaire, Rappolt-Schlichtmann, & Ducharme, 2016). The resulting online environment builds on prior approaches to improving reading comprehension in this population by emphasizing, as the Edmonds et al. (2009) meta-analysis describes, a focus on engaging students, "in thinking about text, learning from text, and discussing what they know" (p. ...
Article
We use the affordances of a supplemental digital literacy platform to study the dynamics of behavioral engagement and reading comprehension among middle-school students in remedial reading classes. All participating students (n = 315) were identified by their schools as needing additional reading support; 56% received special education services. They used the digital literacy platform designed using the Universal Design for Learning (UDL) framework for approximately 1 hr per week for an academic year. Embedded indicators of activity in the digital platform captured whether and how students engaged with available text-related activities. Higher levels of behavioral engagement were not associated with improved reading comprehension in the sample overall. An interaction effect indicated that students who started the year with lower reading comprehension skills were likely to benefit more from higher reading-related behaviors and more use of text-to-speech than those starting with relatively higher comprehension skills.
... With the trend of shifting beneficiary, more types of objectives will be produced because different stakeholders have different expectation on what LA can bring to them (Kwok, 2015). For example, teachers may want to improve the learning design (Hillaire et al., 2016), system designers may want to improve the learning platform (Dowell et al., 2016), parents may want to monitor their children's learning status and senior managements of institutions may want to understand the way to reduce turnover rate and drop-out rate and (Kwok, 2015). ...
Article
Learning analytics is a new research stream developed from other previous related streams such as academic analytics and educational data mining. We conducted a literature review on recent publications related to data analytics in education in various journals and conferences. With the reference model proposed by Chatti et al (2012), we proposed a refined model by introducing the second layer elements to two out of the four dimensions and propose changes to the list of options of some dimensions. We then analysed publications of the initial three years (2014–2016) in the Journal of Learning Analytics based on the refined model. We discovered five trends in learning analytics applications including extending the data landscape to non-academic data and shifting of objectives and analysis methods. Finally, we proposed four future research directions in this area and rationalised the relationship between dimensions in the model.
... When designing data visualizations for learning analytics research or practice, other practical guidelines in the literature would be worth being synthesized with the present paper in guiding practice. See Klerx, Verbet, and Duval (2017) for practical guidelines on how to get started on developing data visualizations for this purpose ;and Hillaire, Rappolt-Schlichtmann, and Ducharme (2016) for prototyping guidelines. ...
Article
Full-text available
Understanding human judgement and decision making during visual inspection of data is of both practical and theoretical interest. While visualizing data is a commonly employed mechanism to support complex cognitive processes such as inference, judgement, and decision making, the process of supporting and scaffolding cognition through effective design is less well understood. Applying insights from cognitive psychology and visualization science, this paper critically discusses the role of human factors — visual attention, perception, judgement, and decision making — toward informing methodological choices when visualizing data. The value of visualizing data is discussed in two key domains: 1) visualizing data as a means of communication; and 2) visualizing data as research methodology. The first applies cognitive science principles and research evidence to inform data visualization design for communication. The second applies data- and cognitive-science to deepen our understanding of data, of its uncertainty, and of analysis when making inferences. The evidence for human capacity limitations — attention and cognition — are discussed in the context of data visualizations to support inference-making in both domains, and are followed by recommendations. Finally, how learning analytics can further research on understanding the role data visualizations can play in supporting complex cognition is proposed.
Conference Paper
Full-text available
Distance Learning has often been side by side with the notion of Openness. The MOOCs' hype was partly due to the wide Accessibility that prestigious universities offered to a wide audience of learners who were following along the technological progress and could afford the cost and time expenses for learning sustainably. However, education should also embrace the concept of inclusiveness. That means that online learning should become more open not only in terms of admission but also in terms of student support. Additionally, learning material should be able to adjust to the diversity of students' needs, backgrounds, and skill levels. Universal Design addresses these issues and can provide general guidelines to achieve the necessary personalization. For an educational environment to address all these needs, the educational stakeholders should be aware of their students' skills and limitations. The idea of "knowing your audience" is the answer to the problems of the "one size fits all" education. In that respect, the educational design can significantly benefit from Learning Analytics (L.A.). The dynamic learning process focused on students' diversity is included in the 21st-century educational policy targets. At the same time, the advances in L.A. provide new opportunities for the adaption of more flexible and effective e-learning environments. The LA approach, which measures, collects, and analyses educational data for reporting and creating actionable knowledge about learners and their contexts, offers the capability to implement real-time assessment and feedback systems and processes focused on improving knowledge, developing self-regulated learning skills, and academic success. Additional aspects of L.A. related to the analysis of students' social behavior, sentiment, and participation could reveal hidden difficulties or even problematic situations that are not always obvious to the tutors by the naked eye, allowing in this way the adoption of a learner-centered educational approach. The incorporation of Machine Learning techniques could enable the creation of digital twins, a digital replica of students' learning-related information for personalizing teaching methods, pace and content. This paper aims to discuss the role of L.A. in the instructional design process for online inclusive learning environments. Based on recent data, the study puts inclusive education on the L.A. community's agenda to identify flexible, personalized instructional design patterns focused on learners' academic needs. Furthermore, the paper discusses the opportunities and challenges which lay the foundations for designing a more dynamic and inclusive learning environment as an ambitious scientific field that showcases how design and control go hand in hand. Innovating Higher Education Conference 2022 271
Article
Productive peer talk moves have a fundamental role in structuring group discussions and promoting peer interactions. However, there is a lack of comprehensive technical support for developing young learners’ skills in using productive peer talk moves. To address this, we designed iTalk–iSee, a participatory visual learning analytical tool that supports students’ learning and their use of productive peer talk moves in dialogic collaborative problem-solving (DCPS). This paper discusses aspects of the design of iTalk–iSee, including its underlying theoretical framework, visualization, and the learner agency it affords. Informed by the theory of Bakhtinian dialogism, iTalk–iSee maps productive peer talk moves onto learning goals in DCPS. It applies well-established visualization design principles to connect with students, hold and direct their attention, and enhance their understanding. It also follows a three-step (code → visualize → reflect) macro-script to strengthen students’ agency in analyzing and interpreting their talk. This paper also discusses the progressive modifications of iTalk–iSee and evaluates its usability in a field study. We present the implications of essential design features of iTalk–iSee and the challenges of using it (relating to, for example, teacher guidance, data collection, transcription, and coding). We also provide suggestions and directions for future research.
Chapter
In this chapter, we explore inclusive learning analytics to predict reading comprehension outcomes in the context of middle school remedial reading classrooms for 235 students from 12 school districts in the United States. The design of the system used in this study followed the Universal Design for Learning Guidelines which have expressed the goal of supporting all learners, a common goal with inclusive learning analytics. We construct a predictive model for reading comprehension, introducing a novel method to apply the same sentiment analysis method to interpret both emotional self-reports and discussion comments, contributing new insights about the relationship between emotion and cognition during learning. The novel method of analysis was inspired by the inclusive design principles of Universal Design for Learning by treating the two data sources as multiple means of expression. The relationship the model suggests is that struggling students with positive emotional reactions coupled with neutral communication about reading materials are more likely to provide correct answers. The psychological phenomena of positive internal experience coupled with downregulated expression have previously been described as “smiling on the inside,” and are considered socially desirable when outperforming peers. We explore the model in terms of bias across a range of groups legally protected from discrimination. With the limitations of the measures and the model in mind, we provide recommendations on how to support students in this psychological state of “smiling on the inside”.KeywordsInclusive learning analyticsUniversal design for learningReading comprehensionMiddle school
Article
We examined the learning analytics literature over the past ten years (2011-2020). Our results show that only 33% of articles we retrieved focused on students with disabilities and 67% of articles we retrieved engaged with students with disabilities tangentially on several themes: detecting difficulties to provide early intervention, using learning analytics as tools to promote learning, addressing accessibility issues and challenges, and discussing ethics and privacy concerns. Articles that focused on students with disabilities reveal that learning analytics can be used to evaluate and optimize the design of educational technologies for students with disabilities and function as a tool to enhance learning experiences for students with various disabilities. We conclude with recommendations for future research of learning analytics community to create an inclusive platform.
Book
Full-text available
Este libro es producto del programa de Investigación en Innovación Educativa, que patrocina el Instituto Antioqueño de Investigación y que desarrolla el grupo de investigación Universus. Está estructurado por capítulos que los investigadores han ido editando a medida que progresan en el programa. Cada uno se enfoca en alguno de los aspectos que se debería incluir en una agenda de trabajo orientada a revolucionar el sistema de educación. El lector podrá darse cuenta de que el contenido se relaciona de forma incremental, partiendo desde algunas conceptualizaciones, luego se presenta recomendaciones para el cambio y se finaliza describiendo los resultados de su aplicación experimental.
Article
Full-text available
As an alternative to using the concepts of emotion, fear, anger, and the like as scientific tools, this article advocates an approach based on the concepts of core affect and psychological construction, expanding the domain of inquiry beyond "emotion". Core affect is a neurophysiological state that underlies simply feeling good or bad, drowsy or energised. Psychological construction is not one process but an umbrella term for the various processes that produce: (a) a particular emotional episode's "components" (such as facial movement, vocal tone, peripheral nervous system change, appraisal, attribution, behaviour, subjective experience, and emotion regulation); (b) associations among the components; and (c) the categorisation of the pattern of components as a specific emotion.
Chapter
Introduction: Simple Ideas about Reading Comprehension. A Framework for Comprehension. Higher-Level Factors in Comprehension. The Linguistic-Conceptual Machinery for Comprehension. Word Identification, Decoding, and Phonological Awareness .Comprehension Instruction. Conclusion: A More General View of Comprehension Development
Article
The ability to collect, store, and manage data is increasing quickly, but our ability to understand it remains constant. In an attempt to gain better understanding of data, fields such as information visualization, data mining and graphic design are employed, each solving an isolated part of the specific problem, but failing in a broader sense: there are too many unsolved problems in the visualization of complex data. As a solution, this dissertation proposes that the individual fields be brought together as part of a singular process titled Computational Information Design. This dissertation first examines the individual pedagogies of design, information, and computation with a focus on how they support one another as parts of a combined methodology for the exploration, analysis, and representation of complex data. Next, in order to make the process accessible to a wider audience, a tool is introduced to simplify the computational process for beginners, and can be used as a sketch- ing platform by more advanced users. Finally, a series of examples show how the methodology and tool can be used to address a range of data problems, in particular, the human genome.