We report the design, development and testing of a fully manipulable computer interface, designed (a) as an intervention tool to support imagination-based integration of multiple external representations (MERs), and (b) as a probe to understand the cognitive processes involved in MER integration. The interface has fully manipulable and interconnected MERs (simulation, graph and equation) representing a simple oscillator (pendulum) system. The interface design is inspired by distributed and embodied cognition approaches.
We assessed MER integration using this interface, with 7th grade students, who interacted with the interface for an hour. They then answered, as well as verbally reasoned about questions that tested MER integration. Students' actions on the interface (gaze and mouse clicks) were recorded during their interaction with the interface. These were correlated to their reasoning, to understand actions (patterns of interaction) that could have potentially led to integration.
The results provide a very nuanced view of the relationship between manipulability/interaction and integration. Good performers (on the MER integration tasks) had a high level of interaction, but with significant variations between participants, suggesting that interaction is needed for integration, but there is no single interaction pattern supporting integration. However, students exhibiting similar interaction patterns as the good performers performed badly in the integration tasks. This suggests interactivity is not sufficient for MER integration. The results indicate that interactivity can scaffold MER integration, but cannot guarantee it. Support from facilitators is needed for MER integration, as in any other complex task where novices' attention needs to be guided.