Reflecting on the think‐aloud method for evaluating e‐learning

British Journal of Educational Technology (Impact Factor: 1.54). 11/2005; 37(1):45 - 54. DOI: 10.1111/j.1467-8535.2005.00521.x

ABSTRACT E-learning is increasingly being used in higher education settings, yet research examining how students use e-resources is frequently limited. Some previous studies have used the think-aloud method (an approach with origins in cognitive psychology) as an alternative to the more usual questionnaire or focus groups, but there is little discussion in the educational literature about the advantages and disadvantages of this approach. In this paper, we discuss our experience of using the think-aloud method in a recent study, and we reflect on its potential contribution as a research method. A number of concerns about the method arose during our study, including the level of guidance given to participants, observer influence, and the complexity of data analysis. We conclude, however, that the richness of the data collected outweighs these constraints, and that the think-aloud method has the potential to enhance research in this field.

  • Source
    [Show abstract] [Hide abstract]
    ABSTRACT: IBS affects 10-22% of the UK population. Abdominal pain, bloating and altered bowel habit affect quality of life, social functioning and time off work. Current GP treatment relies on a positive diagnosis, reassurance, lifestyle advice and drug therapies, but many suffer ongoing symptoms.A recent Cochrane review highlighted the lack of research evidence for IBS drugs. Neither GPs, nor patients have good evidence to inform prescribing decisions. However, IBS drugs are widely used: In 2005 the NHS costs were nearly £10 million for mebeverine and over £8 million for fibre-based bulking agents. CBT and self-management can be helpful, but poor availability in the NHS restricts their use. We have developed a web-based CBT self-management programme, Regul8, based on an existing evidence based self-management manual and in partnership with patients. This could increase access with minimal increased costs. The aim is to undertake a feasibility factorial RCT to assess the effectiveness of the commonly prescribed medications in UK general practice for IBS: mebeverine (anti-spasmodic) and methylcellulose (bulking-agent) and Regul8, the CBT based self-management website.135 patients aged 16 to 60 years with IBS symptoms fulfilling Rome III criteria, recruited via GP practices, will be randomised to 1 of 3 levels of the drug condition: mebeverine, methylcellulose or placebo for 6 weeks and to 1 of 3 levels of the website condition, Regul8 with a nurse telephone session and email support, Regul8 with minimal email support, or no website, thus creating 9 groups.Outcomes: Irritable bowel symptom severity scale and IBS-QOL will be measured at baseline, 6 and 12 weeks as the primary outcomes. An intention to treat analysis will be undertaken by ANCOVA for a factorial trial. This pilot will provide valuable information for a larger trial. Determining the effectiveness of commonly used drug treatments will help patients and doctors make informed treatment decisions regarding drug management of IBS symptoms, enabling better targeting of treatment. A web-based self-management CBT programme for IBS developed in partnership with patients has the potential to benefit large numbers of patients with low cost to the NHS. Assessment of the amount of email or therapist support required for the website will enable economic analysis to be undertaken.
    BMC Gastroenterology 11/2010; 10:136. · 2.11 Impact Factor
  • Source
    [Show abstract] [Hide abstract]
    ABSTRACT: BACKGROUND: Many patients with IBS suffer on-going symptoms. The evidence base is poor for IBS drugs but they are widely prescribed and advised in Guidelines. Cognitive Behavioural Therapy (CBT) can be helpful, but availability is poor in the NHS. We developed a web-based CBT self-management programme (Regul8) in partnership with patients and trialled it and common IBS medications in an exploratory factorial RCT to test trial procedures and provide information for a larger trial. METHODS: Patients, 16 to 60 years, with IBS symptoms fulfilling Rome III criteria were recruited via GP practices and randomised to over-encapsulated mebeverine, methylcellulose or placebo for 6 weeks and to 1 of 3 website conditions: Regul8 with a nurse telephone session and email support, Regul8 with minimal email support, or no website. RESULTS: 135 patients recruited from 26 GP practices. Mean IBS SSS score 241.9 (sd 87.7), IBS-QOL 64 (sd 20) at baseline. 91% follow-up at 12 weeks. Mean IBS SSS decreased by 35 points from baseline to 12 weeks. There was no significant difference in IBS SSS or IBS-QOL score between medication or website groups at 12 weeks, or in medication groups at 6 weeks, or IBS-QOL in website groups at 6 weeks. However, IBS SSS at 6 weeks was lower in the No website group than the website groups (IBS SSS no website =162.8 (95% CI 137.4-188.3), website 197.0 (172.4 - 221.7), Website + telephone support 208.0 (183.1-233.0) p = 0.037).Enablement and Subjects Global Assessment of relief (SGA) were significantly improved in the Regul8 groups compared to the non-website group at 12 weeks (Enablement = 0 in 56.8% of No website group, 18.4% website, 10.5% Website + support, p = 0.001) (SGA; 32.4% responders in No website group, 45.7% website group, 63.2% website + support group, p = 0.035) CONCLUSIONS: This exploratory study demonstrates feasibility and high follow-up rates and provides information for a larger trial. Primary outcomes (IBS SS and IBS QOL) did not reach significance at 6 or 12 weeks, apart from IBS SSS being lower in the no-website group at 6 weeks - this disappeared by 12 weeks. Improved Enablement suggests patients with access to the Regul8 website felt better able to cope with their symptoms than the non-website group. Improved SGA score in the Regul8 groups may indicate some overall improvement not captured on other measures.Trial registration: Identifier (NCT number): NCT00934973.
    BMC Gastroenterology 04/2013; 13(1):68. · 2.11 Impact Factor
  • [Show abstract] [Hide abstract]
    ABSTRACT: When learners use computers, they typically look at the screen, type, use the mouse, talk, write, sketch and make gestures. This paper identifies technical, practical, ethical and methodological challenges associated with traditional methods for studying such interactions. It examines the potential of recent technologies for identifying learners’ attention, recording real-time writing and sketching, and analyzing multiple data feeds in an integrated way. A study of learners’ interactions with multiple representations is used to illustrate the advantages and disadvantages of digital approaches to collecting, coordinating and analyzing observational data. The paper argues that there is a need for research into frameworks for analyzing digital data of learners’ computer interactions in systematic and principled ways.
    Educational Technology Research and Development 10/2012; 60(5):859-881. · 1.09 Impact Factor


Available from
May 26, 2014