Project

Young Person’s Clinical Outcomes in Routine Evaluation (YP-CORE) Scale: psychometric properties and utility

Goal: The overarching aim of this research is to provide firm psychometric evidence for the YP-CORE. I intend to replicate and extend knowledge of the psychometric properties of the YP-CORE in UK community samples. I will also look at patterns of change over time and apply modern psychometric methods, such as nearest neighbour modelling.

For more information on the YP-CORE, please go to https://www.coresystemtrust.org.uk/instruments/yp-core-information/

Updates
0 new
12
Recommendations
0 new
1
Followers
0 new
77
Reads
5 new
1399

Project log

Emily Blackshaw
added an update
I thought I would provide an update to this project log and share that since I last posted I submitted my thesis and passed my viva with minor corrections. I am delighted and feel extremely grateful to my supervisors, examiners, fellow PhD students, and of course the counselling organisations and students that contributed their data to this project.
Now that I have a bit of distance from the experience (my viva was in June), the news is starting to sink in. There are a few further bits of analysis we want to carry out on the data we collected, but the next immediate step is to publish our findings. I'm so excited to be able to contribute something really useful to the field of youth mental health. As we (hopefully!) start to push these publications out, I will make sure to update this log and make the articles available where possible.
 
Thanks Diego and Vera! It was one of the most challenging but worthwhile things I've undertaken in my life so far :) will keep you updated about papers.
 
Great to see continuing energy for a project I was involved with in the olden days! Look forward to more in due course.
 
Emily Blackshaw
added an update
My research involves two distinct, but related studies - the collection of data from non-help-seeking adolescents in schools and the analysis of pre-existing outcome data from help-seeking adolescents receiving psychotherapeutic and counselling services. As you can probably imagine, these two studies involve very different types of focus, energy and timescales, but as I have largely spoken about the school-data study so far, I wanted to draw attention to the second study with counselling organisations.
The voluntary and community sector plays an incredibly important role in meeting key challenges in mental health services for young people and often plugs crucial gaps in service provision. As the Guardian pointed out in 2015 'The role of the voluntary and community organisations in supporting mental health conditions is well established. These organisations are rooted in their communities, are trusted by the people they work with, have a long history of social action and user-led interventions, sit outside of clinical settings, and are able to offer significant and effective levels of support'. Working with such organisations in a research capacity, it has been incredibly motivating to see the dedication of these groups to supporting young people and also to consistently improving the services they deliver.
Working with counselling outcomes is always difficult and can rouse a lot of fears and anxieties for therapists, counsellors and management. However, I have found the three counselling organisations I have collaborated with to be devoted to furthering their services using evidence and research, and unafraid of taking a critical look at the outcome data. I really hope the work we are doing together will be useful for the service that they provide and the young people they support.
 
Emily Blackshaw
added an update
Data analysis - finally! As daunting as it can seem, I am delighted to have gotten my teeth into some data analysis the last couple of weeks. One thing that is making it slightly more challenging is the move from SPSS to R. Having worked exclusively in SPSS for my degree and research assistant positions, moving to R has been a big transition. It is obviously less user-friendly, but I am enjoying the extent to which it really makes me think through the analysis I am doing at a statistical level.
What are your thoughts on R?
And what resources do you draw on for help?
Mostly, when I become stuck, I simply search for a solution on Google, but I am also finding Andy Field's Discovering Statistics Using R a really helpful guide to the basics, through to more complex analysis.
 
Emily Blackshaw
added an update
Throughout this research project, I have been reading and writing about psyhometric theory and trying to clearly establish my own understanding of what the YP-CORE intends and manages to measure.
I think that most would agree that psychometrics is not a unified field of study. Perhaps it is best defined as a library of techniques, ideas and approaches to measurement. This can make writing about it difficult. I have often felt a bit bewildered upon being confronted with a wide and varied array of concepts, argument and epistemologies. Even in terms of reliability, validity and models of measurement, which most can agree are central to psychometrics, there is no clear, unified approach.
One of the areas of literature I have been most drawn towards is the 'secondary' psychometric literature. As opposed to the primary literature, with development and validation articles of specific measures, these papers comprise exploratory, critical commentary on psychometrics, often from an epistemological point of view. For example, Michell (2005, 2008) has published largely on the representational theory of measurement and the fundamental challenge this poses for the field of psychometrics. Such literature can be incredibly interesting and valuable to psychometricians, but it seems somewhat esoteric and inaccessible.
Is there an attitude in the field that there is a certain luxury to being critical without having the responsibility of developing usable measures? How can we make the philosophical/epistemological liteature on measurement more accessible so that it can be of more use to psychometricians and social scientists, in practice?
 
Emily Blackshaw
added an update
Since the last update, a great many hours have been spent tapping away at the keyboard, entering data. However, the hours of tedium now seem worthwhile, as we are getting ready to feed back some of our findings to schools.
We have a wide range of data, including whether students have ever (and weekly over the course of the study) asked for and/or received help for a mental health problem. We also have a rating of students' perceptions of their socioeconomic position, in terms of their family's position in UK society and their position in the school. We have YP-CORE, SDQ and RCADS-25 data - the SDQ at baseline and 6-month follow-up (given its 6-month span) and the YP-CORE and RCADS-25 at baseline, weekly for four more weeks and then again at 6-month follow-up. We have also asked students each week and at 6-month follow-up whether good or bad things have happened to them that week and provided space for them to list significant events that week and rate how good/bad and impactful they have been.
I am currently trying to distill this information into a format that will be useful for schools. Students considered at immediate risk of harm to themselves or others were flagged to school safeguarding so that they could receive immediate support. In some ways this felt like one of the more directly useful aspect of the research for schools, who were often, but not always aware of certain students who were struggling.
Something I am very keen to communicate back to schools is the events that students describe as being particularly good and impactful and particularly bad and impactful. These events span a range of domains, in terms of involving friends, family, hobbies, their personal health and school. I was struck, however, by how many positive and impactful events students listed as happening at school. These have included making friends on school trips, enjoying a particular lesson, speaking with a caring member of staff about losing a pet, and receiving good feedback on a report. When schools are heavily burdened with achieving academic targets and supporting student wellbeing with strained resources, they deserve, more than ever, to hear these positives from their own students.
 
Emily Blackshaw
added an update
I have reached a point in this project, where I am slightly overwhelmed with the amount of data entry that needs to be completed within a reasonable time-frame, in order to provide schools with useful feedback on the wellbeing of their students.
I am curious about how other researchers carrying out large surveys, or psychometric studies cope with this aspect of their research. We recruited large numbers of students (in one case, an entire school) to complete questionnaires once a week and then again at six month follow-up. Due to the nature of the data collection, this took place on one day, at the same time, in a classroom setting. The YP-CORE has been made into an electronic version and it would be easy to recreate this using an online survey host, such as qualtrics. However, schools generally do not have the resources to ensure that students can access one computer, or tablet each during this time and you cannot assume that all students have access to a personal mobile phone, and/or the internet. This made it impossible to carry out an internet survey, which would largely cut out the data entry process.
If you have carried out a large school-based survey, or psychometric study, I would be interested to hear how you got around these issues and how you survived data entry, if you used the paper-and-pen method, as we did.
 
Emily Blackshaw
added an update
Now that I am entering the final wave of data collection, I have turned my attention back to data analysis methods. I was a bit disappointed to find that a lot of the knowledge I had gleaned from reading I had done on psychometrics, classical test theory and item response models at the outset of my PhD was not lying dormant, awaiting a gentle shake back into usefulness, but had seemed to have disappeared.
For this reason, I wanted to outline a couple of resources that I have found particularly useful when trying to get to grips with some of these types of data analysis. Firstly an article: This paper takes you from pre-post ANOVA/ANCOVA analyses to multi-level longitudinal modelling, explaining the benefits and difficulties and carefully explaining the model building and equation formation in a way that was detailed yet clear.
On multi-level modelling, this youtube video https://www.youtube.com/watch?v=kiC71FRItFM by Nick Shyrane takes you through a thorough explanation of the key concepts involved. The University of Manchester has an incredibly helpful youtube page methodsMcr for all of their humanities research methods related videos. I also found Nick's item response theory video https://www.youtube.com/watch?v=QHIKJlcniHA very informative.
I will update this post as I come across particularly useful psychometric learning resources. If you have any favourites in the areas of IRT and longitudinal data analysis, please comment below.
 
Emily Blackshaw
added an update
It has been a while since we finished data collection in our first wave of schools. In that time, as compensation for school staff and students' time, I have been offering my time and experience. So far, this has involved giving a talk to school pastoral staff on current mental health findings and delivering an assembly to year 10 students on what it is like to study psychology at university and pursue a career in research. I am currently working with another school to plan an INSET day session around encouraging conversations in both students and staff about wellbeing and mental health.
I want to talk here about the assembly I delivered. The students there were in year 10 and it was a very hot day to be expected to sit inside, wearing uniform, listening to someone you didn't know talk about careers and research. Some of the students did look incredibly bored, but I was pleased to see others sitting up and making eye contact as I was speaking. I attended a secondary school in a mainly white, middle class suburban area and at the time, I was not aware of research as a possible career path. I was aware of the UCAS process to get in to study at universities and I suppose I knew that people must teach and work there, but I hadn't given it much thought. It makes you wonder if students from a less privileged background are made aware of research as a career option.
Research is not an easy career path. It's difficult to get into and gaining enough experience to open certain doors often involves free labour under the guise of work experience, which already makes for a very uneven playing field of opportunity. Once you're in the door, there is often a large amount of stress and possibilities of insecure employment. However, research has given me the opportunity to take a critical look at the world around me and use high quality methods to answer the questions I want to ask and I am very glad it is the career path I chose to go down.
Building mutually beneficial relationships with those people you are working with in research is something I have always thought of as important. I hope that doing assemblies like these give back something of value to the individuals who have made the research I am doing, possible.
 
Chris Evans
added a research item
Introduction: Given the increasing prevalence of mental health problems in the general population, it is indispensable to use assessment tools aimed to assess the outcome of therapeutic interventions in order to refine the process of psychological rehabilitation. Method: We describe the process of adaptation into Spanish and a first psychometric study of the Young Person's- Clinical Outcomes in Routine Evaluation (YP-CORE), an instrument designed to measure the outcome in terms of general distress of therapeutic interventions in young people (11-16 years). 104 adolescents participated in the clinical and 131 in the non-clinical samples. Results: Analyses showed good levels of acceptability, adequate internal consistency and acceptable test-retest stability, with moderately high correlations between administrations. In addition, the instrument yielded significant correlations with all dimensions of the Youth Self Report, the highest being between both total scores. Crucially, discriminated between clinical and non-clinical samples and showed a small effect of age but a larger effect of gender, with higher scores for females. The Principal Component Analysis replicates the original structure. Cut-off scores to calculate the reliable and clinically significant change are provided. Conclusions: These results support initial use of the instrument though there are certain limitations that indicate the need for more research with larger and more representative samples, in which the psychometric properties of the instrument should be verified.
Emily Blackshaw
added an update
We have recently completed the five-week period of data collection for our first two schools. These schools have just one six-month follow-up to complete.
It has been great to be in schools, speaking with students and being asked about mental health in general and the process of carrying out research. I've enjoyed building partnerships with these schools and am looking forward to delivering a few talks and assemblies as compensation for the time of the staff and students.
Another aspect of working in schools that I used to find a burden, but now quite enjoy, is the completeness of the data we are collecting. There are of course many instances where students do not complete all questionnaires in the packs, or miss out certain items. There are also students who decide to withdraw entirely, citing reasons from the research being too personal, to it being far too boring. I used to see these as nuisances in the data collection process, but I have started to see and respect it as a vital part of working with human participants. It is interesting to see students decide which questions they will and won't answer and choose whether or not they want to continue to take part. It is nice to be reminded that in psychology we have the privilege of working with fully autonomous and agentic individuals, and not 'black box' subjects to which we put a question and invariably expect a simple answer.
 
Emily Blackshaw
added an update
This weekend I had the pleasure of presenting a systematic review of the research literature on life events that occur over the course of counselling/psychotherapy for young people. For the full-text, please follow the reference below.
This review fed directly into my current research project with the YP-CORE. Whilst considering how best to explore patterns of change and stability in YP-CORE scores over time, I found myself returning again and again to the question of 'why would we expect these scores to be so stable?' and how can we account for the fluctuations in circumstances and events in our environments that influence these patterns of change? My thoughts around psychological distress are grounded in a psychosocial and public health persepctive, heavily influenced by researchers such as David Smail, Barabara Dohrenwend and Rappaport. I chose to carry out a systematic review looking into life events and socioeconomic status in studies of psychotherapeutic interventions with young people. I have since developed a brief measure of life events for adolescents, which I am currently piloting in this project.
One of my main observations from the BACP conference this weekend, was how concerned and committed many of the therapists and researchers presenting and in attendance were to improve mental wellness on a large scale, through working closely with policymakers to effect real change. I was particularly inspired by a presentation from Labour MP Luciana Berger, in which she asked the researchers present to help her to do her job. As a backbench MP with a small staff, but strong dedication to mental health, she admitted that the questions she uses to hold the goverment to account could be strenghtened with the support and guidance from mental health researchers, who know how to interpret and present data that is often available in the public sphere, but in many ways hidden from view. For me, this aspect of the conference was the one that affected me the most and I will be thinking about the ways in which my research can inform and challenge policy and contribute to a collective effort in the mental health research community to better public wellbeing.
References
 
Emily Blackshaw
added an update
We are about to start the first wave of data collection in schools, for our 'non-help-seeking' young people sample. Opt-out letters have been sent to parents and guardians in our recruited schools and in the following couple of weeks, students will be invited to consent themselves into the project.
Recently, I have been thinking about the specific challenges that schools pose in terms of conducting mental health research and trying to prepare for them in advance, as best I can. Having previously worked on a feasibility trial of a CBT intervention in schools (DISCOVER at KCL), I am familiar with some of these challenges (timetabling, room availability, avoidance of stigma), which I hope will aid this next phase of my research. If you are interested in research in schools and its feasibility, I would really recommend an article by my colleague and friend Lynn Mckeague, interviewing students and school staff during the DISCOVER project about the experiences, benefits and challenges of being involved in this kind of RCT; file available through Researchgate:
If you have had any formative experiences of research in schools and thoughts on the feasibility of such research, it would be great to hear from you.
 
Emily Blackshaw
added a research item
Background: Life events are recognised to link low socio-economic status (SES) with impaired mental health. Despite attention to patients’ historical environmental circumstances in psychotherapeutic practice, events that occur over the course of counselling and psychotherapy (‘intercurrent’ events) seem to have received little attention in research. Method: Life events were defined to include those that are chronic and severe, as well as minor, everyday occurrences. Outcomes were restricted to internalising problems related to depression and anxiety in child, or adolescent participants. Bibliographic databases and citations and review reference lists were searched, and relevant scholars were contacted. The conceptual and methodological nature of the literature is reported. Results: This review included 42 studies. Intercurrent events varied in severity and duration. Events were most frequently measured using questionnaires. The same questionnaire was rarely used in more than one study, and questionnaires were often adapted for use for the study's purpose/population. Events included in analyses tended to be analysed as a mediator of change in psychiatric symptomatology, or an outcome of therapy. Conclusions: Attention to intercurrent life events appears rare in psychotherapy research. This contributes to a systematic neglect of socio-economic issues in psychotherapy research and arguably psychotherapy more generally. This neglect is exacerbated by a lack of agreed measures of life events, both intensive and routine in nature. Recommendations are made to improve attention to such events.
Emily Blackshaw
added an update
Thank you for following this project. We (myself, Mick Cooper and Chris Evans) hope to provide you with regular updates on the progress of this research project and are keen to hear your thoughts and ideas as we go.
We are currently in the process of recruiting:
  • Schools to take part in the 'non-clinical' or community data collection
  • Clinical, or counselling organisations
We have already had initial conversations with a number of schools and organisations and are hopeful that this will result in a large and varied pool of data to allow us to conduct our longitudinal analysis. If you are from, or know of an organisation that uses the YP-CORE to collect data from young people in counselling, or therapy, please get in touch with me at blackshe@roehampton.ac.uk.
If you are interested in hearing more about this research, I will also be presenting a poster at the BACP conference on Friday 11th May 2018 at the University of Roehampton https://www.bacp.co.uk/events-and-resources/research/conference/ This will focus on my recent systematic review on life events that occur over the course of psychotherapy/counselling for young people, which we intend to look at in relation to change in scores on the YP-CORE. If you are attending and interested in hearing more about life events in therapy, or the YP-CORE project in general, please come and say hello.
Emily Blackshaw
 
Emily Blackshaw
added a project goal
The overarching aim of this research is to provide firm psychometric evidence for the YP-CORE. I intend to replicate and extend knowledge of the psychometric properties of the YP-CORE in UK community samples. I will also look at patterns of change over time and apply modern psychometric methods, such as nearest neighbour modelling.
For more information on the YP-CORE, please go to https://www.coresystemtrust.org.uk/instruments/yp-core-information/