Content uploaded by Ornette D Clennon
Author content
All content in this area was uploaded by Ornette D Clennon on Jan 02, 2016
Content may be subject to copyright.
2
Contents
Introduction p.3
Methodology p.4
Findings p.6
Interim Evaluation – Generation of an Artist’s Toolkit p.6
Final Evaluation of the Programme - A participatory activity p.11
Delivery of Training days p.11
Content of Training days p.19
Management of Training days p.27
Participatory Evaluation Summary p.30
Appendix
LYOT Training day questionnaire for LYOT staff p.31
There will be an accompanying CD with the full photo and audio data
that was collected during the evaluation.
3
Introduction
The purpose of this two day training programme was to introduce artists
to working in the youth justice sector. It was intended that artists would
gain an insight into how to adapt their skills to working within this quite
specific sector. In order to assist artists, practitioners already working in
this field were invited to share their experiences alongside the shared
experiences of the professionals working in this sector. However, over
half the training was given over to exploring work in the youth justice
sector from a practitioner’s perspective by inviting a theatre company
which specialises in working in this area to share their experience and
expertise.
The purpose of this evaluation was three fold. The first part of the
evaluation took place in the form of planning consultation which
allowed the Lancashire Youth Offending Team and Prescap to clarify
their precise aims and objectives for the programme, which also
included the creation of an evaluation framework. Throughout the
discussion section of this report, I will make use of the notes collected
from this initial consultation.
The second part of this evaluation was for the generation of the Artists’
toolkit for working in this area. The discussion took the form of a
reflective conversation at the end of day one and was viewed by
myself and the organisers as an Interim evaluation of the programme.
The third part of this evaluation was a final evaluation of the entire
programme. It was important that this final evaluation was as
participatory as possible so as to elicit all the views of the participants.
4
Methodology
Interviews and field notes generated from being a ‘participant-
observer’ were used throughout the evaluation. A post training
questionnaire was also used to gain the views of the YOT workers who
delivered part of the training on day one. The responses of the YOT
workers are to be found in the appendix (they did not participate in
the evaluation process on day two). Permissions were gained from all
participants (n=18) to record their interviews and take photographs as
part of the evaluation.
Day 1
On day one, the facilitation of the artists’ toolkit acted as an interim
evaluation point, as learning from the day was reflected upon and
analysed. The group discussion followed a session planning and
delivery activity where, as an observer I took notes of issues and
themes which emerged from each group’s presentation. I fed my
observations back to the group, which led to a broader discussion
around the themes. With significant input from the group, the discussion
was summarised into four areas of learning, forming a “toolkit” of
practice.
The toolkIt was intended to be carried through to the next day to help
prepare the participants with next section of the programme. Informal
one to one interviews were also conducted at the beginning of the
day to gauge the levels of expectations of the participants. It was
intended that these selective interviews (because not all the
participants were interviewed at this stage due to the one to one
nature of the training ice breaker that I chose to use for evaluative
purposes) would backdrop the comments from the final evaluation
exercise and perhaps give an insight into some of the learning journeys
undertaken.
Day 2
On day two, the final evaluation took the form of small discussion
groups that were already formed by the day’s task related activity. I
took this opportunity to ask the participants for their views and feelings
about the two day programme in this smaller group, as the smaller
group setting would be less intimidating for some of the participants,
enabling them to fully express themselves. I recorded their responses.
Using the recordings from the small groups, I collated and made
anonymous the main themes brought up in the small-group discussions
and gave them out to the whole group to write down (via volunteers
from the group) on large sheets of paper.
5
The evaluation then took the form of each anonymous theme being
discussed in turn by the whole group. The participants were then given
the opportunity to visit each themed sheet of paper to add their further
comments, in the process creating a “graffiti” table, as all the large
sheets of paper were laid out across all the tables, around which the
participants were sitting. I used a version of NGT (nominal group
technique) to analyse the relative importance and weight of the
written samples that were left on the graffiti tables. The graffiti sheets
showed that each person contributed (on average) one comment to
each sheet, so that the theoretical maximum total of comments would
have been eighteen, the number of participants, if all the participants
only commented once under each category. Each participant’s
response was identified (differentiated) by handwriting and colour of
marker used.
Figure 1, a graffiti sheet
6
Findings
Interim Evaluation – Generation of an Artist’s Toolkit
As a result of the session planning activities on day one, four key areas
were identified:
Risk Assessment
Evaluation, Reflection and Monitoring
Planning
Session Management
The excerpts used in this section are taken from transcripts of the
participatory process of evaluation used on day one.
Risk Assessment
It was acknowledged that artists need to be aware of general health
and safety of the activities they plan. It was discussed that a risk
assessment should be carried out one month before the first session.
There was also agreement that specific time to plan and conduct a risk
assessment should be contracted as non contact time. It was also
mentioned that artists need public liability insurance and those working
with Prescap are automatically covered.
A participant illustrating the importance of carrying out a risk
assessment said:
“There’s been a couple of times that I’ve actually realised that
maybe I really need to be so acutely aware….I was using …silk
the other day with groups in wheelchairs [I] never had a problem
before but then it …literally was just two seconds, this wheelchair
just went off and this bit of silk because it didn’t give it…..totally
freaked me out and I realised that I couldn’t use that again, it
would have to be lycra, it would have to give in case…if
anything got wrapped...”
Group Evaluation discussion, day one
Evaluation, Reflection and Monitoring
Evaluation was characterised by the question, “How would you do it
differently?” Evaluation was described as needing to be ongoing
throughout the session series and at the end.
Reflection was characterised by the question, “What did you do?”
Reflection was also described as recording the feelings and thoughts of
the participants and relating that to how they feel.
7
Monitoring was also discussed, as information on the participants, such
as attendance, ages et al, could be seen as important (especially for
funders). It was explained that for a YOT setting, monitoring was vital in
order to record not just the numbers attending in any one session but
how many participants as a whole accessed the entire intervention. It
was stressed that the artist is often the main link between the
participants and that their monitoring notes would be extremely useful
to the YOT team to evaluate the overall impact of the intervention. A
YOT worker said that it would be useful to them to receive an artist’s
end of session report so that they could be kept abreast of session
developments. However, an artist did question whether this was
actually a job for the artist or the YOT worker. It was argued that due to
the nature of erratic attendance and the low numbers involved in
each session keeping a register of attending participants would be
manageable.
A broader discussion followed about the contribution an artist’s end of
session notes play in the larger monitoring and evaluative processes of
the YOT service at large, as each piece of information contributes to
the bigger picture of the participant.
A participant summarised as follows:
“Summarising and reflecting on what’s happened, what’s been
learnt and working out which bits we can take forward to the
next session……It’s about reflection and learning, not just about
the learning….it’s about the processing of ‘what have I learnt,
here?’ But it’s also about asking them their opinion about what’s
gone on and involving them in the decision making process”
Group Evaluation discussion, day one
Planning
It was recognised that it was important for artists to plan collaboratively
with their YOT workers. Planning was also linked to the risk assessment
phase, mentioned earlier. It was suggested that planning and risk
assessment could be done collaboratively with the YOT worker, one
month before the start of the session. The erratic nature of the
attendance was also discussed and its implications on the planning
process in terms of contingency activities to take account of the
participants’ needs.
8
Session Management
The importance of being adaptable in-session was discussed as was
the importance of grading an activity to the capabilities of the
participants.
This point was mentioned by a participant:
“I think the key….is not to presume….that they can all read and
write. It’s best to presume that you’re going in there to help
them. You give them some of the options, some of the options to
shout out and you do the work”
Group Evaluation discussion, day one
The quote above followed a general discussion about giving the
participant a variety of ways in which they could record their feelings
that were not always based on writing activities. The YOT workers
present said that they could provide information about literacy if
asked.
Being clear about the roles of everyone in the session or space was
deemed important and had implications on how a session was
planned. Here is an excerpt of the discussion around this point:
One of the observations I made was that none of the groups
actually demonstrated how they would actually manage a
session; there was often one person delivering the session and
four of you….you were just standing in the background….
“I think it was because we knew it wasn’t a real scenario…I used
to deliver sessions on my own, just like that….Then I realised the
value of [having] a second person in there…..whereas if you’ve
got some one in there to support, use that person as another
tool”
“It’s worth utilising your YOT worker…using the teaching
assistant….make sure that they do the work for you”
Group Evaluation discussion, day one
This point was summarised by:
“Often you’ll have as many adults as you do [young] people in
the room with you and it won’t be unusual and as long as those
workers…[are] actually doing something, usually doing the
activity, seems to be absolutely ideal…There’s a danger as
well….whether there are young people who are not doing
anything in the session or staff or adults not doing anything; it
9
creates a disruption….there’ll be separate conversations start to
happen somewhere that’s not linked to the activity and its quite
hard to manage in a session as an artist”
Group Evaluation discussion, day one
From a YOT perspective, interventions should be largely planned
around:
Introducing the session
Delivering the session
Reflecting on the session (which should always be ongoing)
In further discussing the above format, the YOT workers said:
“The young people that we deal with; they don’t have a lot of
structure in their lives….and they do like structure and they do
kind of respond to a bit of structure….they like to know what
they’re gonna be going and then they like praise, they like to
have done something, to have achieved something and to be
told what they had achieved at the end”
“I think they need praise all the way through, your enthusiasm
needs to be projected right through the session”
Group Evaluation discussion, day one
How to appropriately encourage participants was also discussed:
“I’ve heard from other YOT workers that giving praise can be
quite difficult for young people to take in front of other people.
So you’ve actually got this safety zone for the fact that you have
got one or two, potentially just one to one; and they do respond
to it as they get to know you…The review process is a good way
of doing it….talking it through then reviewing it at the end. It
really does reinforce them.”
Group Evaluation discussion, day one
It is noted that many of the proposed resources above already exist at
Prescap.
Key Findings
Artists require a pro forma checklist of health and safety issues as
well as permissions (photographic et al) that need to be gained
Artists require pro forma to assist in planning their sessions. This
could possibly be included in the health and safety checklist.
10
Artists require a pro forma a pro forma evaluation/monitoring
form that will assist the artist in gathering the required
information. This could take the form of an ‘artist’s diary’.
Recommendations
Prescap should consider tailoring (with YOT assistance) its
planning and evaluation resources to meet the specific needs of
the YOT setting
Precap should consider making its specially YOT tailored
planning and evaluation resources available to all artists with
whom it works in this sector
Figure 2, completing graffiti sheets
11
Final Evaluation of the Programme - A participatory activity
Here are the themes that were generated by the small groups and
discussed by the whole group on day two:
Delivery of Training days
Lots of Planning, not enough Practice
More balance between listening and doing
More need for role play
More videos of practice and shadowing opportunities
Content of Training of days
Some information on day two was negative
Day one was useful but a bit frustrating going over old ground
[Did] day one prepared for day two?
Not enough information about visiting artists’ case studies
Some content not always new, learning not sufficiently
advanced
Management of Training days
Would be good to have notes to take away
Toolkit evaluation too late in the day
An overview if the induction [was needed]
Delivery of Training days
Here is a table with all of the participants’ graffiti comments around
each issue:
Lots of Planning, not enough
Practice
Need for balance and not
saturated with one or other
Practice sessions seemed
squeezed in as other
sessions ran over, so no
chance to take them apart
and explore stage by stage
[A] Bit bogged down with
official paperwork info that
most of us know
More balance between listening
and doing
Role plays, please
Practical communication
techniques
Contact list, please
Not used to sitting still for
long periods – less tiring to
12
vary activities/move about
a bit
More need for role play
Use Icebreakers
Yes, please
Body language
I’d love to deliver that
session
More real life situations
showing how to address
boundaries
Conflict resolution training,
please
More videos of practice and
shadowing opportunities
Voluntary work
Videos in [the] lunch break
Observation within YOT
1 day training, 1 day
shadowing
Definitely may shadow and
love it or hate it
Yes, please/agree
Very helpful seeing and
meeting variety of artists
with experience
Table 1
Figure 3
13
Discussion
Here is an analysis of the above table
Lots of Planning,
not enough
Practice
Timing issues
around
session
management
Training
content not
adequately
assessed
33%
of graffiti-
table
respondents
66%
of graffiti-
table
respondents
16.6%
of whole
group
More balance
between listening
and doing
The need for
more
practical
activity
75%
of graffiti-
table
respondents
22%
of whole
group
More need for role
play
The need for
specific
targeted
activities
dealing with
group
management
83%
of graffiti-
table
respondents
33%
of whole
group
More videos of
practice and
shadowing
opportunities
The need to
actually see
practice as
well as hear
about it
71%
of graffiti-
table
respondents
38.8%
of whole
group
Table 2, a content analysis of Table 1
More need for role play
Out of those who left a comment this seemed to be the most important
impression left with the participants at 83%. This was relatively very
important to the artists, as it attracted comments from 33% of the
whole group, the second highest response in this category. It was
noted that there was not always a clear link between the activities
presented. This was illustrated in day one, where the safeguarding
exercise designed to promote awareness of all aspects of
‘safeguarding’ was not reflected in the role play-task planning exercise
that followed it. This lack of clarity between activities was reflected by
a frustrated participant:
“I think something else was going on…..at that point in the
afternoon….just be really honest about it…I think the seriousness
that you are actually bringing to this now wasn’t apparent
then…so I think it is a big shift ….you talking about coming in
going, ‘why wasn’t that happening then?’ and I think we’re all
14
[were] a bit silly over it and I don’t think any practitioner in this
room would act in that way. I would be very surprised”
Group Evaluation discussion, day one
By day two, the group said that it wanted more role play but the
session planning activity on day one was a role play but the group at
this point did not seem to fully appreciate the potential learning from
this activity:
It was interesting because there was one person in the group
who actually interacted with us…….whereas a lot of the other
groups gave us a task and let us get on with it..
“Don’t forget it was a role play situation with five minutes to do
it… let’s put it into some sort of context….”
…We’re here to interrogate and reflect on the process, it was a
role play but lets….extract some learning out of it. That’s what
we’re here to do...”
Group Evaluation discussion, day one
However, by day two, the strong need for meaningful role play was
recognised:
“I’ve worked in lots of different places, learnt lots of different
things but I’ve never had any training…just…. played it by ear
and used your common sense but obviously it would be good to
get some top tips from people…”
Group Evaluation discussion, day two
The above quote is especially interesting as this participant came with
an expectation of gaining more practical skills:
“I don’t know…The reason why I wanted to come on the course
in the first place I’ve worked with lots of different people…. but
I’ve never had any training to do that other than being an artist”
Interview with Artists’ about their expectations of the programme, day
one
The group evaluation discussion continued:
“Maybe the same training that you’d get as a youth worker or a
YOT worker…for dealing…[with young people]”
“They don’t get it…that’s the irony”
“Perhaps the biggest insight for me….is how you deal on a one
to one ‘cos it’s so different from a group; it’s such a rarefied
15
atmosphere and it’s quite a different situation from most
participatory arts….”
“I was interested in the body language…where to position
yourself, so as not to be ‘in their face’”
“I thought that was really good actually [referring to segment on
body language, day two]….”
“I think conflict resolution…the stuff that you did about standing
beside side…would like more on it….”
Group Evaluation discussion, day two
More videos of practice and shadowing opportunities
The issue around shadowing and its related issue around the need for
more practical activity rank as being ‘second’ in importance at 71%
and 75% of respondents respectively. However, the combined total of
responses from the whole group for these issues was 93.8%, showing an
overwhelming concern about this issue over demonstration of practice.
This is an interesting point because it was originally planned that artists
would work alongside a YOT worker in their planned interventions
before being reviewed and evaluated for their further training needs.
However, due to funding irregularities, this phase of the training had to
be postponed. Here is a detailed breakdown of the proposed
“practical” phase of the Artists’ training:
Evaluating the Impact of the Induction Programme on Artist and
LYOT practice
Area of enquiry:
What is the impact of the induction programme on the Artist’s
practice and on the LYOT setting?
Data Collection:
Interviews, session observations, questionnaires,
monitoring/evaluation sheets
To be carried out at half way (interim) and end points of
planned interventions (after induction programme) - dates
to be arranged by Project Manager (NH)
Areas to examine:
Usefulness of induction programme
Use of session plans
Use of monitoring and evaluation forms
Debrief Sessions
Effectiveness of Planned Intervention
16
Draft evaluation questions (for both Artist and LYOT worker,
where appropriate) derived from above:
How has the induction programme influenced your
practice?
Do you feel confident that your knowledge of LYOT has
enhanced your practice in the setting? If so, how?
How collaborative has your session been with the LYOT
worker (is, for interim interview) in terms of your session
planning and delivery?
How did you (do, for interim interview) plan sessions?
o What have been (are, interim interviews) your roles
(you and your YOT worker)?
o How useful were (are) the debrief sessions – how did
(do) you use them?
o How useful was (is) the monitoring/evaluation sheet
– how did (do) you use it?
What have you learned about your intervention? (so far,
for interim interview)
o Evidence of benefit (learning) of young person
Educational, behavioural, attitudes, self
esteem, confidence etc
o Evidence of YOT worker learning
o Impact on YOT team – in terms of setting practice
How were the planned intervention, reporting
and planning structures? Did they work?
How embedded was the planned intervention
in the practice of the setting? (or how
embedded could it be?)
How can this impact be sustained (in the
setting)?
Draft Evaluation Framework template 1. 12.8.08
In the whole-group evaluation discussion some of the highlights around
this issue were:
“….maybe everybody’s just saying that we want more of other…
artists coming in …more about the actual running of the art
workshop because the stuff about YOT…you can find out that
stuff from the internet . An actual artist’s workshop, you can’t
actually find that stuff….”
“….It would be nice to see at one point an example of what
actually happens in a workshop…”
Group Evaluation discussion, day two
17
Lots of Planning, not enough Practice
Around two thirds (66%) of the messages left on the graffiti-table
registered a feeling of being “bogged down” with the paper work that
was presented to them over the two days. This percentage probably
reflects the mixed levels of experience of the group, as some did
actually find the training correctly pitched to their level. This
observation of the mixed nature of the response is corroborated by the
lowest group response rate of 16.6% to this issue. However, this point is
interesting as it was recommended at the planning stage that this
information should have been supplied to the participants in advance:
Develop an induction pack
o A pro forma planning tool for artists, a check list of
planning items, you want to see put in action
o A pro forma monitoring/evaluation sheet (see template)
o Background information of LYOT and Prescap – including
policies around health and safety etc, what to do in
disputes (protocol) – who to talk to etc
Draft Evaluation Framework template 1. 12.8.08
The above point was echoed by a participant in their initial day one
interview:
“To be honest I didn’t know what to expect because there
wasn’t really any information sent about what we’d actually be
doing…..sounded like really good training and the fact that you
had to have an interview….made it seem better….”
Interview with Artists’ about their expectations of the programme, day
one
Timing issues around session management
From the lowest group response rating of 16.6%, one point about the
timing of activities was made (by 33% of those who responded) with
the implication that it interfered with the potential for learning:
“…Timing….I would have appreciated it if you’d had been a little
bit tougher…but…I felt like towards the end of the day….like now
[?] we’re rushing it”
Group Evaluation discussion, day two
Key Findings
Activities were not always run on time
Not enough information was given in advance to the artists
Unexpected funding challenges severely curtailed the scope of
the training provision
There was a need for more structured practical activities
18
Recommendations
Time management of activities need to be revisited
Training needs should be adequately assessed during future
interviewing processes. This should include more activities around
role play and behaviour management
Induction material containing all the necessary information
needed to work in the sector should be provided to artists before
training programme
Figure 4
Figure 5
19
Content of Training of days
Here is a table with all of the participants’ graffiti comments around
each issue:
Some information on day two was
negative
[Too many?] extremes
Sometimes need to be
aware of the possibilities
Mmmm? Not sure?
Part of the job!
Don’t agree
Ditto
Day one was useful but a bit
frustrating going over old ground
It is always good to be
reminded of knowledge
already held – there is
always a new angle
It’s good to reinforce
knowledge/practice
(experience of group
broad)
I felt it was good to reinforce
good practice
Ditto
A bit of a sucking eggs
scenario for some of the
oldies
Definitely
I found it useful – and I’m
old but new
Things that aren’t as subject
specific could have been
left out e.g. risk assessments
[Did] day one prepared for day
two?
Day 1 was a good
background knowledge
Day 1 morning was
interesting. The task was not
a very good experience for
me
Enjoyed learning about YOT
Not really but that was fine
Day 1 – too long (start
earlier?)
Good to give practical
techniques to what was
talked about yesterday
Some content not always new,
learning not sufficiently advanced
Interview and application
very advanced so
20
expected same of the
course
I found it all helpful and
relevant
Agree, very much
Would have liked more
debate and more people
contributing in the large
group
A little too much about risk
assessments
Got a little too bogged
down with this
Level was about right for me
Possibly split future session
up, depending on
experience
The nature of a group with
people from [a] range of
backgrounds and
experiences
Not enough information about
visiting artists’ case studies
Disorganised, lack of clarity
They were understandably
nervous
Felt slightly crammed in
Slides were mixed. So it was
difficult to tell between the
two
[Wanted] clear example of
project that worked well
and one that didn’t
More detail about sessions
would have been good
More about visual arts
examples as seems to know
more about drama practice
participation than visual arts
Need to know more about
them, their experience,
practical knowledge, how
[many?] young people,
what worked, what didn’t –
why
I think this may have been
due to time. It would be
difficult to cut other sessions
to make this possible
21
because all was relevant.
Maybe an info pack with
notes, references and case
studies would be helpful?
Table 3
Figure 6
Figure 7
Figure 8
22
Discussion
Here is an analysis of the above table.
Some
information on
day two was
negative
Disproportionate
number of
negative examples
Recognition of
balanced portrayal
of work
16.7%
of graffiti-table
respondents
83.3%
of graffiti-table
respondents
33.3%
of whole
group
Day one was
useful but a bit
frustrating
going over old
ground
Day one providing
good background
knowledge
No new knowledge
from day one
62.5%
of graffiti-table
respondents
37.5%
of graffiti-table
respondents
44.4%
of whole
group
[Did] day one
prepared for
day two?
Day one prepared
for day two
Day one did not
prepare for day
two
50%
of graffiti-table
respondents
50%
of graffiti-table
respondents
33.3%
of whole
group
Some content
not always
new, learning
not sufficiently
advanced
Course content not
sufficiently
advanced
Course content
sufficiently
advanced
55.5%
of graffiti-table
respondents
44.4%
of graffiti-table
respondents
50%
of whole
group
Not enough
information
about visiting
artists’ case
studies
Satisfactory visiting
artists’ involvement
Unsatisfactory
visiting artists’
involvement
0%
of graffiti-table
respondents
100%
of graffiti-table
respondents
44.4%
of whole
group
Table 4, a content analysis of Table 3
Not enough information about visiting artists’ case studies
All of the artists (100%) who chose to leave a response on the graffiti
table had reservations about the segment delivered by the visiting
artists. This was clearly important to the group, as it received the
second highest number of responses at 44.4% of the group. Here is an
excerpt from the whole group discussion which will clarify some of the
participants’ concerns:
“More practical information”
23
“Even the photographs, you couldn’t quite work out what you
were looking at and how it fit together, so you weren’t too sure
really what was made”
“It would have been good to have seen the process”
“I think we needed more time with them ‘cos their time obviously
got shortened and then it was tagged on”
“…I think the fact that they spoke about two projects but the
images of those two projects were mixed together….although
they were talking about two separate projects in a way that they
were, you couldn’t see how that effected….”
“I would have liked them to say who they are, where they come
from….[how] do they decide what they do, how they do it… just
with one project would have been enough “
Group Evaluation discussion, day two
Although, one participant remarked that:
“Although…the actual presentation wasn’t so much about what
they did but about their experiences of how it worked with the
YOT”
Group Evaluation discussion, day two
The overall feeling about this segment was:
“I felt they were speaking to you not us….almost a reflection of
evaluation of working with you rather than for our benefit”
“And also I would have liked to have heard them talk about the
conflict and the positive time…some people could get insight of
that…not just the project”
“’Cos they didn’t talk about how they actually sort of worked
with the young people… what problems ….any positives any
negatives to come out of that…it was all about the youth
workers rather than the young people”
Group Evaluation discussion, day two
This excerpt from the discussion shows that the participants would have
benefitted from a more structured presentation that concentrated on
their needs as artists.
24
Some information on day two was negative
Most of the artists (83.3% of respondents) felt that the level of
challenges discussed on day two were realistic. However, this perhaps
was not felt to be important to the whole group as it only attracted
33.3% response:
“That’s the reality. If you’ve not worked in this setting before you
need to know what you might have to face”
Group Evaluation discussion, day two
This seemed to meet the initial expectations of the participant:
“I think what I want out of it more than anything is some idea of
the expectations to have [in] the settings”
Interview with Artists’ about their expectations of the programme, day
one
The group continued the discussion:
“Yesterday [day one] was a little more sanitised because the
YOT officers were here….it broke the ground for today”
Group Evaluation discussion, day two
Day one was useful but a bit frustrating going over old ground
The quote above “it broke the ground for today” is interesting as 62.5%
of the respondents thought the first day did actually provide useful
knowledge, although 37.5% of the respondents did not find the
knowledge useful, as they already knew much of what was discussed.
The issue Some content not always new, learning not sufficiently
advanced (see table 3), attracted the highest number of comments,
55.5% of the respondents, in so doing showing its relative importance to
the group (at a whole-group response rating of 50%). The mixed nature
of the group was discussed in the whole group discussion:
“I just think that’s the nature of the group when you’ve got
people with vastly different experiences…for me it was pretty
much all new”
“I think it is important to reinforce what you know already”
Group Evaluation discussion, day two
However the mixed nature of the group might contribute to
understanding why only 44.4% of the respondents thought that the
course content was sufficiently advanced, as one respondent wrote:
“Interview and application very advanced so expected same of
the course” Graffiti table, day two
25
The level of training clearly did not meet with this participant’s
expectations:
“…..sounded like really good training and the fact that you had
to have an interview….made it seem better….”
Interview with Artists’ about their expectations of the programme, day
one
In order to manage this disparity of experience within the group, a
respondent suggested:
“Possibly split future session up, depending on experience”
Graffiti table, day two
This will have specific implications for the future interviewing process to
be used as a sharper tool for assessing artists’ needs and experience:
Negotiating the Artist Person Specifications with the Action
Learning Set
Sending out a request to action learning set for input into the
following
Skills – link into CPAL Competency Framework
Knowledge – link into knowledge LYOT
Experience – must have Public Liability Insurance
Draft Evaluation Framework template 1. 12.8.08
[Did] day one prepared for day two?
Only half of the respondents (50%) thought of day one as preparation
for day two, despite the reflective toolkit generation activity on day
one. This received a joint bottom ranking of 33.3% for whole group
response rate:
“I don’t know whether it was supposed to prepare…”
“I don’t know whether it prepared for it but I think day two for
some people would have been a bit heavy had we not had a
lighter weight version of it on day one”
“I thought it was a background…it introduced me to the area…”
“Yeah….I though it was structured in well…”
“There wasn’t that many overlap[s]….between [day one and
day two] they fitted together…”
Group Evaluation discussion, day two
26
This is interesting as it would appear that more people found the
information given on day one useful than they did a preparation for
day two. This would seem to emphasise the earlier finding of the
relative lack of continuity (for most) between the two days. This would
tend to call into question the extent to which day one “broke the
ground” for day two for the entire group. This issue around day one’s
function and its relationship to day two clearly was hugely important to
the group as both combined issues attracted a 77.7% whole-group
response rate.
Key Findings
The case study presentations of the visiting artists needed to be
more focussed and better structured
The mixed nature of the group seemed to hinder the learning
process for some of the more experienced participants
The course content of day two seemed to meet the theoretical
but not the practical expectations of most of the group
Recommendations
Visiting artists need to be better supported to give presentations
on the programme. They should, perhaps, be given a checklist of
what they need to cover in their presentation
The interview process needs to more accurately determine
action learning sets that match up the experience of the
participants more closely
27
Management of Training days
Here is a table with all of the participants’ graffiti comments around
each issue:
Would be good to have notes to
take away
Yes, it would. Or emailed out to
individuals
Yes!
No it wouldn’t
Yes it would
(Oh no you don’t)
I have to feedback to my
manager so notes to take back
would be good
And websites useful, books and
publications
Taking your own is a useful way
to maintain concentration
Contact details of participants
Toolkit evaluation too late in the
day
It was a little rushed
Hard when interspersed as did
cut into planning session with
group and was quite frustrating
when was quite manic
anyway. But know [it is]
important
Defo need a toolkit
Will come later via Prescap
Like the idea of it being
stepped through the day
Stagger it in – too much when
we were all tired
An overview of the induction [was
needed]
Great
We had it written down
5 minute verbal presentation
on what to expect why we are
here today. What to expect
out of the 2 days
Agree!
Yes
Table 5
28
Discussion
Here is an analysis of the above table.
Would be good to
have notes to
take away
Would be
helpful
Would not
be helpful
66.6%
of graffiti-table
respondents
22.2%
of graffiti-table
respondents
50%
of whole group
Toolkit evaluation
too late in the day
Tool kit
should be
staggered
Tool kit
important
50%
of graffiti-table
respondents
50%
of graffiti-table
respondents
33.3%
of whole group
An overview of
the induction [was
needed]
Overview
was needed
Overview
was not
needed
80%
of graffiti-table
respondents
20%
of graffiti-table
respondents
27.7%
of whole group
Table 6, a content analysis of Table 5
Would be good to have notes to take away
This was very important to the group as 50% of the group responded to
this issue and 66.6% of the respondents thought that post training notes
would be useful:
“Yeah. Definitely.”
“Emails, websites and the reference….the…acronyms”
“I would really like everybody’s contact details…”
Group Evaluation discussion, day two
Toolkit evaluation too late in the day
This gained the second highest score in its category with a 33.3%
whole-group response. However only half (50%) of the respondents
thought that the toolkit should be staggered, as mirrored in the excerpt
below:
“…A bit tiring…it was quite intense….the four till five slot…if we
could…bring sections during the day”
Group Evaluation discussion, day two
29
Interestingly only 50% of the respondents found the toolkit important.
This could possibly be explained by the mixed nature of the group
where 55% of the respondents thought that “Some content [was] not
always new, [and] learning not sufficiently advanced” (see table 4).
This is a strong possibility, as 50% of the group chose to respond to that
issue.
An overview of the induction [was needed]
This gained the least response from the group at 27.7%. However, of
those who responded to this issue, an overwhelming 80% of
respondents thought that an induction overview was needed:
“Just to reinforce why we’re here”
Group Evaluation discussion, day two
Key Findings
Post induction notes were very important to the group as a
whole
The group as a whole was not sure about the generation of an
artist’s toolkit at that time of the day and the group was not
perhaps convinced of its importance
An induction overview was found to be important by only a few
participants
Recommendations
Post induction notes should be given to artists. Interestingly, this
request gains more group support than the request to give out
information before the training.
The relatively low whole-group response to the toolkit generation
idea and the ambivalent response towards staggering its
delivery would suggest that this activity would be better placed
being given as either pre or post induction material
An induction overview should be given despite the low group
response, as from those who responded to this issue there was an
overwhelming request for it
30
Participatory Evaluation Summary
Category
Issue of most
importance
Whole group response
Delivery of Training
More videos of practice
and shadowing
opportunities (the need for
practical activity
combined with these issues
command a 93.8% whole-
group response rate)
(More need for role play,
had the largest number of
graffiti table respondents in
favour at 83%)
38.8% (93.8% cluster)
(33%)
Content of Training
Not enough information
about visiting artists’ case
studies (had the largest
number of respondent
agreement at 100%)
Some content not always
new, learning not
sufficiently advanced (this
had a majority 55.5% of
respondents in agreement)
44.4%
50%
Management of Training
Would be good to have
notes to take away (this
had a majority 66.6% of
respondents in agreement)
(An overview of the
induction [was needed]
had the largest number of
graffiti table respondents in
favour at 80%)
50%
(27.7%)
Table 7
Table 7 shows the most important issues that need to be addressed
that the group identified in the evaluation.
Figure 9
31
Appendix
LYOT Training day questionnaire for LYOT staff 19.03.09
1. What were your expectations of the programme?
YOT Worker 1: My expectations were pretty simple. I had been asked
address a number of artists and to give a brief talk focusing on my
experience of working with challenging groups of young people.
YOT Worker 2: To make the group aware of issues surrounding working
with young people at the YOT, help enthuse and inspire the group and
identify any support needs that came out.
2. Were they met?
YOT Worker 1: I gave the talk and answered questions. The majority of
the artists responded well and appeared to show a genuine interest in
working with young people. The afternoon session highlighted the
diverse creative talents of the artists and the results of the workshops
were impressive.
YOT Worker 2: On day 1 yes, day 2 n/a
3. Is there anything you would need from an artist (in terms of their
practice) to work more effectively with them?
YOT Worker 1: Enthusiasm and commitment to the whole project, which
includes good pre planning and good time keeping throughout the
project. They would need a comprehensive risk assessment.
YOT Worker 2: Just a willingness to be flexible and work in partnership
with the YOT staff.
4. What were the most successful parts of the programme?
YOT Worker 1: I thought that the artists were offered a very good
overview of both the positive and negative sides of working with young
people.
YOT Worker 2: The exercise on planning a workshop was excellent,
listening to the planning processes was interesting and the ideas that
came out were wholly workable. The group I worked with got to grips
32
with the context of the task and were very creative in exploring
methods of delivery.
5. What were the least successful parts of the programme?
YOT Worker 1: Difficult to say but I guess the programme failed to run to
schedule and keeping to a timetable is something that we were asking
the artists to bear in mind when working with young people. Pro-Social
modelling eh.
YOT Worker 2: I certainly wouldn't say that anything was unsuccessful.
6. What would you like to see included (or left out) in a future
programme?
YOT Worker 1: Difficult to say as I only attended the first day and there is
nothing I would have left out.
YOT Worker 2: I would have liked to have seen an exercise that allowed
the artists to highlight the work they do, maybe for inclusion in a
directory or similar to enable the artists to contact each other to share
best practice, tips and lessons learned. (I appreciate that this might
have been done on day 2)
7. Are there any other comments you would like to make?
YOT Worker 1: It was an enjoyable and enlightening programme.
YOT Worker 2: Well done Naomi, really enjoyed the day and hope to
be able to get the opportunity to work with some of the artists in the
future. I would like the contact details and a synopsis of each artist to
share with the Young Peoples Service in order for them to be able to
use their services in for further possibly joint projects.
Thank you for your time