OERtest Testing an Open Education Resource Framework for Europe Deliverable D2.2 –Methodological Handbook

Full text

1 | P a g e
Testing an Open Education Resource Framework for Europe
Work Package 2: Laying the Ground Work
Deliverable D2.2 – Methodological Handbook
Due date of deliverable: xxxx
Actual submission date for this version: XXXX
Lead organisation for this deliverable: Scienter
Project co-funded by the European Commission
Dissemination Level
RE
Restricted
X

2 | P a g e
Deliverable fact sheet
Deliverable version:
1.0
Deliverable type:
Manual
Current release status:
Restricted to a group specified by the consortium
Final release status:
Public
Work package:
WP2
Responsible partner:
Scienter
Primary contributor:
Anthony F. Camilleri (EFQUEL), Anne-Christin Tannhäuser
(Scienter), Jeff Haywood (University of Edinburgh), Marcelo Maina
(Fundació per la Universitat Oberta de Catalunya)
Deliverable reviewers:
Version
No.
Date
Sections
Change
Author/ Editor
0.1
09.2011
all
Anthony F. Camilleri
Anne-Christin Tannhäuser
Actual Date of Delivery
ongoing
Audience
public
X restricted
internal
Date
Status
X draft
WP leader accepted
Quality checked
Project coordinator accepted
Action requested
to be revised by partner in charge of the deliverable
to be reviewed by the appointed partners
for approval of the project coordinator
Deadline for action:

3 | P a g e
Contents
1 The OERtest Project: Our Mission .......................................................................................... 4
2 Scope of this document .......................................................................................................... 6
3 Main project activities: Workflow .......................................................................................... 6
4 Type of Open Educational Resources targeted by OERtest .................................................... 9
5 Recruiting OERtest Partner University Experts (WP2) ............................................................ 9
6 Briefing Papers and Guideline Production (WP3) ................................................................. 11
7 Feasibility Testing: Scenario Approach, Accreditation Flow Charts and Focus Groups (WP4)
............................................................................................................................................... 12
Annex 1 Form Expert Profiles for Recruitment Meetings ............................................................ 15
Annex 2 Form Initial Feedback from Partner HEI Experts ........................................................... 18

4 | P a g e
1 The OERtest Project: Our Mission
The OERtest project is a two year (Oct. 2010 – Sept. 2012) project funded by the Lifelong Learning
Programme of the European Commission. Its aim is to support the mainstreaming of OERs within
Higher Education and to test the feasibility of assessing learning exclusively achieved through the use
of OERs.
What is our vision?
How will we get there?
Universities taking open educational
resources one step further.
OERtest will assess how new learning
pathways can be created for
Your own students: How can we
certify our own students’ learning
through OERs offered by our own
universities, e.g. as “supplementary
credits”?
Students of other universities
especially those from Europe: How
can we open our certification of
learning through the use of OERs to
students from other universities?
Non-traditional students & working
adults: How can we offer
certification of learning outcomes
acquired through the use of OERs to
non-working adults and to working
professionals and in doing so
facilitate their continuing
professional development?
Developing a validated assessment model for
learning outcomes acquired through study
using OERs shared among our partner
universities. This model will include quality
standards, assessment guidelines, financial
models, curricular provisions and other
administrative requirements.
Creating an up-to-date European
clearinghouse for high-quality OERs offered
by the OERtest partnership
Establishing a network of universities
prepared to certifying learning based on the
use of OERs sharing a similar model?
Project partners are:
P1 - University of Granada(Miguel Gea)
P2 - Scienter (Claudio Dondi)
P3 – Fundació per la Universitat Oberta de Catalunya (Albert Sangrà)
P4 – University of Edinburgh (Jeff Haywood)
P5 – ALMA MATER STUDIORUM – Università di Bologna (Roberto Farnè)
P6 – UNU (Virginie Aimard)
P7 – EFQUEL (Rolf Reinhardt)
P8 – University Duisburg Essen (Ulf Ehlers)

5 | P a g e
Sharing expertise and developing a common understanding
In order to develop a framework of learning based on study using OERs, that is shared among several
universities, OERtest will stimulate exchange between Higher Education experts in quality assurance,
recognition of prior learning, credit transfers and institutional issues such as strategy development and
HEI financing from our partner universities as well as researchers/practitioners of open educational
practice. These activities will firstly result in four briefing papers on
1. Assessment methods
2. Requirements and standards of resources
3. Credentilization, certification and recognition (institutional)
4. Inter-institutional collaboration
Secondly, these briefing papers will be the basis to develop a draft Regulatory Framework for OER
Assessment, i.e. a set of rules and guidelines that might enable the universities to certify learning
outcomes acquired through the use of OERs. It will contain
assessment guidelines
quality standards
verfication of workload
award criteria and associated mapping to ECTS-equivalents
Testing and Feasibility Analysis
The framework will be tested in the OERtest partner universities and experiences gathered via
interviews with all different groups involved. As a main outcome a ”Feasibility Analysis of OER Provision
in Higher Education” will be published including full-cost economic modelling and impact projections of
the framework to the core mission of Higher Education Institutions in Europe.
OER Europe Network
OERtest will invite further universities to participate in an initiative for mutual recognition of OER-
based learning and further mainstreaming the use of these educational resources. Our aim is to
establish a network committed HEIs evaluationg and then making use of the validated framework,
adapting it to their own needs and contributing their own resources to the OER clearinghouse.

6 | P a g e
2 Scope of this document
This handbook was produced to guide project activities of the OERtest project. It lays out the
parameters of a framework development targeted towards European Higher Education institutions for
certification of learning outcomes acquired through OERs and a methodology for assessing its'
feasibility. It is thus a product to support project partners in implementing complex tasks and remain
consistent both operationally and methodologally.
Since some key aspects evolved further during the first 6 months of the project, it was decided early that
the first handbook version shall be a live document at the OERtest collaborative online workspace with
initial suggestions from the work package leader (Scienter) and EFQUEL. All partners involved made
adjustments, fine-tuned and developed shared ideas further. The methodological handbook in the
current version is a compilation of the final outcomes of that on-going dialogue on methodological
refinements to which the whole partnership contributed.
3 Main project activities: Workflow
The overall workflow of ground work, framework development and feasibility testing can be visualized
in the following way.

7 | P a g e
WP
Visualised
Further information
WP2
Preparation and human resource capacity
building
Invited key individuals:
Academic development
Curriculum development
Quality assurance,
Finances, administrative
Recruit subjects for feasibility testing
Travel budget for inviting key individuals
Originally aligned with 2nd and 3rd project
meeting
WP3
Briefing papers for OER assessment testing
Expert group meetings: workshop style discussing on four themes
Each briefing paper will contain an initial
overview of the topic based on 'scoping' desk
research, appropriate references to relevant
examples, a needs analysis and the position of
leading figures/associations in the field.
Each thematic group will work over several
months making use of online cooperation
tools:
Partner HEI experts +OPAL experts + experts
determined by the OERtest consortium
Set of rules and guidelines will bring together
the entire regulatory framework necessary for
providing training through OER and its
assessment
FUOC
Initial recruitment meetings with internal experts/key
university staff
Theme 1
Theme 2
Theme 3
Theme 4
1.
2.
3.
4.
Content:
1. Assessment methods
2. Requirements and standards of Resources
3. Credentilization, certification and recognition
(institutional)
4. Inter-institutional collaboration
Assessment guidelines Quality standards
Verification of workload Quality criteria
Regulatory framework for
OER assessment
2 Meetings in workshop form with
university internal experts/decision
makers to prepare briefing papers
OER Clearing House
Phase 1: Aggregator of
related documents
Laying the groundwork
Framework development
UniEd
UNIBO
UNU

8 | P a g e
WP4
Feasibility test
Partner
universities
Other HEIs
Interviews: Potential for
Implementation?
Regulatory framework for
OER assessment
Dry Test 1
HEI partner 1
Dry Test 2
HEI partner 2
Dry-testing of parts of the assessment
framework (where the framework is
applied to live situations in mock
assessments) will be carried out across
the consortium to aid the accuracy of the
analysis.
Feasibility Analysis of OER Provision in HEIs
Full-Cost Economic Modelling of
OER Provision
Scenarios: costs of providing
OER and potential income
menu of sustainability models
for continued provision (e.g.
measures required for
subsidisation, measures
required for self-sufficiency,
measured required for mixed
models ec)
Impact Projections of OER
Provision on Institutions
Analysing the potential impact a
properly elaborated system of
OER Provision within a
framework similar to that
developed in the project
in terms of contribution to the
core missions of the HEI.
Results
1. Interviews
2. Dry tests
The main deliverable of the project, the
feasibility analysis will bring together the
results of all tasks within
WP4. It will be given an ISBN number,
published and distributed widely
amongst HEI professionals, OER
experts and policy/decision makers.
OER Clearing House
Phase 2: Fill with
courses
Feasibility testing

9 | P a g e
4 Type of Open Educational Resources targeted by OERtest
The project will look at assessment of OER‘s which meet the following criteria:
Are created as entire course modules / credits
Are already offered by institutions participating in the consortium
Can be taken as stand-alone modules
Can be taken through distance learning
Can be largely self-taught, with little-to-no tutor intervention during the course
Those features are not intended to form a new definition of OERs, but rather to narrow down suitable
resources for certification and integration into the OER clearing house to be developed by the project.
Alternative terms to coin those resources are “Certified Open Course Ware” or “OERs for Self-Study”.
5 Recruiting OERtest Partner University Experts (WP2)
The partnership invites key individuals from their institutions regarding:
Academic development
Curriculum development
Quality assurance and
HEI Finance and administration
with expertise and competences in the following areas:
quality assurance
credit transfer, recognition type competence (international office staff)
OER / OER Initiatives / OpenCourseware
competence recognition / certification
validation non-formal / informal learning
institutional strategy and
financing of HE (strategic)
For this purposes internal recruitment meetings will be organized by the partnership. Their content of
may cover the following topics

10 | P a g e
However, content and form can be adapted freely e.g. depending on the previous knowledge of the
respective experts.
To gather information on the composition of the future international expert group and their initial
feedback towards the OERtest project objectives two questionnaires will be appointed. These can be
found in Annex 1 and Annex 2.
Introduction to OERtest
Resources: “OERtest Overview”, “OERtest visualized”
What does the OERtest project offer to you?
Sharing expertise and developing a common understanding regarding
- Assessment methods
- Requirements and standards of Resources
- Credentilization, certification and recognition (institutional)
- Inter-institutional collaboration
Meeting (other) experts on Open Educational Resources, one of the emerging fields in HEI innovation
Networking opportunity beyond the OERtest project (OER Europe initiative)
What results we would like to achieve with you?
The Briefing Papers: Key questions, authoring teams
Information on the 2-day International Workshop
1. Format
- In a highly-interactive format
- room for your ideas, opinion and critical thoughts
- anticipate barriers and hindrances
2. Expected workshop outcomes
- Initial draft of the four briefing papers
- List of issues for a shared regulatory framework
Initiate first discussion
Resources: Key Questions, Form “Expert Profiles”, Form “Reporting initial feedback”
Are all issued addressed?
What hindrances do you perceive?
To reach the OERtest goals which further key persons of your universities could be involved?
Practical information
Resources: “OERtest Bonn Directions”
Flight and hotel arrangements

11 | P a g e
6 Briefing Papers and Guideline Production (WP3)
The briefing papers are intended to
1. Identify Barriers to the adoptions of the developed scenarios by the partner institutions,
through literature review, experience and reaction with institutions
2. Suggest solutions to these barriers, from best practice examples, and from discussions within
institutions
Papers will be composed by project partners after input from experts has been received in the 1st
international workshop and fine-tuned after a feedback round in the 2nd workshop.
For this purpose authoring teams will be set up who will create a set of key questions to be answered for
each of the thematic areas.
Briefing paper #
1 Assessment Methods
2 Requirements and Standards of
Resources
Lead author
University Duisburg Essen
University of Granada
Co-author
Fundació per la Universitat Oberta de
Catalunya
University of Bologna
Briefing paper #
3 Credentialization, certification and
recognition
4 Inter-institutional collaboration
Lead author
University of Edinburgh
Scienter
Co-author
United Nations University
EFQUEL

12 | P a g e
7 Feasibility Testing: Scenario Approach, Accreditation Flow
Charts and Focus Groups (WP4)
The overall goal of the feasibility test is apply the whole process as laid out in the guidelines to the
OERtest universities, evaluate its potential for successful implementation, identify enablers and barriers,
and make recommendations how to improve the guidelines.
Building scenarios supports decision-making and aids to develop strategies for innovation. They
furthermore ease creating a common understanding in multidisciplinary teams on ill-defined or fuzzy
concepts. Given the methodological complexity and geographical dispersion of involved individuals in
different national HEI context, the partnership developed two sets of scenarios in consecutive steps with
different purposes:
A) Narrowing down the scale of the OER assessment and the type of learners involved
B) Adding components of internal HEI cooperation and agreements to the potentially testable
possibilities (derived from A) which shall provide the basis for feasibility testing
Scenarios A
Certified specialisation to a degree offered through OER*
Joint Degree, with different modules offered by different institutions through OER*
Full ‚OER Degree‘*
Certifying studies of OER, offered as ‚supplementary credits‘ within an institution to its own students
Certifying studies of OER, by the institution producing the OER, to non-(traditional) students taking
credits as continuing professional development
Certifying studies of OER, by the institution producing the OER, to students who are not from the
institution, leading to a recognised certification
Scenarios marked with * display a future perspective, but will not be tested in the project

13 | P a g e
Scenarios B
scenario name
learner is
studies
OER
module
at
requests
assessment
from
uses
credits
at
notes
OER
traditional
student at
U1
U1
U1
U1
only difference from traditional is OER-
based materials
self-study modules with credit are not
really unusual
OER Erasmus
student at
U1
U2
U2
U1
converse applies for students at U2 taking
OER module at U1
OER Summer
School
student at
U1
U3
U3
U1
U1 has no agreement on standards etc
with U3 and so must assess quality of the
credits, perhaps using exam or portfolio
model
OER Credit
Market
person not
currently a
student at
U1 or U2
U1
U1
not at
U1 or
U2
U1 assesses learner using the methods it
has decided are appropriate for its own
OER module and offers ECTS credits to be
taken away and used as learner wishes/is
able
OER
Anywhere
student at
U1
U3
X
U1
U1 has no knowledge of the curriculum or
standards etc of the offering at X, and so
must assess quality of the credits using
RPL methods, eg exam or portfolio model
OER RPL
person not
currently a
student at
U1 or U2
X
U1
U1
learner wishes to enter U1 and offers
learning from OER as basis for entry. U1
must assess using RPL type protocols as it
has no prior basis for evaluating standards
of the OER curriculum
U1 and U2 have a bilateral agreement on quality and standards so that students can transfer
credits obtained.
U3 has no agreement with U1 or U3 - it is an unknown quality organisation from the viewpoint
of U1 and U2
X is any internet resource used by any learner
U1 or U2 are always free to decline to assess learner under any scenario other than Traditional
and Erasmus
With regards to the above scenarios OERtest will
1. Map of the organization units based on the dimensions involved in the accreditation process
2. Identify key informants
3. Implement focus group with the responsible of each technical unit involved in the process:
enablers and barriers

14 | P a g e
4. Implement focus group with students
1
5. Create an accreditation flowchart (OERtest researcher in each institution)
6. Lead interviews with the head-responsible for accreditation for the flowchart validation
purposes.
Detailed methodological planning will take place as soon as the guidelines for OER assessment are
available, since testing is dependent on their specific nature.
1
Still under discussion

15 | P a g e
Annex 1
Form Expert Profiles for Recruitment Meetings

16 | P a g e
Recruitment Meetings – Expert Profiles
The information collected aims to feed into the preparation of the expert meeting in Bonn.
Please feel free to provide all kind of input that you feel is relevant for the OERtest project.
Title:
Your first name:
Your last name:
Your position:
Your email address:
What can the OERtest project do for you?
I am most interested in knowledge exchange with other universities about
Assessment methods
Requirements and standards of Resources
Credentilization, certification and recognition (institutional)
Inter-institutional collaboration
I am interested in contributing/leading a publication in the field of Open Educational
Resources
I am interested in meeting experts on Open Educational Resources
I am interested in networking and the OER Europe initiative
Other
After having been introduced to the OERtest project: What expertise/experiences do you
consider yourself most valuable to share with colleagues at the international meeting?
Quality assurance
International cooperation (e.g. credit transfer)
Assessment and certification of learning outcomes
Institutional strategy
Financing of HE (strategic)
Validation of non-formal / informal learning
Curriculum development
Open Educational Resources/OER Initiatives/OpenCourseware
Other

17 | P a g e
How familiar are you with Open Educational Resources
It is one of my primary fields of expertise
I am somewhat familiar with OERs, but would appreciate further input
I am not familiar with OERs and feel the need for an introduction to the field.
Comments
If there is anything you would like to share with the OERtest team before the international
meeting in Bonn, please add here:

18 | P a g e
Annex 2
Form Initial Feedback from Partner HEI Experts

19 | P a g e
Recruitment Meetings – Local coordinator feedback
What was the general view with regards to the feasibility of the planned OERtest activities?
Very negative
Somewhat negative
Somewhat positive
Very positive
Further comments
What feedback did invited experts provide on the suggested key questions to be discussed in
the international meeting?
Further questions suggested
Questions perceived as irrelevant or questions that should be less prioritized
Please add further comments
What hindrances do you immediately perceive? Which issues should be dedicated sufficient
time during the international meetings to achieve the OERtest project goals?
What further decision makers of your university should be involved now or later during the
run of the project in order to reach the goal of a tested assessment framework for OERs
among our partner universities?