Can Performance-Based Funding Enhance Diversity in Higher Education Institutions?

Abstract

Content Overview and Test-Reading in URL: http://tinyurl.com/obshy3t

Full text

RENÉ KREMPKOW
CAN PERFORMANCE-BASED FUNDING
ENHANCE DIVERSITY IN HIGHER EDUCATION
INSTITUTIONS?
In addition to producing knowledge through research activities, higher
education systems and their institutions are shaped by the socio-political task of
guaranteeing the transfer of knowledge via academic teaching. Some recent and
expected developments have led to the discussion of ways to more thoughtfully
address a diverse (potential) student population: The anticipated shortage of skilled
labor intensifies discussions about teaching in the areas of science, politics and
higher education research. This discussion suggests that to an ever-increasing
degree, students with varying conceptual backgrounds, individual objectives,
qualifications and capabilities will be entering higher education institutions
(HEIs).
1
Consequently, any discussion of these developments in the higher
education system implies at least two questions: First, do we expect every HEI to
meet the specific needs of each and every student, or do we expect that, in addition
to the differences among universities and universities of applied sciences, there
will be other types of HEIs that develop specific offerings for specific target
groups? Second, how can we consider the effects of a particular institutional design
on the successful graduation of students from specific HEIs?
In this paper
2
, we present a possible way to evaluate the diversity of
higher education systems at the institution level and to consider such diversity in
terms of performance ratings. In the first section, we explain the underlying
concept of diversity and illustrate the connection between the various types of
HEIs, particularly universities and universities of applied sciences. Furthermore,
we provide examples of using the social origins of students in the German higher
education system to evaluate diversity. In the second section, we introduce a model
that has been in use for more than a decade in Australia but has rarely been adopted
in Europe.
3
It provides a well-established option for recording differences among
HEIs, using a performance indicator-based statistical balancing method that
considers the composition of the student body. A calculation example shows that
universities with many students from low socio-economic backgrounds (SEB) can
reach performance levels that are almost the same as those of universities with
many students from more comfortable backgrounds. In this case, universities with
a high percentage of lower-SEB students produce greater “added value” in relation
to their initial conditions. Based on the results, these HEIs should be supported
(financially) to enable them to react adequately to the specific needs of their
students, so that students can be successful in their studies regardless of their
individual circumstances (DETYA 1998).
In section three, we weigh the pros and cons of the added-value approach
and discuss whether it might be worthwhile to adapt it for use in the German
system.

RENÉ KREMPKOW
2
1. DIVERSITY IN THE GERMAN HIGHER EDUCATION SYSTEM
The alternative approaches to describing the variety of HEIs and the
subsequent procedures for distributing funds, which are discussed in this paper, are
both based on the assumption that modern higher education systems need to fulfil a
multitude of needs for different target groups. For that reason, different types of
higher education institutions (HEIs) have been developed, constituting systems
characterized by a more or less extensive institutional diversity.
Institutional and/or external diversity in a higher education system is
defined not so much in terms of internal diversity as in terms of variance among
the HEIs at a specific point in time (cf. van Vught 1996: 44; van Vught et al. 2010:
11). Diversity may be achieved via diversification (Goedegebuure et al. 1996: 5).
The diversity of higher education institutions may refer to aspects such as the
organizational structure; to procedural aspects such as the execution of teaching
and research; or to differences in the organizational culture and the orientation
toward (differently) defined target groups (van Vught 2009: 1). These pertain to
horizontal diversity among HEIs, and the point of reference is the institutions’
different objectives and their implementation. At the same time, there may be
differences among the higher education institutions in terms of performance and
reputation. These differences can be defined as vertical diversity (Teichler 2005:
65, 99) which has most recently been promoted in Germany by the Excellence
Initiative (Neidhardt 2010: 57). The advantages of an institutionally diversified
higher education system have been summarized as follows: “Diversified higher
education systems are believed to produce higher levels of client-orientation (both
regarding the needs of students and of the labour market), social mobility,
effectiveness, flexibility, innovativeness, and stability” (van Vught et al. 2010: 12;
also cf. van Vught 1996).
Institutional diversity in a higher education system is, at least implicitly,
expected to have an integrative effect at the student level. With a variety of
different types of HEIs with different teaching profiles and study programs, a
system can be expected to meet better the specific needs of various groups of
students and thus affect the diversity of the student body (van Vught 2009: 4-6).
Against this backdrop, we will illustrate some connections between the type of HEI
and the composition of the student body.
1.1 Differences between universities and universities of applied sciences
The politically supported differences between universities and universities
of applied sciences may be considered a central development in the movement
toward institutional diversity in Germany. In the late 1960s, we witnessed the
introduction of a newer type of HEI, the universities of applied sciences
(Fachhochschulen (FHs), to supplement the existing general universities. This was
meant to help accommodate the increasing numbers of students seeking enrollment
as the higher education system transformed into a mass system. The FHs had the
major objective of providing students with job-oriented academic training as

CAN PERFORMANCE MEASUREMENT AND PBF ENHANCE DIVERSITY OF HEI?
3
opposed to the more research- and teaching-oriented focus of the traditional
universities. Because less research work was conducted at FHs, their introduction
resulted in a horizontal (formal) differentiation of university types in Germany.
4
Since the end of the 1980s, however, we have witnessed a trend towards
further differentiation in the academic system. At the universities, students
increasingly demand that their courses be more oriented towards practice. The FHs
have developed research skills, initially only in the field of applied research but are
now moving into other fields (also of more basic research). With the conversion of
the programs to modular courses, the two types of institutions have become more
similar in the field of teaching (ibid.: 446-449). Additionally, universities of
applied sciences are currently gaining competence in promoting young academics,
particularly by cooperating with universities to award doctorates. But at present
they are not allowed to award PhDs independently. Because of these developments,
other (informal) criteria for differentiation between the different types of HEIs are
gaining importance. The universities and Fachhochschulen can be distinguished by
differentiating between the Bachelor’s, Master’s and doctoral degrees
5
; by
examining different (teaching) profiles as a kind of (horizontal) differentiation; and
by the (vertical) differentiation between universities that have successfully applied
their future concepts within the Excellence Initiative (ibid.; Wissenschaftsrat 2010:
22-24).
Many HEIs, however, remain oriented toward the model of a research
and/or “Excellence University”. This focus is particularly successful in the global
rankings because the criteria are known (cf. Wende and Westerheijden 2009: 71),
and highly ranked universities have better reputations in the academic community.
This focus is supported by performance-based incentive and funding systems
(PBFs) which often regard research success more highly than the performance of
teaching tasks (cf. König 2011). Consequently, there may be convergences in
university profiles that involve the risk of reducing differentiation and encouraging
the majority of HEIs in the direction of research. For that reason, the German
Council for the Sciences and Humanities (Wissenschaftsrat 2010, p. 116) wrote
that displays of diversity may end up producing similarities.
In the Wissenschaftsrat´s November 2010 recommendations on the
differentiation of HEIs in Germany, the Council supports the idea of developing
new types of HEIs to supplement the typical German types (in particular,
universities and Fachhochschulen). The new types of HEIs are to reflect a wide
variety of institutional objectives, organization types and tasks (cf.
Wissenschaftsrat 2010: 66-67).
Along with the differences between universities and FHs established in
the recent past, additional changes have occurred in the institutional environment
of the German higher education system. The academies of cooperative education
(Berufsakademien), which offer a combination of vocational training and
theoretical studies, are traditionally not part of the higher education system in
Germany. However, the Berufsakademien are institutions that could be considered
potential competitors against the established university types in Germany
(Krempkow & Pastohr 2009). The eight Berufsakademien in Baden-Wurttemberg

RENÉ KREMPKOW
4
were united in 2009, and as the Baden-Wurttemberg Cooperative State University
(Duale Hochschule Baden-Württemberg, DHBW), they were institutionally
upgraded to a university (BMBF 2010: 66).
1.2 Diversity of the students in accordance with the university types
Institutional diversity is accompanied by differences in the composition of
the student body. This can be illustrated by the social survey of the German
National Association for Student Affairs (Deutsches Studentenwerk, BMBF 2013;
2010). According to this survey, 96 percent of the students at universities had a
general university entrance qualification (Abitur); at Fachhochschulen, the rate was
only 57 percent (BMBF 2013: 56); and the proportion of students who had
accomplished vocational training was 42 percent, compared with 13 percent of
university students (BMBF 2013: 58). On average, the parents of the university
students have higher qualification levels compared with the parents of FH students.
Fifty-five percent of the first-degree students at a university had at least one parent
with a university degree compared with only 38 percent of the first-degree FH
students (BMBF 2013: 83). These figures are confirmed by another student survey
that has regularly been conducted by the Higher Education Research working
group (AG Hochschulforschung) of Konstanz University at universities and FHs in
Germany since winter of 1982/83. According to that study, in 2010, 58 percent of
the students at universities had at least one parent with a university degree,
compared with 40 percent of the students at universities of applied sciences
(Multrus, Ramm and Bargel 2011).,
In the social survey (BMBF 2013), social backgrounds are also classified into
four groups according to the parent with the highest occupational position, using
the presence or absence of a university degree as the control criterion. This
categorization provides the basis for social origin groupings: lower, middle, upper
middle and upper. Here again, students from the university have a higher social
origin, on average, than do FH students. At the universities, 27 percent of students
come from the upper social origin group, 29 percent come from the upper middle
group, 37 percent come from the middle group, and 7 percent come from the lower
group. At the FH, the students’ social origin groups are as follows: upper, 13
percent; upper middle, 25 percent; middle, 50 percent; and lower, 12 percent (ibid.:
95). These data can be summarized as follows: The Fachhochschulen confirm their
reputation as educational institutions that are of particular interest to those who are
interested in studying, yet come from social strata in which there is not much
contact with higher education (ibid.). Accordingly, universities of applied sciences
are likely to admit students who are not members of the classic target group of a
system that focused on supporting the elite, as was very true of the German higher
education system until the 1960s.
6

CAN PERFORMANCE MEASUREMENT AND PBF ENHANCE DIVERSITY OF HEI?
5
1.3 Differences in student diversity among single higher education
institutions
A further question, and one that is only partly answered for Germany,
results from the systematic differences among the compositions of the student
populations of each single HEI: Do the research-oriented, traditional HEIs attract
more traditional “elite” students because these HEIs receive more financial
support, while more regional HEIs are left with the less traditional students? In a
former article we described big differences in the social background of students
between the group of German “elite”-universities (the winners in the German
Excellence Initiative) and “normal” (in the meaning of regular) universities (Kamm
and Krempkow 2010, 2013).
Furthermore we asked, do those institutions admitting specific groups of
students who are not considered the future academic elite face an additional
(financial) disadvantage because of their admission practices?
We assume that differences in the success ratios for graduates (ratios
similar to the OECD completion rates, cf. Krempkow, 2010; 2007) depend to some
extent on the students’ socioeconomic backgrounds. This relationship has been
found in other countries (cf. Dill 2009; Dill and Soo 2005)
7
, and already confirmed
in Germany (Kamm and Krempkow 2010: 75). In regression analyses we found a
significant relationship: that HEIs with more traditional students have higher
success ratios, despite of the control of other factors like university entrance
qualifications and study quality. Greater diversity in this context can only be in the
interest of the university leaders if the differences in the student body in the future
will not incur financially negative consequences.
8
Therefore, we will discuss the
added-value approach for adjusted indicators that can potentially solve this
problem in the following section.
2. THE ADDED-VALUE APPROACH: ADJUSTED INDICATORS FOR
DIFFERENT INITIAL CONDITIONS
As explained above, research orientation has the best reputation in the
academic community, and PBF systems mostly rate research success higher than
the completion of teaching tasks. PBF may include a multitude of different
indicators
9
for various university performance areas and can be used as a basis for
separating the allocation of funds in so-called multi-circle models via the afore-
mentioned university types.
10
At present, however, the different initial conditions
of HEIs in institutionally differentiated systems have rarely been considered when
calculating performance indicators in Germany.
11
The Australian model of adjusted indicators, which was introduced in
1998 for the performance-oriented allocation of funds from the Learning and
Teaching Performance Fund, and is currently not well known in Germany,
provides a different picture. The indicators used in this model include the share of
non-native (English) speakers, the students’ socio-economic status, the number of
part-time and full-time students (type of enrolment), students’ genders and ages,

RENÉ KREMPKOW
6
the population density in students’ areas of origin, the students’ cultures, the types
of courses offered, the required admission qualifications, and the students’
indigenous Australian status. To calculate the performance indicators, their
relevant specific predominance in their institutions is considered (DETYA 1998:
70). The following considerations led to the development of this model: “The
simplistic use of performance indicators can produce misleading impressions of
institutional performance. Institutions have diverse missions, backgrounds, course
offerings and students” (ibid.).
12
For that reason, a method was developed to
balance the effects of various factors. Regression analyses were conducted, and
only the significant influential factors
13
were considered for the adjustment at the
institutional level.
14
Essentially, the approach adopted in the Australian model for
indicator adjustment is a comparison of institutional performance with a set of
national values regarding the composition of the student body (DETYA 1998).
This approach may be transferrable to German HEIs, if the relevant data are
available (for application of the approach to German Federal State, see Krempkow
and Kamm 2012). When considering the composition of the student body and
before the indicator adjustment is demonstrated via a calculation example, the first
question to be answered is whether there are palpable differences among German
HEIs in terms of the social origin of students
2.1 Selected differences in the composition of the student body among HEIs
in Germany
The institutional diversity of HEIs in Germany may be illustrated via
selected features of the student body composition
15
that have been ascertained
using secondary analyses of existing data sets (cf. Bargel et al. 2011). The share of
students whose parents have no university degree
16
, a feature frequently used to
identify the students’ social (or, more precisely, educational) origin, varies in the
available data
17
from approximately 65 percent (Kassel, Duisburg-Essen,
Oldenburg, Bochum) to approximately 40 percent (Freiburg, TU Berlin, LMU
Munich, Leipzig). These results partly depend on the subject combinations,
because students with lower social background more often choose subjects like
social sciences or business administration, while students with higher social
background often choose medicine (cf. BMBF 2013; Bargel et al. 2011). But they
probably also depend on other factors (such as the HEI’s location and/or the
recruiting potential) because even within the same HEI, there are differences. In
sociology, the share of students whose parents are not university graduates ranges
from approximately 70 percent (Kassel, Duisburg-Essen, Rostock, Bochum) to
approximately 40 percent (Freiburg, followed by TU Berlin, Potsdam, Leipzig). It
is probably no coincidence that the locations are very similar in each case.
The differences in the proportion of part-time students, which are more or
less equivalent to the “type of enrolment” indicator in Australia, are also
comparatively large. According to the available data, the proportion of part-time
students ranges from approximately 15 percent (Freiburg, followed by TU
Dresden, Karlsruhe) to 35 percent (Duisburg-Essen, Frankfurt/Main, Hamburg). In

CAN PERFORMANCE MEASUREMENT AND PBF ENHANCE DIVERSITY OF HEI?
7
sociology courses alone, the proportion ranges from 19 percent (TU Dresden,
followed by 27 percent at the TU Berlin and 30 percent at Freiburg) to 60 percent
(Frankfurt/Main, which differed by only a small margin from Hamburg and by a
larger margin from the 40 percent rate at Bochum). Again, it is striking that from
both the cross-subject point of view and the subject-specific point of view, most of
the universities had similar proportions of part-time students (some, such as
Karlsruhe, do not offer sociology). The comparison could be continued for other
subjects in a survey that specifically focused on that point. In general, there is
definitely some diversity in terms of the composition of the student body at
German HEIs. Consequently, the transfer of the Australian model to Germany
might be worth considering.
2.2 The four essential steps of the calculation method within the model
In the following, the four essential steps of the model are explained using
two fictitious HEIs as examples (following the calculation example in DETYA
1998: 71).
18
As table 1 illustrates, institution 1 has a small share of students with a
low socio-economic background (20 percent); in contrast, institution 2 has a high
share (70 percent). Table 2 shows a higher uncorrected success ratio for institution
1 and illustrates the expectation that subgroups of students with low socio-
economic backgrounds have lower success ratios. In table 3, the calculation of the
expected success ratio of both institutions is presented, and the different initial
conditions are taken into consideration. Given the national set of performance data
and the composition of the student body, institution 1 should have a success ratio
of 85 percent which would exceed the 82 percent achieved. Therefore, in the fourth
step – shown in table 4 – the difference in the crude and expected success ratios
(“adjusted performance”) is calculated. The adjusted performance is also
considered “added value”. For HEIs with disadvantageous initial conditions, the
adjusted performance represents the value that can be added to the expected value.

RENÉ KREMPKOW
8
Table 1. Initial conditions: Share of "Low socio-economic background
status" (SEB) versus "other SEB"
Institution 1 Institution 2 Total
Low SEB 20 percent 70 percent 45 percent
Other SEB 80 percent 30 percent 55 percent
Table 2. Success ratio as a crude performance indicator by institutions and
sub-groups
Institution 1 Institution 2 Total
Low SEB 70 percent 75 percent 74 percent
Other SEB 85 percent 95 percent 88 percent
Total 82 percent 81 percent 81.5 percent
Table 3. Expected success ratio (Exp. SR) based on the example of
Institution 1
Exp.SR= Low SEB-
share1 *
Low SEB
perf. +
Other SEB-
share1 *
Other
SEBperf.
Exp.SR= 20% * 74%
+
80% * 88%
Exp.SR= 85 %
Table 4. Adjusted performance indicator as the difference crude – expected
success ratio
Institution 1 Institution 2 Total
Total exp.
SR
85 percent 78 percent 81.5 percent
Diff. cr.-
Exp.SR
= adj. perf.
82
- 85
= - 3 percent
81
- 78
= + 3 percent
0 percent
The “adjusted performance” values resulting from the above calculation
method indicate what the results would be like for the institutions if only the “low
SEB” proportions were considered influential factors for the adjustment (cf. table
4). In this case, the ratio differs from the one that results when the SEB proportions
are not considered: Institution 1, which clearly has fewer low-SEB students, has a
negative value (-3) because of the higher expected success ratio, while Institution 2
has a positive value (+3) because of the lower expected success ratio.
The calculation example shows that even for large differences in the SEB
distribution, the adjusted performance values remain in the single-digit percentage
area. Because the intent was to adjust the existing performance rating and incentive
systems and not to create new incentives to change the composition of the student
body, short-term changes in the student body composition would have less of an
effect than changes in the success ratios would, as intended. Major changes
resulting from a different student body composition might result from adjustments

CAN PERFORMANCE MEASUREMENT AND PBF ENHANCE DIVERSITY OF HEI?
9
only when locations simultaneously showed clearly less favorable initial conditions
compared with the rest of the country in terms of several factors influencing the
adjustment.
In addition to the SEB status, the original and more comprehensive
Australian model calculated eleven influential factors. Later, the calculation was
performed with a simplified model using four influential factors with almost
identical results.
19
The calculations used as an example here were conducted for a
total of 43 HEIs in Australia. Some institutions that showed higher-than-expected
success ratios despite less favorable initial conditions were allocated higher-than-
average funds. This is the core of the approach, because this shows that the HEIs
were rewarded for adding value. There is of course the interesting and
pedagogically fascinating question: how did the HEIs achieve this? We shall come
back to this question in the discussion. Several institutions suffered minor losses,
but for many institutions, there were hardly any differences in funding (DETYA
1998; Krempkow 2010).
2.3 External reviews of the model
The Australian model of adjusted indicators underwent an external review
in 2005. While the suitability of some individual performance indicators was
sharply criticized and the advancement of these indicators was recommended, the
overall concept indeed received a positive assessment:
"Access Economics found that the overall concept (…) attempting to
create a ‘level playing field’ by removing differences in university performance
due to exogenous factors (such as the age and gender mix of students) is a sensible
and fair approach. The set of exogenous variables used is also sensible and covers a
good range of social and demographic factors that are beyond the control of the
institutions. [It] has also been careful to exclude any factors that are within the
control of a university." (Access Economics 2005: 4).
20
Another analysis of the model resulted in the conclusion that even if the
distributed amounts are comparatively small, the model – with its indicators and
their relative weights – still has the potential to develop strong incentives for the
institutions' policy inter alia as a result of public discussions of the performance
comparisons (Harris 2007: 69).
3. DISCUSSION AND PROSPECTS: IS IT WORTHWHILE TO ADAPT THE
AUSTRALIAN MODEL TO GERMANY?
This contribution is based on the argument that there is a connection
between systems for recording and rating university performance and incentives
such as PBF in research and teaching areas on the one hand; and institutional
diversity and the degree of variety in the student body on the other. Although
rankings do not counterbalance the fact that indicators not originally designed for
that purpose will be considered directly in the PBF, it seems rational for individual
institutions to follow the example of successful institutions to secure their existence

RENÉ KREMPKOW
10
permanently. As argued in Section 1, this approach could result in a reduction in
institutional diversity if all HEIs strive to be accepted in the particularly attractive
best-performer group of research excellence universities. This approach could also
have a problematic effect on the diversity of the student body if all HEIs prioritize
a research orientation over an orientation toward different student target groups.
Second, the development of HEIs might yet turn towards an increase in the number
of institutional types to be considered. This evolution would be expected if
additional student characteristics are considered in performance ratings, and
individual institutions are encouraged to focus on certain groups of students and to
adjust their teaching contents and organizational offerings to these students'
demands. For example, one interesting potential target group might be non-
traditional students who are considered in HEI performance ratings in the UK.
21
At present, there is no accepted and practicable solution for specifically
dealing with the very different initial conditions of the HEIs in Germany. This lack
might be one reason for the relatively low acceptance and the many unintended
effects of PBF in Germany. Previous research on the effects of PBF shows that
such funding achieves its objectives only to a limited extent (cf. the contributions
in Grande et al. 2013; Wilkesmann and Schmid 2012; Winter and Würmann 2012).
This limitation even applies under relatively comfortable conditions for PBF in
academia, as in the case of university medical departments in Germany (cf.
Krempkow et al. 2012, 2013). Therefore, this paper considers the application of the
Australian model. It has the potential to increase the acceptance of performance
ratings, and it avoids possible classification problems by referring each institution’s
performance to its initial conditions without having to group the HEIs in advance.
22
This adjustment would include a higher degree of transparency and, via the actual
performance, would consider the “added value” of higher education in the
performance rating and the PBF. Value would be added when HEIs with a student
body composition that is not conducive to high success ratios have better-than-
expected success ratios.
23
According to initial empirical analyses, higher-than-
expected success might be achieved via an improved quality of courses and the
promotion of subject specific and social competences of graduates (cf. the
assumption based on empirical findings in Krempkow et al. 2010: 57, and the
significant effects of these aspects for success ratios reported in Kamm and
Krempkow 2010: 76). Improved quality of teaching and the promotion of
competencies have been important objectives of the Bologna process, and these
objectives have been assigned a higher priority than before – in terms of both
political attention and funding – by the federal government’s most recent Bologna
summits and by the BMBF promotion initiatives (such as the 2 billion Euro spent
to improve the quality of teaching and studies under the Higher Education Pact
2020). Therefore, in Germany an adaptation of the Australian model of
performance indicator adjustment in r and of the concomitant PBF of German
Federal States (Länder) might indeed provide effective flanking support for
achieving the objectives of improved course delivery quality and promotion of
competencies. A version of the Australian model would also allow consideration of

CAN PERFORMANCE MEASUREMENT AND PBF ENHANCE DIVERSITY OF HEI?
11
the different initial conditions of the HEIs, unlike the overarching promotion of
HEIs’ diversity efforts in the Berlin PBF model.
The aspects mentioned here refer to the obvious risks and opportunities of
the added-value approach which may raise some concerns. Will it be possible for
this approach to gain acceptance via illustration of the relevant statistical
calculation methods? And how is it possible to combine approaches such as the
classification approach and the added-value approach, in a way that has a positive
effect on diverse university profiles?
NOTES
1
Cf., for example, for the expected student enrolment in Germany in times of
increasing heterogeneity of student populations Krempkow and Dohmen (2014).
2
This article is based on a previously prepared text (Krempkow and Kamm 2011)
that has been revised and updated.
3
There exists another option for recording institutional diversity developed in the
European context: the so called “U-Map classification” of European HEIs, according to the Carnegie
Classification in which HEIs are classified in different classes and/or types, following certain criteria
(cf. Carnegie Foundation, 2014; Wissenschaftsrat 2010). (“U” is an abbreviation for “university”; in
the German context, “U” also stands for universities of applied sciences.) For further information,
see the project home page: www.u-map.eu or Mahat et al., 2014; or for a publication more focused
on the strengths and weaknesses of the classification approach and the added-value-approach
(Krempkow and Kamm 2014).
4
Universities of art and music (mostly called “Akademien” in Germany) and
universities of teacher education (“Pädagogische Hochschulen”, still existing in Baden-
Württemberg) are additional HE types in Germany; however, their impact is negligible because of
their small size. Universities of technology, which had been a separate type of university subordinate
to the general universities until the end of the 19th century, were included in the circle of
universities in the 20th century (Enders 2010).
5
Cf. e.g., the UAS7 association of seven German universities of applied sciences,
which welcomed the German Science Council’s (Wissenschaftsrat 2010) recommendation to
support the further differentiation of the higher education system and to grant some universities of
applied sciences the right to award doctorates.
6
In addition to the differences in the composition of the student community
between the two major types of HEIs in Germany, there are general distinctions superimposed on
those differences, such as the distinctions between students in East and West Germany, between
female and male students, between students with and students without children, and/or students with
and students without a migration background (BMBF 2010, 2013).
7
Dill (2009) found that is that the overall quality of students entering a university
has an independent influence on graduates’ success. Furthermore Dill formulated that simply
admitting higher ability students will not necessarily increase the learning of all and that in support
of this point a recent study (Kuh and Pascarella, 2004) on the relationship between admissions
selectivity and the presence of educational practices known to be associated with student learning in
the US confirms that they are largely independent. In an earlier study Dill and Soo (2005) analyzed
the validity of measures used in the commercial league tables in Australia, Canada, the UK, and the
US. Their result is that league table rankings are heavily biased toward measures known to be
associated with research performance including financial resources, numbers of faculty and research
activity, student selectivity, as well as university reputation.

RENÉ KREMPKOW
12
8
Dill (2009) reminds us that a national study of the US higher education market
researchers at the Rand Corporation (Brewer et al., 2002) discovered that many institutions are
attempting to alter their standings in university rankings by “cream skimming” the student market.
9
In the PBF systems in the German Bundeslaender in the last years were used up
to 11 indicators, including the number of graduates, or the number of graduates in relation to the
number of study beginners (as success ratio similar to the OECD completion rate). Although the
weightings for single indicators of research performance often were higher than the indicators of
teaching performance, the indicator weighting for graduates or success ratios were relatively high
(11 percent up to 50 percent in PBF models with only a few indicators).- cf. Dohmen et al. (in
preparation).
10
In recent years, there has been a PBF system in almost every German Bundesland
(cf. König 2011). However, this PBF distributes potentially very different shares of the overall
budget; according to König (2011), these shares range from two percent in Saxony to 25 percent in
Bavaria. In other Bundesländer, the share is clearly higher, too (Baden-Wurttemberg and North
Rhine Westfalia receive 20 percent; several other Bundesländer receive 15 percent). In the
meantime, that share has increased again in some of the abovementioned Bundesländer and in
others, too (Berlin, e.g., which has a share of 30 percent).
11
The supporter of the classification approach or the U-Mapping also try to avoid
simplistic measures. However, the added-value-approach calculates the (adjusted) performance in
the area of teaching in relation to the composition of the student body at the HEIs.
12
Finland, which offers boni for schools in socially underprivileged areas, and the
UK, which provides “special funding for ‘high-risk’ students with a statistically high propensity to
drop out” (Sörlin 2007: 422) are exceptions. Some years ago, the UK also used the term of measures
taken for “non-traditional students”. The most recent Berlin system of performance-based university
funding is an exception in the German higher education system. Here, diversity is considered
explicitly, for example, by crediting higher education institutions an additional € 10,000 for each
new student who has a migration background or who comes from an applicant groups without
“Abitur” (the German university entry qualification) but who are qualified because of the trade they
have learned (Senatsverwaltung für Bildung, Wissenschaft und Forschung Berlin 2011).
13
These influencing factors include age, gender, non-English speaking background
(NESB) status, indigenous Australian status, socio-economic status, rural status, isolated status,
broad field of study, level of course, basis of admission and type of enrollment (cf. DETYA 1998:
70).
14
In France, CĖREQ (2009) conducted regression analyses and a simulation of a
similar PBF procedure. A similar regression analysis was also conducted at higher education
institutions in Germany, cf. Kamm and Krempkow (2010). The influencing factors were gender,
broad field of study, socio-economic status, and type of enrollment (cf. ibid.; Kamm and Krempkow
2013). In an adapted version of this paper, the authors exemplarily transferred a simplified version
of the Australian model to the universities of one German Federal State (Krempkow and Kamm
2012).
15
In Germany, the influence of the social and educational background on the results
of the PISA surveys and similar studies has long been discussed. Many publications show a strong
relationship between both aspects and indicate that they are under the control of other influencing
factors (cf. e.g., OECD 2013: 40, Lehmann and Lenkeit 2008: 42). These findings have led to a
calculation of adjusted mean performance (after taking socio-economic status into account). Our
paper addresses a very similar issue: What would be the average performance if all students had the
same socio-economic status? A figure in the OECD (2013: 42) publication shows that some
countries, e.g., Portugal, Turkey and Vietnam, perform much better in the adjusted condition.
16
The proportion of students whose parents are not university graduates was
calculated using the variable “father’s educational degree combined with vocational qualification” in
Bargel et al. (2011); the qualifying locations remain almost the same when the highest educational
degree of both parents is used. The calculation was based on the last four waves of a survey

CAN PERFORMANCE MEASUREMENT AND PBF ENHANCE DIVERSITY OF HEI?
13
conducted at 17 representative universities and ten universities of applied sciences throughout the
Federal Republic of Germany. At 14 of these university locations, studies in sociology are offered.
At those universities, more than 20 interviewed students answered the questions about their
educational origin (exception: Duisburg-Essen: 18 students; however, this location was included
only in the most recent surveys.). Information about educational origin was available for 33,175
students, including 665 sociology students. Very few of the interviewed students (only ten of the
sociology students) provided no relevant information.
17
If the universities of applied sciences had been included, the range would have
been even wider.
18
The example is not based on real HEIs, but it illustrates the basic function of the
approach. Therefore, institution 1 and institution 2 in this calculation example have an equal size.
19
Some time ago, another advanced version of the model was initiated, but the
relevant results were not known when this paper was submitted.
20
The question of which influencing factors must be incorporated into outcome
comparisons in studies on quality and the output/outcome of teaching and learning has a long
tradition. The Access Economic report represents the main features of a typical meta-analyses in this
field. It brings forward the argument that only “external” influences independent of teaching and
learning should be accounted for in outcome comparisons. At the same time, conditions that can be
influenced by actors should not be regarded as potentially distorting “bias variables” and should
therefore not be incorporated into the indicator adjustment calculation. The multitude of studies on
this subject cannot be addressed in detail in this paper (for a detailed discussion of this topic, cf.
Krempkow 2007: 145).
21
Here is a potential risk of “gaming by numbers”: If the necessary data consist of
soft information (e.g., parents’ school degrees), university administrations might have an incentive
to inflate the numbers of this group of students. To our knowledge, no misuse of data has been
reported to date in the UK. Nevertheless, to avoid potential misuse, the indicator adjustment should
combine multiple aspects (as the Australian model does).
22
The German PBF usually distributes money only to universities and universities
of applied sciences (because of different research shares). Consequently, adapting an Australian
scheme would not automatically merge the 2-tier German system into one tier.
23
Of course, it is not possible to solve other problems that are immanent to the
performance rating just by means of indicator adjustment. In particular, we refer to the possible
unintended effects of a PBF systems with few indicators that may be easily manipulated, are highly
oriented towards quantity, and offer little incentive for promoting or at least securing quality (for
more details, cf. Krempkow 2007). To process this problem, either an stronger quality assurance or
indicators that are (supposed) to more accurately record the quality of the provided performance
were introduced in nations with a longer PBF experience. For the latter objective, both the
Australian experience and the Swiss indicator developments based on graduate studies may be
helpful for continuing the discussion (for a relevant overview, cf. Krempkow 2009: 49 et seq.). For
some subjects who have taken exams hat are the same throughout a state or even throughout the
Federal Republic, examination marks may be worth discussing. In the long run, recordings of
university graduates’ skills may be used as a tool that offers the potential for better quality
recordings (cf. the international AHELO project).
ACKNOWLEDGEMENTS
I gratefully acknowledge the cooperation of Ruth Kamm, Kiel University. I
am also indebted to many colleagues and participants of the EAIR Annual Forum
2014 and the 2013 annual conference of the German speaking Higher Education
Society (GfHf) for their advice about former versions of this contribution.

RENÉ KREMPKOW
14
REFERENCES
Access Economics (2005). Review of Higher Education Outcome Performance Indicators. Report by
Access Economics.
Bargel, E.T., Multrus, F. and Ramm, M. (2011). Forschungsprojekt Studiensituation, Datensätze der
Erhebungen zum 7-10. Konstanzer Studierendensurvey. Konstanz: Arbeitsgruppe
Hochschulforschung der Universität Konstanz.
Brewer, D., Gates, S.M. and Goldman, C. A. (2002). In Pursuit of Prestige: Strategy and Competition
in US Higher Education. New Brunswick: Transaction Press.
Bundesministerium für Bildung und Forschung (BMBF) (2010). Die wirtschaftliche und soziale Lage
der Studierenden in der Bundesrepublik Deutschland 2009. 19. Sozialerhebung des Deutschen
Studentenwerks durchgeführt durch HIS Hochschul-Informations-System. Berlin: BMBF.
Bundesministerium für Bildung und Forschung (BMBF) (2013). Die wirtschaftliche und soziale Lage
der Studierenden in der Bundesrepublik Deutschland 2012. 20. Sozialerhebung des Deutschen
Studentenwerks durchgeführt durch das HIS-Institut für Hochschulforschung. Berlin: BMBF.
Carnegie Foundation (2014). The Carnegie Classification of Institutions of Higher Education. Retrieved
21 Feb 2014 from http://classifications.carnegiefoundation.org.
CÉREQ (2009). Comparer les universités au regard de l’insertion professionnelle de leurs étudiants.
Net.doc 54, Strasbourg: Centre d´etudes et de recherches sur les qualifications.
CHEPRA-Network (2011). U-Multirank. Design and Testing the Feasibility of a Multidimensional
Global University Ranking. Final Report. CHEPRA-Network (Consortium for Higher
Education and Research Performance Assessment). Retrieved 30 June 2014 from
http://ec.europa.eu/education/library/study/2011/multirank_en.pdf.
DETYA (1998). Department of Education, Training and Youth Affairs (1998). The Characteristics and
Performance of Higher Education Institutions, Occasional Paper Series 98-A.
Dill, D. D. (2009). Convergence and diversity: The role and influence of university rankings. In: B. M.
Kehm & B. Stensaker (eds.), University rankings, diversity, and the new landscape of higher
education (pp. 97–116). Rotterdam: Sense Publishers. Retrieved 12 Nov. 2014 from:
www.unc.edu/ppaq/docs/Convergence.pdf
Dill, D. D. and Soo, M. (2005). Academic quality, league tables, and public policy: A cross-national
analysis of university ranking systems’, Higher Education, 4, 495-533.
Dohmen, D., Cleuvers, B., Henke, J., Köller, M., Kamm, R., Krempkow, R. (in preparation).
Untersuchungen auf Länderebene. In: Dohmen, D. (ed.): QualitAS-Lehre. Theorie und Praxis
von Anreiz- und Steuerungssystemen im Hinblick auf die Verbesserung der Hochschullehre.
Endbericht. FiBS - Forschungsinstitut für Bildungs- und Sozialökonomie. Berlin.
Enders, J. (2010). Hochschulen und Fachhochschulen. In: D. Simon, A. Knie & S. Hornbostel (eds.),
Handbuch Wissenschaftspolitik (pp. 443-456). Wiesbaden: VS Verlag.
Goedegebuure, L., Meek, V.L., Kivinen, O and Rinne, R. (1996). On Diversity, Differentiation and
Convergence. In: Meek, V. L., Goedegebuure, L., Kivinen, O. and Rinne, R. (eds.). The
Mockers and Mocked: Comparative Perspectives on Differentiation, Convergence and
Diversity in Higher Education. Oxford/New York: IAU Press, 2-13.
Grande, E., Jansen, D., Jarren, O., Rip, A., Schimank, U. and Weingart, P. (eds.)(2013). Neue
Governance der Wissenschaft: Reorganisation, Externe Anforderungen, Medialisierung.
Bielefeld: transkript.
Heinrich-Böll-Stiftung (2011). Öffnung der Hochschule. Chancengerechtigkeit, Diversität, Integration.
Dossier. Retrieved 31 January 2014 from http://heimatkunde.boell.de/dossier-oeffnung-der-
hochschule-chancengleichheit-diversitaet-integration.
Harris, K.-L. (2007). A critical examination of a recent performance-based incentive fund for teaching
excellence in Australia. In: B. Longden & K. Harris (eds.), Funding Higher Education: A
Question of Who pays? EAIR-Monograph Nr. 2. Amsterdam, 62-78.

CAN PERFORMANCE MEASUREMENT AND PBF ENHANCE DIVERSITY OF HEI?
15
Kamm, R. and Krempkow, R. (2013). Wie „gerecht“ ist leistungsorientierte Mittelvergabe für
Hochschulen gestaltbar? In: Knoll, C. (ed.): Gerechtigkeit. Multidisziplinäre Annäherungen an
einen vieldeutigen Begriff. Kassel: Athena, 129-144.
Kamm, R. and Krempkow, R. (2010). Ist leistungsorientierte Mittelvergabe im Hochschulbereich
„gerecht“ gestaltbar? Qualität in der Wissenschaft, 4, 71-78.
König, K. (2011). Hochschulsteuerung. In: P. Pasternack (ed.), Hochschulen nach der Föde-
ralismusreform. Leipzig: Akademische Verlagsanstalt, 106-154.
Krempkow, R. (2010). „Leistungsorientierte Mittelvergabe und wissenschaftliche Nach-
wuchsförderung“. Workshop Chancengerechtigkeit in der Wissenschaft?, Institut für Hoch-
schulforschung (HoF), 18.-19.11.2010. Wittenberg: Martin-Luther-Universität Halle-Witten-
berg.
Krempkow, R. (2009). Von Zielen zu Indikatoren – Versuch einer Operationalisierung für Lehre und
Studium im Rahmen eines Quality Audit. Qualität in der Wissenschaft, 3, 44-53.
Krempkow, R. (2008). Studienerfolg, Studienqualität und Studierfähigkeit. Eine Analyse zu
Determinanten des Studienerfolgs in 150 sächsischen Studiengängen. Die Hochschule, 1, 91-
107.
Krempkow, R. (2007). Leistungsbewertung, Leistungsanreize und die Qualität der Hochschullehre.
Konzepte, Kriterien und ihre Akzeptanz. Bielefeld: Universitätsverlag Webler.
Krempkow, R. and Dohmen, D. (2014). The prognostic validity of student enrolment prognoses in times
of increasing heterogenity of student populations. In: Schmidt, M. and Bargel, T.: Expansion of
Higher Education. New students, more problems? Hefte zur Bildungs- und
Hochschulforschung Nr. 73, S. 94-100.
Krempkow, R., Landrock, U., Neufeld, J., Schulz, P. (2013). Intendierte und nicht-intendierte Effekte
dezentraler Anreizsysteme am Beispiel der fakultätsinternen leistungsorientierten
Mittelvergabe in der Medizin. Abschlussbericht des Projektes GOMED – Governance
Hochschulmedizin. Berlin: iFQ Berlin. Retrieved 30 June 2014 from:
www.forschungsinfo.de/Projekte/GOMED/GOMED-Abschlussbericht.pdf.
Krempkow, R., Landrock, U. (2012). What Leads PBF to Success? The Meaning of Justice Perception
of Performance Based Funding in German University Medicine. Paper for the EAIR 34rd
Annual Forum. Stavanger, Norge.
Krempkow, R. and Kamm, R. (2014). Can we support diversity by performance measurement of
European higher education institutions? In: Managing Diversity. Management Revue - The
International Review of Management Studies, 227-242.
Krempkow, R. and Kamm, R. (2012). Leistungsbewertung unter Berücksichtigung institutioneller
Diversität deutscher Hochschulen: Ein Weg zur Förderung von Vielfalt? In: Klein, U. &
Heitzmann, D. (eds.), Diversity und Hochschule. Teilhabebarrieren und Strategien zur
Gestaltung von Vielfalt. Weinheim: Juventa, 64-181.
Krempkow, R. and Kamm, R. (2011). Leistungsklassen oder „Added Value“? Zwei Ansätze zur
Berücksichtigung unterschiedlicher Startbedingungen im Wettbewerb von Hochschulen.
Qualität in der Wissenschaft, 5, 115-120.
Krempkow, R., Vissering, A., Wilke, U. and Bischof, L. (2010). Absolventenstudien als outcome
evaluation. Sozialwissenschaften und Berufspraxis, 1, 43-63.
Krempkow, R. and Pastohr, M. (2009). Berufsakademien: Unterschätztes Erfolgsmodell tertiärer
Bildung? Stärken, Schwächen, Chancen und Risiken des dualen Berufsakademiestudiums am
Beispiel Sachsen. Die Hochschule, 2, 71-86.
Kuh, G. D. and Pascarella, E. T. (2004). What does institutional selectivity tell us about educational
quality?, Change (5), 52-58.
Lehmann, R. and H. Lenkeit, J. (2008). ELEMENT. Erhebung zum Lese- und Mathematikverständnis.
Entwicklungen in den Jahrgangsstufen 4 bis 6 in Berlin. Abschlussbericht über die
Untersuchungen 2003, 2004 und 2005 an Berliner Grundschulen und grundständigen
Gymnasien. Berlin (Senatsverwaltung für Bildung, Jugend und Sport. Retrieved 30 June 2014

RENÉ KREMPKOW
16
from www.berlin.de/imperia/md/content/sen-bildung/schulqualitaet/element6_bericht_
komplett.pdf).
Mahat, M., Coates, H., Edwards, D., Goedegebuure, L., Brugge, E. and Vught, F. van (2014). Profiling
Diversity of Australian Universities. In: Krempkow, R., Pohlenz, P. and Huber, N. (ed.):
Diversity Management und Diversität in der Wissenschaft. Bielefeld:
UniversitätsverlagWebler, 229-248.
Multrus, F., Ramm, M. and Bargel, T. (2011). Studiensituation und studentische Orientierungen. 11.
Studierendensurvey an Universitäten und Fachhochschulen. Kurzfassung. Bonn/Berlin:
Bundesministerium für Bildung und Forschung.
Neidhardt, F. (2010). Exzellenzinitiative - Einschätzungen und Nachfragen. In: S. Leibfried (ed.), Die
Exzellenzinitiative. Zwischenbilanz und Perspektiven. Frankfurt/ New York: Campus, 53-80.
OECD (2013). PISA 2012 Results: Excellence through Equity (Volume II).
DOI:10.1787/9789264201132-en.
Rolfe, H., (2003). ‘University strategy in an age of uncertainty: the effect of higher education funding
on old and new universities’, Higher Education Quarterly, 1, 24-47.
Senatsverwaltung für Bildung, Wissenschaft und Forschung Berlin (ed.) (2011). Wissenschaft in Berlin.
Leistungsbasierte Hochschulfinanzierung. Retrieved 31 June 2014 from
www.berlin.de/imperia/md/content/sen-wissenschaft/hochschulen/leistungsbasierte_hoch
schulfinanzierung.pdf
Sörlin, S. (2007). Funding Diversity: Performance-based Funding Regimes as Drivers of Differentiation
in Higher Education Systems. Higher Education Policy, 20, 413-440.
Teichler, U. (2005). Hochschulsysteme und Hochschulpolitik. Quantitative und strukturelle Dynamiken,
Differenzierungen und der Bologna-Prozess. Münster/ New York/ München/ Berlin:
Waxmann.
UAS7 (2010). Hochschulprofilierung im Verbund: UAS7 auf der Linie des Wissenschaftsrats. Press
release dated 26 November 2010. Retrieved 01 June 2014 from idw-
online.de/pages/de/news399106.
Vught, F. van (2009). Diversity and Differentiation in Higher Education. In: Ibid. (ed.), Mapping the
Higher Education Landscape. Towards a European Classification of Higher Education.
Dordrecht: Springer, 1-16.
Vught, F. van (1996). Isosmorphism in Higher Education? Towards a Theory of Differentiation and
Diversity in Higher Education Systems. In: Meek, V. L., Goedegebuure, L., Kivinen, O. and
Rinne, R. (eds.), The Mockers and Mocked: Comparative Perspectives on Differentiation,
Convergence and Diversity in Higher Education. Oxford/ New York: IAU Press, 42-58.
Vught, F. van, Kaiser, F., File, J.M., Gaethgens, C., Peter, R. and Westerheijden, D.F. (2010). U-Map.
Mapping Diversity. Developing a European Classification of Higher Education Institutions.
Enschede: Center for Higher Education Policy Studies, University of Twente.
Wende, M. van der and Westerheijden, D. (2009). Rankings and Classifications: The Need for a
Multidimensional Approach. In: F. van Vught (ed.), Mapping the Higher Education Land-
scape. Towards a European Classification of Higher Education. Dordrecht: Springer, 71-86.
Wilkesmann, U., Schmidt, C. (eds.)(2012). Hochschule als Organisation. Münster: VS – Verlag für
Sozialwissenschaften.
Winter, M., Würmann, C. (eds.)(2012). Wettbewerb und Hochschulen. 6. Jahrestagung der Gesellschaft
für Hochschulforschung in Wittenberg 2011. die hochschule, 2.
Wissenschaftsrat (2010). Empfehlungen zur Differenzierung der Hochschulen (Drs. 10387-10).
Retrieved 10 June 2014 from www.wissenschaftsrat.de/download/archiv/10387-10.pdf.
Zechlin, L. (2014). Multiversity – horizontale und vertikale Differenzierung im Hochschulsystem. In:
Krempkow, R., Pohlenz, P. and Huber, N. (ed.): Diversity Management und Diversität in der
Wissenschaft. Bielefeld: UniversitätsverlagWebler, 225-228.

CAN PERFORMANCE MEASUREMENT AND PBF ENHANCE DIVERSITY OF HEI?
17
AFFILIATIONS
Dr René Krempkow
Institute for Education and Socio-Economic Research and Consulting,
Berlin and the Quality Management Unit at the Humboldt University, Berlin.