No full-text available
To read the full-text of this research,
you can request a copy directly from the authors.
The Measurement, Evaluation, and Publication of Performance in Higher Eduaction: An Analayis of the Che Research Ranking of Business Schools in Germany from an Accounting Perspective
Abstract
Universities do not tend to systematically record their own benefits, leaving it to independent institutions to summarize and publish in the form of ratings and rankings information that such institutions have acquired by themselves. In Germany, for the academic discipline of Business Administration, the Centre for Higher Education (CHE) is the major publisher of such rankings. The challenges in the context of performance measurement and evaluation on the one hand as well as the publication of achieved performance (in terms of benchmarking) on the other hand show considerable analogies to those of managerial cost and activity accounting as well as external financial reporting. Since an analysis of CHE’s potential compliance with the relevant accounting principles has not yet been undertaken, the objective of this paper is to systematically and critically analyze from an accounting perspective the method by which CHE actually evaluates research performance of business schools in Germany. Accordingly, we will demonstrate that standard accounting principles, such as completeness – which is an integral requirement of faithful representation – or consistency, are not continuously satisfied. Moreover, classifications of research cost objects, that are made within the meaning of cost and activity accounting, are not unequivocal and can, under certain circumstances, result in creating misguided incentives for the participating business schools.
... Considering human capital theory by Becker (1975), multiple researchers such as Mohapatra and Mishra (2017), Silva and Lima (2017) and Blázquez et al. (2018) claim that the responsibility for enhancing graduates' skills, abilities and employability is on universities' shoulders. However, few researchers, such as Clermont and Dirksen (2016) and Cacciolatti et al. (2017) highlighted the vital role of employers which is missing in universities. Kong and Fu (2012) stated that the performance of HEIs can be measured on the basis of the knowledge they produce for the industry, refining their students' abilities and their graduates' job performance. ...
... Chan and Lin (2016) divided abilities into two categories, i.e. intellectual and physical abilities. An intellectual ability, in a broad sense, is the ability to receive, evaluate and recall data, while physical abilities are more related to muscular endurance requiring stamina, strength, dexterity and related characteristics (Clermont and Dirksen, 2016). Silva and Lima (2017) suggested that students must concentrate on sharpening their intellectual abilities so that they can obtain a decent job and also ensure compliance with their changing requirements. ...
... In the highly competitive business environment, employers look for candidates who have problem-solving attitudes, deal effectively with people and situations and possess analytical and decision-making attitudes Evaluation of university graduates' performance (Clermont and Dirksen, 2016). A number of employers also look for reliable, confident, innovative and mature graduates (Desai and Berger, 2016). ...
Purpose
The purpose of this paper is to investigate the managers’ views on the characteristics which university graduates should have to obtain the job and efficiently perform in Chinese industrial organizations, and how are they performing in the industry.
Design/methodology/approach
Mix-method approach was followed to identify the key employability attributes, employers’ expectations and students’ performance. Analysis of interviews and literature provided the foundation for the development of the employability instrument used. In total, 11 identified attributes were grouped into four dimensions, i.e. knowledge, skills, abilities and professional attitude. Structural analysis was performed to test the hypotheses and theoretical model. The importance of the identified attributes was checked via the ranking technique, followed by an evaluation of students’ actual performance and a comparison between them.
Findings
Empirical analyses indicate that the managers in China consider “skills” as the most important dimension for employability in Chinese industrial organizations and “reliability” is the highly demanded subsequent factor. A significant gap has been found in all studied dimensions and 10 out of 11 sub-factors.
Originality/value
This study not only identifies key employability attributes, but also proposes and validates an instrument for measuring students’ industrial performance. It also highlights the factors where industry and academia should focus to improve graduates’ employability and their industrial performance.
... Even if, in formal terms, a citation is only a reference to a publication in the bibliography of another publication, the citation nevertheless symbolizes that a flow of information or a perception and utilization of the information and/or research findings has taken place (Stock 2001). Accordingly, the utilization of citations, especially as part of a comprehensive set of indicators, generates in principle a more complete, more characteristic picture of the research profile of a university or faculty (Jensen et al. 2009;Clermont and Dirksen 2016). With respect to the conception of the corresponding impact indicators, different citation metrics have been proposed and discussed in the scientometric literature. ...
... However, various authors (e.g. Linton et al. 2011;Albers 2015;Clermont and Dirksen 2016) draw attention to the fact that performance at universities whether between different subject areas or different countries can hardly be compared with each other. For this reason, the results of the above-mentioned studies are not necessarily transferable to German BuSs. ...
... Whether the CHE research ranking can provide a solid basis for a performance analysis of BuSs is confirmed by some authors (Tavenas 2004;Usher and Savino 2006;Marginson and van der Wende 2007;Stolz et al. 2010) but also regarded critically by others (e.g. Ursprung 2003;Ahn et al. 2007;Frey 2007;Clermont and Dirksen 2016). However, in general, CHE provides a comprehensive, regularly surveyed data basis that has been successively further developed since the first survey in the year 2001 and is adequate to serve as a basis for empirical analyses (see, e.g. ...
This paper investigates the potential expansion of an indicator set for research performance evaluation to include citations for the mapping of research impact. To this end, we use research performance data of German business schools and consider the linear correlations and the rank correlations between publication-based, supportive, and citation-based indicators. Furthermore, we compare the business schools in partial ratings of the relative indicators amongst themselves and with those business schools that are classified in other studies as being strong in research and/or reputable. Only low correlations are found between the citation metrics and the other indicator types. Since citations map research outcome, this is an expected result in terms of divergent validity. Amongst themselves, the citation metrics display high correlations, which, in accordance with the convergent validity, shows that they can represent research outcome. However, this does not apply to the J-factor, which is a journal-based normalizing citation metric.
... This is done by a specific evaluation and merging of certain selected input and output indicators as well as data specifically collected for that purpose (Berghoff et al. 2011). The research rankings are discussed controversially in the literature (see, e.g., Ahn et al. 2007;Marginson and van der Welde 2007;Stolz et al. 2010;Clermont and Dirksen 2016). Despite the criticism, by determining the indicators used in the ranking, the CHE provides a comprehensive, up-to-date and advanced database, which facilitates further empirical analyses of the effectiveness and/or efficiency of research of business schools (BuSs) in Germany. ...
... 14 For a more detailed analysis, see Clermont and Dirksen (2016). 15 In the datasets analysed, there are only three BuSs of private universities, but they do perform significantly better. ...
This article analyses the development of effectiveness and efficiency of German business schools’ research production between 2001 and 2009. The results suggest that effectiveness for most of the examined business schools increases initially. Then, however, a declining trend in the further course of time can be observed. Similar tendencies can be stated considering efficiency, even though they are slightly less pronounced. An analysis of the reasons for these observations reveals that the initial positive developments of effectiveness and of efficiency are mainly due to technology advances, whereas the following decreases are basically a result of technology backwardness. In regard to different types of business schools, a strong relation between the reputation of a school and the research effectiveness of that school becomes apparent. With reference to geographical regions, Western and Southern German business schools feature higher effectiveness than their Northern or Eastern counterparts do. This statement, however, is not valid in terms of efficiency.
... Especially German governmental bodies and university managements utilize the CHE data and results in order to evaluate research between different BuSs in a comparative manner. Based upon these assessments, several governmental institutions and universities derive objectives and performance agreements, thus co-opting the results of the CHE research ranking (Clermont and Dirksen 2015). ...
... Business Informatics, might not always have been executed in a consistent manner (for a detailed analysis from an accounting perspective, cf. Clermont and Dirksen 2015). ...
In order to efficiently allocate academic resources, an awareness of the properties of the underlying production function’s returns to scale is of crucial importance. For instance, the question arises as to what extent an expansion of a university department’s academic staff would be advisable in order to utilize increasing marginal gains of research production. On the other hand, it is disputable whether an optimal university department size exists. Empirical studies covering these questions render various answers. In this paper, we analyse which properties of returns to scale the Business Administration research of universities in Germany exhibits. On the basis of research data from 2001 until 2009 provided by the Centre for Higher Education, and using Data Envelopment Analysis, we demonstrate that typically sized business schools show nearly constant returns to scale. Furthermore, we observe tendencies of decreasing returns to scale for large-sized business schools. Diverse robustness and sensitivity analyses confirm the validity of our inferred empirical findings.
... However, performance indicators are frequently used in higher education evaluations to generate rankings of individual scientists, departments, or universities. For this purpose, as already mentioned, CHE has developed a research ranking, which has undergone critical analysis by Clermont and Dirksen (2016). In rankings, the precise characteristic of the indicator is irrelevant to the viewer. ...
To date, there has not been any definitive statement for bibliometric analyses from the scientific community as to how long to make the period for analysis in order to guarantee the maximum possible validity of citation analyses. Up to now, it has been particularly unclear as to whether different disciplines and citation indicators have any effect on what period for analysis to select. This is why this study aims to provide answers to these issues. Therefore, publications in the fields of business studies, biology, and medicine from 2007 to 2009 are surveyed. The citations from ten separate consecutive citation periods from 2009 up to 2018 are generated for these publications from Web of Science and selected citation indicators are calculated for them. The results show that the fundamental validity of citation indicators increases with time. However, if the goal of the analyses is to establish a ranking order, then even short citation periods are informative. Our results are stable in terms of the disciplines, although differences do occur in the citation indicators under consideration.
... For some universities, it may be more important to support the development of social identity [11][12][13][14], whereas others have a vision of structuring students' futures [15,16], or others wish to invest in public relations and comparative evaluations with other universities [17]. Overall, it is not clear how these particular foci of interest are experienced by student applicants from outside the university and how this affects their (future) student commitment and engagement. ...
A university’s presentation of its programs to the public should provide potential students with information about what they can expect as students at that university. However, it is largely unclear what kind of self-presentation affects different applicants and their commitment. In a laboratory experiment with N = 116 participants, we examined the emotional, behavioral and cognitive impact of a university’s self-presentation (either emphasizing chances for students or emphasizing their obligations) on student engagement. We also measured the participants’ regulatory focus (promotion and prevention focus). We found interaction effects of the university’s self-presentation and participants’ promotion and prevention focus on student engagement. There was a regulatory fit of promotion focus in the chances condition for emotions and behavior. There was also a regulatory fit of prevention focus in the obligations condition for cognitive processes. We conclude that universities should dedicate time and effort to create a clear presentation of their offerings in the implementation of digital learning environments.
... It is well-acknowledged fact that learning is situated in a cultural and environmental context; the environment is of primary importance in higher learning. The environment and surroundings have a major influence on building the reputation of HEIs among students and stakeholders and it also motivates the employees for quality services (Clermont and Dirksen, 2016). One of the reasons for the apparent success of private HEIs is the quality of learning environment, facilities provided and reputation with employers, as well as the ability to flexible courses timing and facilitate student growth (van Schalkwyk and Steenkamp, 2016). ...
Higher education institutions (HEIs) are responsible for training and transforming the students into valuable resources. Although students are believed to be the principal stakeholders in HEIs, limited research studies are available on service quality (SQ) in HEIs from students' perspectives. This study aims to bridge this gap by investigating the factors, which constitute SQ in HEIs, specifically from students' perspective, as existing literature on this topic is either from management and general perspective or is time dated.
The current study contributes by reviewing qualitative responses received through interviews and focus groups session with 43 students from 3 Turkey-based private HEIs. Data were collected from 43 students through 26 individual interviews and 3 focus group sessions and was analyzed through deductive reasoning using narrative and framework analysis with open coding.
The analysis of data indicated six main themes, specifically: teaching quality, facilities, support staff quality, employability links, safety and security and extra-curricular activities as indicators of SQ from students' perspective. The findings of the study strongly comply with the Maslow's hierarchy of needs and take steps by identifying employability and safety and security as new emerging indicators of the existing literature of SQ in HEIs.
The existing literature lacks to provide qualitative data on SQ in HEIs from students' perspectives in Asian countries, particularly, in Turkey, the place of current research. The findings of the present research provide valuable insights to HEIs' management to understand students' perceptions of SQ, their expectations and experiences.
... Usher and Savino 2006;Frey 2007;Marginson and van der Welde 2007;Jarwal et al. 2009;Stolz et al. 2010;Kieser 2012). For a comprehensive, critical discussion of the CHE approach, especially from the perspective of German business administration, see Clermont and Dirksen (2016). ...
Data Envelopment Analysis (DEA) is a deterministic method for the aggregation of multidimensional measures and subsequent efficiency analysis. Due to its inherent determinism, however, it reacts sensitively to outliers in datasets. Existing methods for identifying such outliers have two main disadvantages. First, from a more conceptional point of view, a uniform definition of an outlier is missing. Second, there are technical disadvantages of each method. For instance, arbitrarily limited values have to be set by the user, like the amount of efficiency value from which on a decision making unit is regarded as an outlier. This paper initially presents a definition of outliers, which explicitly takes the specifics of DEA into account. Based on this definition, an approach for identifying outliers in DEA is introduced which explicitly tackles the technical disadvantages and takes them into account in the developed algorithm. The plausibility of this approach is validated on the basis of empirical examples from performance measurement at the university level.
... Up until now proponents of tenure have countered that for profit practices will not succeed in universities because the goals of the two types of organizations differ. For profit companies want to make money for their owners while universities aim to discover and disseminate knowledge (Clermont and Dirksen 2016); different personnel systems forward each of these goals. However, this argument seems to concede that unfettered managerial discretion works well on its own for- profit turf. ...
This article relates shifts in American university tenure policies to historical changes in management debates on the relative importance of expertise and hierarchical standing in organizational decision making. The analysis contrasts the importance of expertise in Progressive Era (ca. 1895–1917) scientific management theories with the emergence of a more hierarchically grounded managerialist perspective in the second half of the twentieth century, and explores the implications for university policy.
Purpose: Knowledge has increasingly become one of the most critical determining factors for organizational competitiveness. In order to acquire and sustain a competitive advantage, it is vital for an organization to develop effective strategies that foster knowledge creation, transfer and integration in the performance of the organization. Knowledge organization refers to activities such as document description, indexing, and classification that provide systems of representation and order for information and knowledge materials. The purpose of this study was to establish the relationship between knowledge organization capability and competitiveness of chartered public universities in Kenya. Methodology: The study adopted a descriptive design. The population of the research consisted of 31 chartered public universities authorized to operate in Kenya as at December 2020. The study adopted a census methodology while purposeful and convenient sampling was adopted to select respondents amongst the middle level management staff. The study used both secondary and primary data. A self-administered questionnaire was used as the primary data collection instrument. Out of 155 questionnaires that were sent out, 123 were filled and returned. Statistical Package for Social Studies (SPSS) was used for data analysis. The study used regression model to estimate the level of significance which yielded a positive and significant influence of knowledge organization capability on competitiveness of chartered public universities in Kenya. Findings: The findings indicate that organizational initiatives such as effective knowledge collaboration and utilization, deployment of information technology systems that support knowledge organisation and easy access to information and data contribute to a healthy environment conducive for knowledge management. Owing to the foregoing, the study concluded that knowledge organization capability had a significant positive impact on competitiveness of chartered public universities. Unique Contribution to Theory, Practice and Policy: The study recommended that public universities needed to embrace knowledge organization capability in order to obtain sustainable competitive advantage.
A tanulmány célja: A Pécsi Tudományegyetem Közgazdaságtudományi Kar (PTE KTK) Kompetencia-és Tehetségfejlesztő Központjában (KTK Tehetségpont) új módszerek alkalmazásával segítjük a tehetséges hallgatók kibontakozását. Fő kutatási kérdésünk az volt, hogy az egyetem falai között mivel töltik idejüket a kiemelkedő teljesítményű hallgatók? Milyen tevékenységeket végeznek, és hogyan lehet őket tanterven felüli erőfeszítésre ösztönözni? Tudatosan kifejlesztett motivációs és teljesítménymérő új eszközünk a tehetségpont-gyűjtő füzet, amit három féléve alkalmazunk. Ebben a hallgatók összegyűjthetik féléves aktivitásaikat, és segítségével a KTK Tehetségpont folyamatosan bővítheti tehetségbankját, hiszen akik gyűjtik a pontokat, bekerülnek az adatbázisunkba. Az ebből keletkező információhalmazból állítottuk elő a tehetségprofilokat, amellyel az aktivitások fókuszait tudtuk azonosítani szakonként, képzési szintenként és nemenként. Célunk az volt, hogy feltárjuk, vannak-e szignifikáns különbségek a hallgatói aktivitásprofilok között a fenti ismérvek alapján. Alkalmazott módszertan: Kutatócsoportunk empirikus kutatásának alapját egy 132 fős minta (tehetségbank) képezte. A hallgatók összesen 5 602 sikerpontot gyűjtöttek három félév alapján. Ebben az adatbázisban a legaktívabb hallgatók szerepelnek, így az általuk megjelenített tehetségprofilokat először leíró statisztikák alapján generáltuk, majd szakok, nemek és képzési szintek szerint leszűrtük. Ezek alapján varianciaanalízissel vizsgáltuk, hogy mely aktivitások tekintetében azonosíthatóak szignifikáns eltérések a tevékenységpreferenciák között. Legfontosabb eredmények, újdonságok: Az új módszerrel, a tehetségpontok gyűjtésével magasabb aktivitásra lehet ösztönözni a hallgatók egy részét, és teljesítményükről információkat szerezhetünk, ez alapján kirajzolódnak a tehetségprofilok. Kimutattuk, hogy a képzési szinteken felfelé haladva (alapképzéstől a mesterképzés felé), egyre inkább jellemzőbbek a tudományos tevékenységek a tehetségprofilokban, míg az üzleti élettel kapcsolatos aktivitások ennek komplementereként csökkennek. Gyakorlati/gazdaságpolitikai javaslatok: A három féléven keresztül alkalmazott tehetségpont-gyűjtési rendszerünk bevált, félévente legalább 50-60 hallgatót tudtunk mozgósítani, aktivizálni vele, jó lehetőséget ad a tehetségek azonosítására, eredményeik elismerésére (pl. díjátadó keretében), tevékenységük megismerésére. Az általunk kifejlesztett pontgyűjtési módszerrel széles körű információkkal tudjuk feltölteni a tehetségbankunkat, amellyel körvonalazhatóvá válnak a tehetségprofilok, elkülöníthetővé válnak a különböző orientációval, tevékenységfókusszal (pl. tudományos, üzleti, művészeti vagy sport) rendelkező tehetségek.
In diesem Beitrag werden die Einflussfaktoren auf die Forschungsreputation deutscher betriebswirtschaftlicher Fachbereiche analysiert. Mittels Partial Least Squares-Methodik werden identifizierte Einflussfaktoren hinsichtlich ihrer empirischen Relevanz ausgewertet. Im Resultat beeinflussen insbesondere die Publikationstätigkeit und die internationale Sichtbarkeit betriebswirtschaftlicher Fachbereiche deren Forschungsreputation positiv. Ein direkter Einfluss der Größe eines Fachbereichs kann hingegen nicht festgestellt werden. Vielmehr wird der Einfluss der Größe über die internationale Sichtbarkeit mediiert. Zusätzlich kann gezeigt werden, dass sich die Intensität der Lehrtätigkeit negativ auf die Publikationstätigkeit auswirkt.
Veröffentliche oder stirb! – Forscher stehen heutzutage unter dem enormen Druck,
ihre wissenschaftlichen Leistungen in Form von Publikationen zu vermarkten. Doch in einer der führenden Fachzeitschrift zu veröffentlichen, ist sehr schwierig. Wie können Wissenschaftler ihre Publikationschancen erhöhen? Welchen Kriterien muss ein Artikel entsprechen, damit er möglichst positiv begutachtet wird? Der Blick auf 50 Jahre Organisationsforschung liefert die nüchterne Erkenntnis: Mäßige Originalität nahe am »Mainstream« zahlt sich aus.
The selection, aggregation and analysis of research performance data acquired by the Centre for Higher Education (CHE) as well as its assessment method identifying leading research business schools are controversially discussed in literature and praxis. Data Envelopment Analysis (DEA) provides an alternative method to aggregate research performance data. Thereby however, business schools can achieve high effectiveness or efficiency scores by taking into account only some and not all indicators considered as relevant, resulting in an incomplete picture of business schools’ research performance. In this paper we therefore measure and analyze a new key performance indicator, the balance score respective the specialization degree of a business school by using the new method Balanced-DEA. This indicator reflects to which extent the research performance of a business school is balanced or specialized relative to predefined virtual balanced reference points.
In order to efficiently allocate academic resources, an awareness of the properties of the underlying production function’s returns to scale is of crucial importance. For instance, the question arises as to what extent an expansion of a university department’s academic staff would be advisable in order to utilize increasing marginal gains of research production. On the other hand, it is disputable whether an optimal university department size exists. Empirical studies covering these questions render various answers. In this paper, we analyse which properties of returns to scale the Business Administration research of universities in Germany exhibits. On the basis of research data from 2001 until 2009 provided by the Centre for Higher Education, and using Data Envelopment Analysis, we demonstrate that typically sized business schools show nearly constant returns to scale. Furthermore, we observe tendencies of decreasing returns to scale for large-sized business schools. Diverse robustness and sensitivity analyses confirm the validity of our inferred empirical findings.
Many universities in Germany and other countries have introduced financial (or commercial) accounting to manage effectively their finances. It aligns with so-called new public sector management reforms worldwide. In this paper we analyze whether the components of this type of financial accounting reform suit the nature and objectives of German public universities. While the analysis mainly relates to the German situation, there are likely to be implications for public universities universally. Drawing on an analysis of the reports of two well known German-speaking universities, Heidelberg and Vienna, we analyze whether the components of the new financial accounting reforms suit the nature and objectives of those public universities. While we argue that cash-flow statements and balance sheets remain important, it is shown that it is necessary for state-run educational institutions to change several key elements of traditional commercial accounting. Because the success goals of such universities are non-profit-oriented, their financial accounting should be augmented by a ‘change in value statement’, replacing the traditional income statement. As their valuation cannot be correlated with the definition of profit in (German) public universities this term is suggested. By change in value we mean specifically that assets may experience a total loss or a decrease in value. In management accounting output measures and performance indicators should substitute revenues as the counterpart of costs. Furthermore, long-term financial decisions play a crucial role in universities. Therefore, a form of investment accounting is very important for them. The conventional form has to be modified by ‘investment statements’ and ‘knowledge balances’ of their intellectual capital. In this paper we also show how the concept of a balanced scorecard can be applied to public universities and how specific accounting instruments can be integrated into it.
The simplicity and apparent objectivity of the Institute for Scientific Information’s Impact Factor has resulted in its widespread use to assess the quality of organization studies journals and by extension the impact of the articles they publish and the achievements of their authors. After describing how such uses of the Impact Factor can distort both researcher and editorial behavior to the detriment of the field, I show how extreme variability in article citedness permits the vast majority of articles — and journals themselves — to free-ride on a small number of highly-cited articles. I conclude that the Impact Factor has little credibility as a proxy for the quality of either organization studies journals or the articles they publish, resulting in attributions of journal or article quality that are incorrect as much or more than half the time. The clear implication is that we need to cease our reliance on such a non-scientific, quantitative characterization to evaluate the quality of our work.
Maintains that there is little or no evidence that citation frequencies are useful for evaluating psychologists other than to identify those whose names are household words in the discipline. It is also stressed that the citation impact factor penalizes journals that publish articles that are not cited: The journal's impact factor is likely to be reduced by its publication of articles that do not conform to current customs, fads, and fashions. (13 ref) (PsycINFO Database Record (c) 2012 APA, all rights reserved)
This paper examines whether three bibliometric indicators—the journal impact factor, citations per paper and the Excellence in Research for Australia (ERA) initiative's list of ‘ranked journals’—can predict the quality of individual research articles as assessed by international experts, both overall and within broad disciplinary groupings. The analysis is based on data obtained from a Mock Research Quality Framework (RQF) exercise conducted by Monash University during 2006–07 in which external assessors rated research articles for their quality using a five-point scale. Although a significant relationship exists between all three bibliometric variables and the overall Mock RQF assessor quality scores, only a relatively small amount of the variance (generally
Purpose
– The purpose of this paper is to examine the effects of the use of the citation‐based journal impact factor for evaluative purposes upon the behaviour of authors and editors. It seeks to give a critical examination of a number of claims as regards the manipulability of this indicator on the basis of an empirical analysis of publication and referencing practices of authors and journal editors
Design/methodology/approach
– The paper describes mechanisms that may affect the numerical values of journal impact factors. It also analyses general, “macro” patterns in large samples of journals in order to obtain indications of the extent to which such mechanisms are actually applied on a large scale. Finally it presents case studies of particular science journals in order to illustrate what their effects may be in individual cases.
Findings
– The paper shows that the commonly used journal impact factor can to some extent be relatively easily manipulated. It discusses several types of strategic editorial behaviour, and presents cases in which journal impact factors were – intentionally or otherwise – affected by particular editorial strategies. These findings lead to the conclusion that one must be most careful in interpreting and using journal impact factors, and that authors, editors and policy makers must be aware of their potential manipulability. They also show that some mechanisms occur as of yet rather infrequently, while for others it is most difficult if not impossible to assess empirically how often they are actually applied. If their frequency of occurrence increases, one should come to the conclusion that the impact of impact factors is decreasing.
Originality/value
– The paper systematically describes a number of claims about the manipulability of journal impact factors that are often based on “informal” or even anecdotal evidences and illustrates how these claims can be further examined in thorough empirical research of large data samples.
This study examines the question of whether the journal ranking VHB-JOURQUAL 2 can be considered as a good measure for the construct “scientific quality”. Various rankings in business research provide the database for the analysis. The correlations between theses rankings are used to assess the validity of VHB-JOURQUAL 2 along various validity criteria. The correlations with rankings that measure the same construct based on different methods show that VHB-JOURQUAL 2 has acceptable, but moderate convergent validity. The validity varies considerably across disciplines, showing that the heterogeneity of business administration is not sufficiently represented by this overall ranking. The variability is related to the variation in members per discipline represented by the German Association for Business Research. Furthermore, the measure shows a weak correlation with acceptance rates as an indicator of nomological validity in some disciplines.
Resources within and between higher education and research institutions are increasingly allocated according to scientific performance. Evaluation exercises and the measurement of research performance take on a new role in this context. Third party research income is a performance indicator which is rather easy to measure and is used in most of the new performance-based evaluation procedures. This paper sets out to scrutinize the meaning and validity of third party research income. We studied research teams from three different research fields with a mixed quantitative / qualitative approach. The focus is on the causal relations between institutional / input indicators, third party research income as another input indicator, and a variety of output indicators of scientific production. An important result is that third party money has a positive effect on performance only below a certain and discipline-specific threshold. Further analysis shows that output performance is to a great extent field-specific. The context conditions for scientific production differ greatly, so that comparative performance assessments are only valid within homogeneous research fields. Another important result is that output performance is multidimensional and cannot be measured by bibliometric indicators only. These findings have implications for the rationality of the evaluation assessment exercises as well as for the funding decisions of science foundations, ministries of science and — increasingly, the heads of universities and departments. These topics are discussed in the last chapter as well as the implications for further research.
Der Beitrag befasst sich mit alternativen Konzepten der Datenaggregation zur Leistungsbeurteilung wertschöpfender Einheiten.
Ausgehend von einem kritischen Vergleich der entsprechenden Methodik des Centrums für Hochschulentwicklung (CHE) mit der Data
Envelopment Analysis (DEA) wird die so genannte NACHE als Kompromissansatz abgeleitet. Einerseits bewahrt der neue Ansatz
als Modifikation der CHE-Methodik weitgehend deren Pragmatik; andererseits führt er zu sehr ähnlichen Ergebnissen wie die
methodisch anspruchsvolle DEA. Auf diese Weise verkörpert die NACHE ein Bindeglied zwischen Wissenschaft und Praxis der Leistungsbeurteilung.
The paper discusses alternative concepts of data aggregation for performance assessment. A critical comparison of the corresponding
method of the ‚Centrum für Hochschulentwicklung’ (CHE), Gütersloh/Germany, with the Data Envelopment Analysis (DEA) leads
to the development of the so-called NACHE as a compromise solution. The NACHE is nearly as pragmatic as the CHE method, while
its results are very similar to those of the sophisticated DEA. Therewith, the NACHE represents a link between practice and
science of performance assessment.
Global university rankings have cemented the notion of a world university market arranged in a single "league table" for comparative purposes and have given a powerful impetus to intranational and international competitive pressures in the sector. Both the research rankings by Shanghai Jiao Tong University and the composite rankings by the Times Higher Education Supplement have been widely publicised and already appear to have generated incentives in favour of greater system stratification and the concentration of elite researchers. However, global comparisons are possible only in relation to one model of institution, that of the comprehensive research intensive university, and for the most part are tailored to science-strong and English-speaking universities. Neither the Shanghai nor the Times rankings provide guidance on the quality of teaching. It is important to secure "clean" rankings, transparent, free of self-interest, and methodologically coherent, that create incentives to broad-based improvement.
VHB-JOURQUAL represents the official journal ranking of the German Academic Association for Business Research. Since its introduction in 2003, the ranking has become the most influential journal evaluation approach in German-speaking countries, impacting several key managerial decisions of German, Austrian, and Swiss business schools. This article reports the methodological approach of the ranking’s second edition. It also presents the main results and additional analyses on the validity of the rating and the underlying decision processes of the respondents. Selected implications for researchers and higher-education institutions are discussed.
In this paper we consider the problem of determining teaching and research efficiencies for university departments concerned with the same discipline. Considering this problem highlights the issue of how to determine efficiencies when resources are shared between different activities, and a non-linear approach to this issue based upon data envelopment analysis is presented. Computational results are given for chemistry and physics departments in the United Kingdom.
Das Lehrbuch bietet eine leicht zugängliche Einführung in die entscheidungsorientierte Produktionstheorie industrieller Leistungsprozesse. In einem einheitlichen Rahmen behandelt es sowohl wesentliche Modelle und Aussagen der traditionellen Produktions- und Kostentheorie als auch grundlegende Aspekte des Produktionsmanagements. Der system-, prozess- und erfolgsorientierte Ansatz integriert auf organische Weise Gesichtspunkte des Umweltschutzes und bildet so auch eine Grundlage für das betriebliche Umweltmanagement und das industrielle Controlling. Das didaktische Hilfsmittel der Input/Output-Graphen führt zu einer konstruktiven Ausrichtung der Theorie und fördert zudem das Verständnis komplexer, vernetzter Produktionsstrukturen.
Dieses Lehrbuch befasst sich mit der konzeptionellen Gestaltung und den Einsatzbedingungen von Rechnungs- und Informationssystemen im Unternehmen. Themen sind Methoden und Instrumente der Kosten- und Leistungsrechnung, des Kostenmanagement und des Controlling. Das Buch stellt drei Hauptfunktionen in den Vordergrund:
1. Entscheidungsrechnungen: Analysiert werden entscheidungstheoretische Grundlagen, Beziehungen der internen Unternehmensrechnung zur Investitionsrechnung, Produktionsprogrammplanung bei Sicherheit und Unsicherheit, Preisbestimmung, Break-even-Analysen und Instrumente des Kostenmanagement.
2. Kontrollrechnungen: Im Mittelpunkt stehen die Berechnung und die Auswertung von Abweichungen sowie Anreizwirkungen von Kontrollen.
3. Koordinationsrechnungen: Untersucht werden Konzepte und Wirkungen von Budgetierung, von Kennzahlen zur Performancemessung und von Verrechnungspreisen.
Systeme der Kostenrechnung runden das Lehrbuch ab.
In der Neuauflage wurden kleine Ergänzungen in mehreren Themenbereichen vorgenommen sowie aktuelle Entwicklungen in der internen Unternehmensrechnung eingearbeitet.
Der Inhalt
• Entscheidungsrechnungen
• Kontrollrechnungen
• Koordinationsrechnungen
• Systeme der Kostenrechnung
Die Zielgruppen
• Dozenten und fortgeschrittene Studierende der Betriebswirtschaftslehre an Universitäten und Hochschulen
• Praktiker im Rechnungswesen, die ihre Kenntnisse vertiefen möchten
Die Autoren
Prof. Dr. Ralf Ewert ist Vorstand des Instituts für Unternehmensrechnung und Wirtschaftsprüfung und stellvertretender Direktor des Center for Accounting Research der Universität Graz.
Prof. Dr. Dr. h.c. Alfred Wagenhofer ist Vorstand des Instituts für Unternehmensrechnung und Controlling und Direktor des Center for Accounting Research der Universität Graz.
We investigate the impact of choosing between four data sources on the relative performance measurement of scholars’ research output. Our results show that rankings can differ considerably and that the observed correlations are often even lower than those reported in previous studies. We show that data source selection generally has a higher effect than does measure selection and that performance measures based on Google Scholar might provide information that complements the more traditional data sources. We find greater differences for accounting, which suggests that decision makers must pay more careful attention when using rankings in this field.
Welche Unis sind die besten Deutschlands? Mit dem Ziel, auf diese Frage eine Antwort zu liefern, werden von verschiedenen Zeitungen und Magazinen, wie Die Zeit, Focus, Spiegel und Wirtschaftswoche, regelmäßig Hochschulrankings herausgegeben. Diese werden mit verschiedenen Methoden erstellt. So werden Befragungen von Studierenden, Professoren und Unternehmensverantwortlichen durchgeführt, aber auch quantitative Indikatoren erhoben. Die Analysen führen nicht selten zu unterschiedlichen Ergebnissen und werden daher vielfach kontrovers diskutiert. Diesen konträren Auffassungen wird im vorliegenden Beitrag anhand leicht nachvollziehbarer, konstruierter Beispiele näher auf den Grund gegangen. Zunächst wird kurz auf Forschung und Leistung im Allgemeinen sowie auf Möglichkeiten zu deren Beurteilung eingegangen (Abschnitt 1). Anschließend wird die Beurteilung der Forschungsleistung mittels Drittmittelindikatoren betrachtet, wobei diese aus produktionstheoretischer Perspektive analysiert und im Hinblick auf ihre prinzipielle Eignung zur Bewertung der Forschungseffektivität und -effizienz untersucht werden (Abschnitt 2). Auf diesen Grundlagen basieren die im dritten Abschnitt in diesem Kontext durchgeführten Überlegungen zur Bedeutung der Periodenabgrenzung, die sich zum einen auf individuelle, zum anderen auf aggregierte Forschungstätigkeiten beziehen.
This paper investigates the effects of going beyond publication counts on the relative performance measurement of German-speaking business administration scholars. Based on data from the Social Science Citation Index, Scopus, and Google Scholar it compares rankings based on publication, citation, and combined measures, such as the h-index. The results from 298 accounting and marketing scholars show that the move from publication to citation counts is a greater step than from citations to the h-indices. A similar observation can be made with respect to refinements of the h-index. We investigate several causes of these effects and show that citation counts and combined measures specify the information content of data sources. The results also suggest that data source coverage is a larger driver of differences than measures. Finally, we find that correlations between rankings based on different data sources can be improved by extending beyond publication measures.
Recently, literature-and citation-databases have been used to acquire data for the analysis of research quantity and quality of German-speaking researchers in business administration by means of bibliometric indicators. Authors of such bibliometric analyses usually don't question the content of the applied database. The first researchers, who pick up this relevant question by checking the acquisition of (VHB-) JOURQUAL1 rated economic journals in the ISI-databases and Scopus, are Clermont/Schmitz (2008). The present paper enlarges these analyses by using the JOURQUAL2 as a standard for comparison and by including the literature-databases WISO and EconLit.
What decision analysis is about.- Structuring the decision problem.- Generating objectives and hierarchies.- Generating and preselecting alternatives.- Decision making under certainty with one objective.- Decision making under certainty and with multiple objectives: multiattribute value functions.- The generation of probabilities.- Simulation of an objective variable's probability distribution.- Decisions under risk and one objective.- Decision under risk: incomplete information and multiple objectives.- Time preferences under certain expectations.- Group decisions.- Descriptive aspects of decision making.
Academics in the field of economic sciences are increasingly expected to solicit (research based) third party funding, even in competition with other disciplines. The recent Excellence Initiative in Germany, in which economic sciences have performed disappointingly, has shown this pressure quite plainly. This article gives an overview of the main funding programmes of the DFG (German Research Association), the assessment and decision-making process in these programmes and discusses the position of economics in the DFG funding context.
Prior studies on whether or not International Financial Reporting Standards (IFRS) adoption improves earnings quality have found mixed results. We note that some of the countries that have adopted IFRS had national accounting standards similar to IFRS prior to adopting IFRS, while others had national accounting standards divergent from IFRS. We examine the effects of IFRS adoption by taking into account the level of divergence prior to the adoption of IFRS. We find that countries experience a greater drop in earnings management when they have a higher level of divergence from IFRS prior to IFRS adoption. More specifically, high divergence countries with higher levels of enforcement benefit the most followed by high divergence countries with lower levels of enforcement. Lower divergence countries with higher levels of enforcement do not significantly benefit from IFRS adoption. Lower divergence countries with lower levels of enforcement do not benefit from IFRS adoption at all. Our results support the contention that countries with lower quality local accounting standards prior to IFRS adoption benefit more from IFRS adoption.
Cone ratios, assurance regions and similar data envelopment analysis (DEA) approaches measuring the efficiency of decision making units (DMU) restrict their ability to “show themselves in the most favorable light”. Indeed they limit the relations or the multipliers of inputs and outputs as efficiency factors and thus induce a certain balance among them. We develop an alternative approach which – based on the conventional DEA methodology and additionally to the usual efficiency score – assesses a degree of balance or balanced efficiency for each DMU. For that purpose we assume that a region within the data envelopment can be specified in which all DMUs are 100% balanced. In a case study, measuring the balanced effectiveness of German business schools’ research performance, we obtain plausible results providing informative insights into German business research. This confirms the reasonable use of the new method.
The annual "U.S. News and World Report" "America's Best Colleges" issue has become the nation's de facto higher education accountability system, evaluating colleges and universities on a common scale and creating strong incentives for institutions to do things that raise their ratings. But, says the author, the "U.S. News" ranking system is deeply flawed. Instead of focusing on the fundamental issues of how well colleges and universities educate their students and how well they prepare them to be successful after college, the magazine's rankings are almost entirely a function of three factors: fame, wealth, and exclusivity. As a result, the rankings lead colleges and universities to focus their energies on becoming wealthier, more famous, and more exclusive, sometimes at the expense of educating their students well. New research and advances in technology, writes Carey, in the last several years have led to a variety of new metrics and data sources that together offer a significant opportunity to measure how well colleges and universities are preparing their undergraduate students. The new measures provide information about a range of factors like teaching quality, student learning, graduation rates, and success after college, and offer the opportunity to replace existing college rankings with a significantly improved system. This report explains what the new measures can show, how those measures can be combined into new college rankings, and why the new rankings would benefit both students and colleges. (Contains 20 endnotes, 6 figures and 5 tables.)
The Educational Policy Institute, Inc. (EPI) is a non-partisan, non-governmental organization dedicated to policy-based research on educational opportunity for all students. With offices in Washington, DC, Toronto, ON, and Melbourne, Australia, EPI is a collective association of researchers and policy analysts from around the world dedicated to the mission of enhancing our knowledge of critical barriers facing students and families throughout the educational pipeline. In addition, EPI has developed extensive partnerships and collaborative arrangements with other leading research and educational organizations, further supporting our mission and ability to conduct policy-relevant research for practical use. The mission of EPI is to expand educational opportunity for low-income and other historically-underrepresented students through high-level research and analysis. By providing educational leaders and policymakers with the information required to make prudent programmatic and policy decisions, we believe that the doors of opportunity can be further opened for all students, resulting in an increase in the number of students prepared for, enrolled in, and completing postsecondary education.
This paper develops a model of accounting measurement to study the design of the optimal measurement rule. The core of the model is a representation of accounting measurement process that features the manager’s opportunistic influence and the use of verification as a response. To safeguard against the manager’s ex post opportunism, the optimal measurement rule is conservative in the sense that it requires more verification of the transaction characteristics favorable to the manager. The model clears up some criticisms of conservatism by differentiating the property of an accounting measure from that of an accounting measurement rule that generates the accounting measure.
Many of the complex problems faced by decision makers involve multiple conflicting objectives. This book describes how a confused decision maker, who wishes to make a reasonable and responsible choice among alternatives, can systematically probe his true feelings in order to make those critically important, vexing trade-offs between incommensurable objectives. The theory is illustrated by many real concrete examples taken from a host of disciplinary settings. The standard approach in decision theory or decision analysis specifies a simplified single objective like monetary return to maximise. By generalising from the single objective case to the multiple objective case, this book considerably widens the range of applicability of decision analysis.
This article focusses on third party funding of research in German universities. The centralquestion is, whether funding data can function as suitable indicators for the measurement ofresearch performance of university departments. After a brief description of the importance andthe extent of third party funding in the German system of research funding, the quality of data isdiscussed and the funding indicator is compared with bibliometric indicators. Resultened, one cansay that in subjects where external funding of research is usual, the funding indicator points to thesame direction as other indicators do. Because of the peer review process involved in grantawarding, a funding indicator is in many subjects a suitable indicator to evaluate R&D impacts.
Publikations- und Zitationsindizes können insbesondere zur Quellenerschließung und Forschungsevaluation genutzt werden. Die
Eignung entsprechender Indizes zur adäquaten Erfüllung dieser Funktionen hängt maßgeblich von der Auswahl der indexierten
Publikationen ab. In diesem Beitrag wird die Publikationsauswahl des Social Sciences Citation Index (SSCI), des Science Citation
Index-Expanded (SCI-X) und der Datenbank Scopus aus Sicht der deutschsprachigen Betriebswirtschaftslehre analysiert. Als Basis
für die Untersuchung dienen die Ergebnisse des VHB-JOURQUAL. Neben einem Gesamtüberblick zur Erfassung betriebswirtschaftlich
relevanter Zeitschriften in den Datenbanken wird auch die Abdeckung einzelner betriebswirtschaftlicher Forschungsfelder untersucht.
Da die Nutzung der Indizes mit einigen Kosten für den Erwerb entsprechender Lizenzen verbunden ist, erfolgt darüber hinaus
eine vergleichende Analyse von SSCI, SCI-X und Scopus hinsichtlich der Zeitschriftenabdeckung. Aufgrund der internationalen
Ausrichtung der betrachteten Zitationsindizes wird die Untersuchung zusätzlich auf ein internationales Zeitschriftenranking
ausgedehnt.
Publication- and citation-indices are especially used for searching literature and evaluating research performance. Their
suitability strongly depends on the publications taken into account. In this paper we analyse the selection of publications
in Social Sciences Citation Index (SSCI), Science Citation Index-Expanded (SCI-X) and Scopus for G erman speaking researchers
of business economics. Therewith, the VHB-JOURQUAL provides a basis for our analysis. In addition to an overview of economic
relevant journals in the indices we analyse the coverage of special fields of business economics. Due to the fact that using
such indices is very expensive, we compare SSCI, SCI-X and Scopus concerning their coverage of journals. As the indices are
of international orientation, we enlarge the analysis onto an international ranking of publications.
In Fandel and Gal (2001) [Fandel, G., Gal, T., 2001. Redistribution of funds for teaching and research among universities: The case of North Rhine-Westphalia, European Journal of Operational Research 130, 111–120] a solution for a real process of redistributing funds for teaching and research among the universities in North Rhine-Westphalia in Germany according to specific criteria was presented. The solution was based on aggregated proportional data and was determined by a negotiation process. Now that the absolute data with respect to the different subject areas of the universities are available, it is studied, to which extent the results of the redistribution can be justified by the relative efficiency measures using data envelopment analysis. Moreover, the inefficiencies or slacks, respectively, in the usage of personnel give reasons to reallocate the staff among the universities or to reduce it correspondingly.
Incl. abstract, tables, bibl. The purpose of this study is to evaluate the ranking practices of 25 European higher education ranking systems (HERSs). Ranking practices were assessed with 14 quantitative measures derived from the Berlin Principles on Ranking of Higher Education Institutions (BPs). HERSs were then ranked according to their degree of congruence with the BPs. Additionally, the three domains of methodology, transparency, and consumer-friendliness were proposed to underlie the BPs, and the measures were also applied for assessing HERSs regarding these domains. Results indicate that the cooperating HERSs by CHE (Germany), AQA (Austria) and swissUp (Switzerland) exhibit the highest congruence with the BPs. However, no European HERS demonstrates good overall congruence with the BPs, mainly due to methodological shortcomings. Results further show that HERSs compiled and published by non-profit research entities seem to perform better than the majority of HERSs published by for-profit news outlets. International comparisons indicate that HERSs published in German-speaking countries and the Netherlands tend to exhibit a higher congruence with the BPs. Overall, this study hopes to stimulate the improvement of ranking practices through benchmarking with existing exemplary models. The quantitative assessment tool further promises to be useful in explaining relative stability or change of higher education institutions in HERSs, as well as in helping to explain resource allocation pressures within higher education institutions.
In this paper we present a quantitative model for comparing university departments concerned with the same discipline. This model is based upon ideas drawn from data envelopment analysis. Computational results are given for chemistry and physics departments in the United Kingdom.
In the sciences the outside evaluation of past performances of universities, faculties, departments, research groups and of individuals has become more and more frequent, nearly incessant. It could be said that the sciences are afflicted with "Evaluitis", a creeping and widespread illness. Besides the obvious costs that arise for those being evaluated and for those doing the evaluation there are additional costs that weigh heavily but are usually disregarded: incentives are distorted systematically and ossification is promoted. Furthermore, the whole decision approach is wrongly conceived. For these reasons there are too many and too thorough evaluations. A useful alternative is an appropriate design of institutions guiding incentives and a careful selection of persons - who thereafter should be free to pursue their tasks. Copyright 2007 die Autoren Journal compilation 2007, Verein für Socialpolitik und Blackwell Publishing Ltd.
Financial accounting: An international introduction. 5 th edition
- D Alexander
- C Nobes
Alexander, D., & Nobes, C. (2013). Financial accounting: An
international introduction. 5 th edition. Harlow, England:
Pearson.
Vielfältige Exzellenz 2011: Forschung, Anwendungsbezug, Internationalität, Studierendenorientierung im CHE Ranking
- S Berghoff
- P Giebisch
- C.-D Hachmeister
- B Hoffmannkobert
- M Hennings
- F Ziegele
Berghoff, S., Giebisch, P., Hachmeister, C.-D., HoffmannKobert, B., Hennings, M., & Ziegele, F. (2011).
Vielfältige
Exzellenz
2011:
Forschung,
Anwendungsbezug, Internationalität, Studierendenorientierung im CHE Ranking. Gütersloh, Germany:
Centrum für Hochschulentwicklung.
Boor, M. (1982). The citation impact factor: Another dubious
index of journal quality. American Psychologist, 37(8),
975-977.
Is there life after rankings? The Atlantic
- C Diver
Diver, C. (2005). Is there life after rankings? The Atlantic Nov.
1 st,
2005.
http://www.theatlantic.com/magazine/archive/2005/11/is
-there-life-after-rankings/304308/2/ (2014-07-25).
JOURQUAL: Its use, not its misuse, is the problem. Or: Why Wirtschaftsinformatik is the best journal of the German-speaking countries
- A Kieser
Kieser, A. (2012). JOURQUAL: Its use, not its misuse, is the
problem. Or: Why Wirtschaftsinformatik is the best
journal of the German-speaking countries. Die
Betriebswirtschaft, 72(2), 93-110 [in German].
Kosten-und Leistungsmanagement in Hochschulen: Grundlagen und Konzepte für ein zweckorientiertes Rechnungswesen
- S Kirchhoff-Kestel
Kirchhoff-Kestel, S. (2006). Kosten-und Leistungsmanagement
in Hochschulen: Grundlagen und Konzepte für ein
zweckorientiertes Rechnungswesen. Lohmar, Germany:
Eul.
Qualitative indicators for research assessment: A literature overview
Research Evaluation and Policy Project (2005). Qualitative
indicators for research assessment: A literature
overview. Canberra, Australia: The Australian National
University.
Rechnungslegung nach IFRS und HGB. 3 rd edition
- K Ruhnke
- D Simons
Ruhnke, K., & Simons, D. (2012). Rechnungslegung nach IFRS
und HGB. 3 rd edition. Stuttgart, Germany: Schäffer
Poeschel.
Quality assurance: A reference system for indicators and evaluation procedures
- F Tavenas
Tavenas, F. (2004). Quality assurance: A reference system for
indicators and evaluation procedures. Brussels,
Belgium: European University Association.
Conceptual framework for financial reporting
International Accounting Standards Board (2010). Conceptual
framework for financial reporting 2010. London,
England: IFRS Foundation Publications Department.