Article

The science citation index and the measurement of national performance in terms of numbers of scientific publications

Authors:
To read the full-text of this research, you can request a copy directly from the author.

Abstract

A debate has occurred recently over the issue whether it is possible to account for differences in results when using various versions of theScience Citation Index for the measurement of national performance in terms of numbers of scientific publications. This article provides an overview of the various arguments which have been made, and reports that recent reorganization in the on-line installations (SciSearch) should make it possible to circumvene one of the major sources of error.

No full-text available

Request Full-text Paper PDF

To read the full-text of this research,
you can request a copy directly from the author.

... If we turn our attention to the most quoted works within the corpus, we have extracted these from the 305 articles which together contain 12,695 references in Table 1 using the ISI.exe program (see Leydesdorff, 1989). ...
Article
Full-text available
Nordic countries are known for having extensive welfare services, a highly compressed wage structure owing to strong social partners, as well as effective regulation and governance in public administration. Various typologies capture aspects of the institutional features of families of nations across various policy areas, showing that there is a specific Nordic variant of political economy. While there is an extensive literature focusing on socio‐economic outcomes in the Nordic countries, there is less scholarly focus on the linkages between the regulatory processes, and their policy output, in response to various challenges. This volume examines how exogenous challenges (market liberalization promoted by EU integration and the gig economy, as well as the COVID‐19 pandemic) and endogenous challenges in the welfare state (regulation of child‐care quality and retirement ages) are tackled in a selection of Nordic countries. After a bibliometric analysis on the state of the literature, features of the Nordic model are presented. Then, the contributions of the articles to the special issue are summarized, after which lessons for other models of political economy are pinpointed. We find that although there is high variation within the Nordics in the studies of the special issue, there is a trend whereby, over time, a broader range of actors involved in the policy and regulatory process. Although not perfect, challenges are solved incrementally and often at an early stage. In other words, the Nordic regulatory model is highly adaptable to different challenges. Thus, the Nordic model does present crucial lessons for other types of political economy.
... Although the whole-count method overcounts the number of articles, it is the only one that does not biases the countries or institutions occurrences due to co-authorship (Leydesdorff, 1989). ...
Article
Full-text available
Glyphosate is one of the most used herbicides worldwide. However there still exists a strong debate about its effects on the environment or people health. In this context, the metric analysis is a methodology increasingly used to quantify and evaluate the scope of the scientific production around a specific topic. The goal of this work was to characterize the context of knowledge generation about glyphosate. All the scientific production linked to glyphosate was analyzed in the period from 1974 to 2016, using the Web of Science (WOS) as a source of information and "glyphosate" as a search criterion. A total of 8174 records were found. The bibliographic production on this herbicide experienced a sustained increase. There was a concentration in knowledge generation led by the United States. Monsanto Company produced the largest number of articles during the first thirty years of the development of this topic. Research around glyphosate in South America gained importance as of the year 2000, with Brazil being the country with the highest production followed by Argentina. A large proportion of research was focused on agricultural science, while the analysis of toxicology or environmental effects did not begin until the year 2000, and it is still insufficient. [Available for free download until July 10, 2019 here: https://authors.elsevier.com/a/1Z5ObB8cckOhA]
... While our network measures retrospectively mark the divergences among our three areas, a question for future research is whether or not these methods can be adapted to capture research dynamics as they emerge. If so, focusing on citation ecologies could fill a gap between assessments that focus on individual scholars' publication records [58][59][60][61], and those concerned with the research programs of larger-scale entities-such as institutions [60,62], innovation networks [63], or nations [64][65][66]. That is, by clustering together measures of the structures of individual networks, projects that characterize citation ecologies have the potential to clarify the links between the micro level of individual papers, and their relations to the larger, macro-structures to which they contribute. ...
Article
Full-text available
Certain areas of scientific research flourish while others lose advocates and attention. We are interested in whether structural patterns within citation networks correspond to the growth or decline of the research areas to which those networks belong. We focus on three topic areas within optical physics as a set of cases; those areas have developed along different trajectories: one continues to expand rapidly; another is on the wane after an earlier peak; the final area has re-emerged after a short waning period. These three areas have substantial overlaps in the types of equipment they use and general methodology; at the same time, their citation networks are largely independent of each other. For each of our three areas, we map the citation networks of the top-100 most-cited papers, published pre-1999. In order to quantify the structures of the selected articles' citation networks, we use a modified version of weak tie theory in tandem with entropy measures. Although the fortunes of a given research area are most obviously the result of accumulated innovations and impasses, our preliminary study provides evidence that these citation networks' emergent structures reflect those developments and may shape evolving conversations in the scholarly literature.
... Conceptually, however, one can argue that a coauthored publication can be counted as an achievement on both sides, and should thus be honored with a full point ("integer counting"). A disadvantage of fractional counting is that the numbers decline with increased collaboration, ceteris paribus (Leydesdorff, 1989). However, one can solve the problem that the numbers may not always add up to 100% by using relative frequencies. ...
Article
Full-text available
Using co-authored publications between China and Korea in Web of Science (WoS) during the one-year period of 2014, we evaluate the government stimulation program for collaboration between China and Korea. In particular, we apply dual approaches, full integer vs. fractional counting, to collaborative publications in order to better examine both the patterns and contents of Sino-Korean collaboration networks in terms of individual countries and institutions. We first conduct a semi-automatic network analysis of Sino-Korean publications based on the full-integer counting method, and then compare our categorization with contextual rankings using the fractional technique; routines for fractional counting of WoS data are made available at http://www.leydesdorff.net/software/fraction . Increasing international collaboration leads paradoxically to lower numbers of publications and citations using fractional counting for performance measurement. However, integer counting is not an appropriate measure for the evaluation of the stimulation of collaborations. Both integer and fractional analytics can be used to identify important countries and institutions, but with other research questions.
... The previously debated decline of British science in terms of scientometric indicators (Anderson et al., 1988; Braun et al., 1991; Leydesdorff, 1989) has during the 1990s come to a definite halt. All European nations and Japan have been able to improve their performance rates gradually at the expense of the USA and Canada. ...
Article
Full-text available
Institutions and their aggregates are not the right units of analysis for developing a science policy with cognitive goals in view. Institutions, however, can be compared in terms of their performance with reference to their previous stages. King's (2004) 'The scientific impact of nations' has provided the data for this comparison. Evaluation of the data from this perspective along the time axis leads to completely different and hitherto overlooked conclusions: a new dynamic can be revealed which points to a group of emerging nations. These nations do not increase their contributions marginally, but their national science systems grow endogenously. In addition to publications, their citation rates keep pace with the exponential growth patterns, albeit with a delay. The center of gravity of the world system of science may be changing accordingly.
... Version einer Datenbank für die Relevantere zu halten. Man erwartet dann eine höhere Qualität, was aber nicht den Fakten entspricht.Leydesdorff (1989) hat dazu gearbeitet. Das ISI selbst hat sich bislang noch nie zur Qualität der einzelnen Versionen geäußert. ...
Article
Full-text available
Die Bewertung wissenschaftlicher Produktivität und Produktion bzw. deren Resonanz gemessen an Zitaten ist überaus beliebt. In Zeiten knapper Kassen ist die leistungsorientierte Mittelvergabe dringend auf objektive Parameter der Leistung und Leistungsfähigkeit der Wissenschaft und der Wissenschaftler angewiesen. Dabei soll die Frage, ob die Qualität wissenschaftlicher Arbeit einer letztendlichen quantitativen Begründung zugänglich ist ebenso unkommentiert bleiben, wie die bereits Realität gewordene quantitative Beurteilung von einzelnen Wissenschaftlern, Instituten, Einrichtungen oder Ländern selbst. Die Tatsache allein, daß die Anwendung bibliometrischer Methoden zur Wissenschaftsevaluation nicht mehr aufzuhalten ist, macht es notwendig, das überaus komplexe Thema für alle beteiligten Gruppen einerseits leicht verständlich und schnell lesbar, andererseits aber auch in der gebotenen Tiefe des komplexen Themas aufzubereiten. Ein kleines Büchlein für den schnellen Zugang zur Thematik haben Dirk Tunger und Rafael Ball bereits an anderer Stelle vorgelegt (Rafael Ball, Dirk Tunger: Bibliometrische Analysen - Daten, Fakten und Metho- den : Grundwissen Bibliometrie für Wissenschaftler, Wissenschaftsmanager, Forschungsein- richtungen und Hochschulen. Jülich, 2005). Nun folgt hier eine ausführlichere Darlegung der Materie, die es aber noch erlauben soll, auch vom Nicht-Bibliometriker gelesen und verstanden zu werden. Deshalb haben wir auf ein allzu tiefes Eindringen in die mathematisch-statistische Grundlage der Bibliometrie verzichtet und das Buch so gestaltet, dass es je nach Interessenlage selektiv und kapitelweise gelesen werden kann. Trotzdem haben wir versucht, zumindest in Grundzügen die Breite des Themas darzulegen, auf grundlegende Formeln und Gesetze nicht gänzlich zu verzichten und vor allem die aktuelle und relevante Literatursicht auszubreiten und zu zitieren. Dieses Buch wendet sich deshalb gleichermaßen an den Fachwissenschaftler selbst, ebenso wie an den interessierten Wissenschaftspolitiker und -manager, also an all diejenigen, die quantitative Wissenschaftsbewertung einsetzen, Empfehlungen und Richtlinien für Promotionen und Berufungen erlassen oder in solche Verfahren involviert sind. Es eignet sich aber auch für Studenten der Informations- und Bibliothekswissenschaft und ist für den innovativen Fachreferenten in Bibliotheken unentbehrlich. Das Werk richtet sich deshalb weniger an den spezialisierten Bibliometriker. Es ist entstanden aus dem Wunsch, die Interpretation bibliometrischer Indikatoren durchschaubarer zu machen, ihre Anwendung und Interpretation zu erleichtern und das interessierte Publikum mit den Möglichkeiten bibliometrischer Analysen der wissenschaftlicher Kommunikation fundiert bekanntzumachen. In der Einführung des Buches wird der Begriff der Bibliometrie erläutert und eine kurze Geschichte ihrer Entstehung dargelegt. Im zweiten Kapitel werden Autoren als Produzenten wissenschaftlicher Publikationen, die Begriffe der Autorenschaft und der Mehrautorenschaft, das Thema Frauen in der Wissenschaft und der Einfluss und die Bedeutung internationaler wissenschaftlicher Kooperationen als Paradigma der heutigen Kommunikationsprozesse vor dem Hintergrund bibliometrischer Messungen bearbeitet.
Conference Paper
Full-text available
The number and types of Open Source Software (OSS) are increasing year by year. The information technology (IT) industry uses OSS and utilize intellectual properties and human resources across organizational boundaries. In present OSS expands across industry boundaries and is used in more industries such as the energy industry, automotive industry, medical industry. In addition, some of OSS have been developed with new concepts. Some OSS organizations such as the Linux Foundation, the Cloud Native Computing Foundation and the Japan OSS Promotion Forum provide categories, but contributors in them are also time-consuming to provide the latest information. Meanwhile, it is inconvenient for every Open Source Program Offices (OSPO), which oversees open source software management and strategy, to update manual OSS categorization frequently. Thus, OSPOs want to get a universal methodology to restructure OSS categories. This research provides the current categories of OSS and its methodology in order to build a consistent strategy. This methodology aims to be a methodology to support in the standardization of OSS categorization.
Article
In order to examine potential effects of methodological choices influencing developments in relative citation scores for countries, a fixed journal set comprising of 3232 journals continuously indexed in the Web of Science from 1981 to 2014 is constructed. From this restricted set, a citation database depicting the citing relations between the journal publications is formed and relative citation scores based on full and fractional counting are calculated for the whole period. Previous longitudinal studies of citation impact show stable rankings between countries. To examine such findings coming from a dynamic set of journals for potential “database effects”, we compare them to our fixed set. We find that relative developments in impact scores, country profiles and rankings are both very stable and very similar within and between the two journal sets as well as counting methods. We do see a small “inflation factor” as citation scores generally are somewhat lower for high-performing countries in the fixed set compared to the dynamic set. Consequently, using an ever-decreasing set of journals compared to the dynamic set, we are still able to reproduce accurately the developments in impact scores and the rankings between the countries found in the dynamic set. Hence, potential effects of methodological choices seem to be of limited importance compared to the stability of citation networks.
Chapter
The evaluation of research and development activities is one of the most challenging, and significant issues in R&D managment today. The last few decades have seen considerable efforts in the development and testing of various measures and quantitative techniques in order to aid in project selection, project appraisal, and measurement of research outputs. These techniques are distributed across a methodological spectrum, ranging from the qualitative case study to the highly quantitative econometric analyses.
Article
Traditionally, the United States has not set a specific national technology policy or plan like those seen in many other nations. However, the US Government spends large amounts of money on research and development (R&D) through such agencies as Department of Defense (DoD) and National Aeronautics and Space Administration (NASA). This case study investigates the relationship between defense and space R&D expenditures and national technology advancement in the aerospace industry. The lag between R&D expenditures and technology advancement is also examined and modeled.
Article
We weighted the output of SCI items from Australian universities using journal impact factors. This provides us with an accessible quality indicator of science journal publishing, and allow us to scale for institutional size in terms of output and research staff. Use of this indicator for the 20 pre-1987 Australian universities demonstrates that although some universities rank highly on output, when scaled for institutional size they are overtaken by some of the smaller, more recently established universities.
Article
From an analysis of bibliographic data on 430 journal articles on liquid crystals covered inPhysics Abstracts 1976 and the 4729 citations to them up to the end of 1987, we have identified the geographic origin, the prominent institutions, language and journal-wise distribution of the papers, the citedness of these papers, and the distribution of citations as a time series for the highly cited papers. We have also analysed the 126 papers published by authors from India, Canada, Australia, Israel, Japan and the United Kingdom and covered inPhysics Abstracts 1978, and the 1154 citations to them up to 1987. Unlike in most other high tech areas of physics, in LC research the difference in performance between the USA and the other leading countries is not very pronounced. Publication data from 1976, 1978 and 1985 reveal that LC literature is on the rise and that the percentage share of the Soviet Union is rising fast and that of the USA is on the decline.
Article
Full-text available
In order to attribute journals to specialties in a dynamic journal set by using aggregated journal-journal citations derived from theScience Citation Index, it is necessary to complement the multi-variate analysis of this data with a time-serices perspective. This calls for a more analytical approach to the problem of choice among the many possible parameters for clustering. Changes in the disciplinary structure of science are tracked by using thedifferences among the multi-variate analyses for the various years. It is impossible to attribute change systematically to structure, noise, or deviance if these uncertainties are not clearly definedex ante. The study discusses the various choices which have to be made, in both conceptual and methodological terms. In addition to hierarchies among journals, one has to assume heterarchy among journal groups (and their centroids). For comprehensive mapping, a concept of macro-journals as representations of a density of points in the multi-dimensinoal space is defined. Empirical reslts indicate the feasibility of dynamic journal-journal mapping by using these methods.
Article
Full-text available
In 1987, an analysis of the CHI/NSFScience Literature Indicators Data-Base by the author and his colleagues suggested that the UK's percentage share of the world publication and citation totals had continued to fall over 1981–84, although at a slower rate than previously. That finding has recently been challenged byBraun, Glänzel andSchubert who, by combining 28 publication-based indicators, concluded that there was no statistically significant evidence for such a decline. This paper examines the reasons for the discrepancy. It is argued that the methodology ofBraun et al. is seriously flawed, as well as being inconsistent with work that they have published elsewhere. By adopting a more consistent and realistic set of indicators and applying them to the data ofBraun et al., one arrives at results entirely consistent with those derived from the CHI/NSF data-base.
Article
Full-text available
With respect to the issue of whether the scientometric measurement of the decline of British science is an artifact of the specific database and underlying assumptions in methods, I argue that there are fewer analytical objections against measurement by usingSciSearch Online than against other methods (based on the fixed journal set and fractional counting). The measurement of international co-authorship, i.e. a network indicator, should not be confounded with measurement of performance of a single nation. The time series for the different subsets of UK-publications, which have been proposed, are given. None of the indicators can be shown to exhibit a trend (in contrast to a drift). The hypothesis of a decline has therefore to be rejected.
Article
Full-text available
How can the EC's cclence polzcy mteruentzons be evaluated at the supranational level? For scientometrics, this problem has raised various interesting research questions ( I ) how can the system(s) be mearuled at the national and at the supranational level, (2) how me the dynamics between these two levels of analysis to be understood; and (3) if a European system does emerge, how should it be decomposed with reference to explunatory factors like policy interventions? In addition to descriptive statistics, these questions require a systems theoretical elaboration.Entropy measures from chaos theory enable usto integrate the systems theoretical perspective with scientometric observations. The conclusions are that the EC does not function currently as a single publication system, but the development of international co-authorship relations among member countries does exhibit a systematic pattern. The co-authorship network has grown vigorously during the 1980s. notably in terms of co-authored article. The possible effects of external interuention in tuch a process are discussed.
Article
Evidence from Greece suggests that after its accession to the EC the country benefited from a limited transfer of resources in form of R & D subsidies, which were heavily concentrated both sectorally and institutionally, hardly leading to marketable innovations. A detailed analysis shows though that the country benefited more from the EC technology policy in the form of a) subsidies for R & D infrastructure and b) quick adaptation of its institutions to new challenges. Based on this experience it is suggested that STRIDE, a Community R & D programme for less favoured regions, be evaluated with both cohesion and excellence indicators.
Article
The study discusses the application of various forms of time series analysis to national performance data for EEC countries and the US. First, it is shown that at the aggregated level, a straightforward relation exists between output and input, which varies with time. Various analytical techniques to account for the time factor are discussed. By using information theory, a simple formula can be derived which gives the best prediction for the following year's data. Subsequently, this model is extended to multi-variate forecasting of distributions. Additionally, it can be shown by using this method that in terms of percentage of world share of publications the hypothesis that the EEC develops as a single publication system has to be rejected. However, when co-authorship relations among EEC member countries are used as an indicator, the predominance of a system is suggested.
Article
Full-text available
Previous research by the authors shows that the output and impact of British science has declined significantly over the last decade relative to other major industrial nations. This paper responds to the claim made by Leydesdorff elsewhere in this issue of the journal that British research output has instead exhibited “a remarkable increase” from 1981 onwards. A detailed analysis of the available statistical evidence shows that Leydesdorffs claims are spurious, and serves as a caution to those attempting to use on-line bibliometric data-bases for science-policy purposes without first processing the ‘raw’ figures.
Article
Full-text available
The dynamic mapping of science using the data in theScience Citation Index was put on the research agenda of science studies byDe Solla Price in the mid 1960s. Recently, proponents of co-citation cluster analysis have claimed that in principle their methodology makes such mapping possible. The study examines this claim, both methodologically and theoretically, in relation to other means of mapping science. A detailed study of a co-citation map, its core documents' citation patterns and the related journal structures, is presented. At these three levels of possible study of aggregates of citations, an analysis is pursued for the years 1978 to 1984. The many different statistical methods which are in use for the analysis of the respective datamatrices—such as cluster analysis, factor analysis and multidimensional scalling—are assessed with a view to their potential to contribute to a better undérstanding of the dynamics at the different levels in relation to each other. This will lead to some recommendations about methods to use and to avoid when we aim at a comprehensive mapping of science. Although the study is pursued at a formal and analytical level, in the conclusions an attempt is made to reflect on the results in terms of further substantial questions for the study of the dynamics of science.
Article
Full-text available
Recent data on the comparative scientific performance of different nations suggest that the decline in British basic science apparent during the 1970s began to level off at the start of the 1980s. However, rather than heralding the beginning of a revival, the evidence suggests that this is merely a temporary halt to the long-term slide.
Article
Full-text available
New and more methodologically advanced publication and citation data for the period up to 1984 indicate that the relative decline of British science is continuing, albeit at a slower rate than in the 1970s.
Article
Full-text available
In principle, a scientometric transaction matrix can be modelled by assuming that the number of transactions is the result of independent row and column contributions. More often one is primarily interested in the cross-structural relations between the participating entities, whereas the row and column marginls are of lesser or no importance. The values of the residuals after fitting an independence model to a complete transaction matrix can be analyzed by correspondence analysis to investigate the structure of the transactions between the rows and columns, after correcting for their marginal frequencies. Recently a modification of correspondence analysis has been developed, quasi-correspondence analysis, which seems quite suitable for the analysis of citation-based transaction matrices which are incomplete or in which the incorporation of certain transactions may seem inappropriate. An illustration of both data analysis-techniques will be given using a journal-to-journal citation matrix.
Article
Full-text available
This paper presents a methodological analysis of the latest update of the CHI/NSF Science Literature Indicators Data-Base. The data-base contains a range of publication and citation indicators borken down by country and field or subfield, and now convers the period from 1973 to 1984. It can be used to draw comparisons of the changing output and impact of basic research in different countries. Earlier applications of the data-base have been constrained by various technical limitations, and have been subject to certain criticism. In this article, after some conceptual analysis of what aspects of scientific performance the different indicators relate to, we show that much of the criticism is misplaced. We also describe subsequent methodological improvements to the indicators and the effect these have on the policy use that can be made. Finally, we examine what the latest statistics reveal about the relative international standing of seven leading scientific nations.
Article
National science policy efforts can be understood, from an information science point of view, as deliberate attempts to change the relationship of research groups to the international literature over time. These relationships can be either active (publishing and citing) or passive (being cited, co-cited, etc.). This study examines the development of two highly comparable Dutch national research programs (one on "wind energy" and one on "solar energy") in terms of factors which can be associated with such relationships. The different outcomes of the two programs are analyzed in terms of the emergence of new journals in both areas. The key findings are: (1) An international core literature emerges between 1974 and 1984 in the solar but not in the wind energy literature. (2) The Dutch solar energy researchers, both in their journal articles and in their reports, link themselves to this literature. (3) The contributions of the solar researchers are in turn recognized in the international literature. (4) The wind energy researchers have less opportunity to participate with a clearly defined external professional community.
Article
This paper describes the evolution over six decades of ionospheric physics — a complex, well-funded field — and concentrates upon research at US universities in the years following World War II. An historical overview is followed by presentation of data gathered from literature samples and interviews, including funding acknowledgement data. Responses from twenty successful ionosphere research leaders as to their views on (and their strategies for obtaining) funding, the role of funding programme managers, and the nature of various funding agencies, are then summarized. There is and has been a plurality of both research centres and funding sources in ionospheric physics, and there is constant interaction both in ideas and in funding between basic research and civilian and military desires. Finally, to discuss entrepreneurs and resources in scientific research, four cases are briefly introduced concerning building research teams and facilities, defining new directions in science, and introducing large-scale social organization of research. It is argued that the situation in ionospheric physics is probably similar to wide areas of the physical and engineering sciences in the US.
Article
Beginning in 1957, the literature explosion in science eliminated many personal subscriptions to journals and scientists with interdisciplinary interests demanded new approaches to current awareness. One of these, Current Contents now forms a major link in scientific communications. New ‘Selective Dissemination of Information’ services have helped to mechanize the process of scanning titles for keywords, and new tools for retrospective searching have facilitated historical studies of the scientific literature. The journal literature continues to evolve by means of a process analogous to natural selection.
Article
Two journal-to-journal matrices for psychology in 1950 and 1960 are analyzed in terms of structural equivalence. Structurally nonequivalent positions have a strong correspondence with a categorization based on the intended audiences and objectives of these journals, supporting the hypothesis that journals of a discipline function as a status-role relational system. The hypothesis that interdisciplinary journals are distant from journals of a field is also supported. However, the hypothesis that journal networks have a core-periphery structure is not supported. The network of psychological journals has a discernable core, but not a clear core-periphery structure. Some propositions concerning the causes and consequences of the structure of journal-to-journal networks for fields are advanced.
Article
As the costs of certain types of scientific research have escalated and as growth rates in overall national science budgets have declined, so the need for an explicit science policy has grown more urgent. In order to establish priorities between research groups competing for scarce funds, one of the most important pieces of information needed by science policy-makers is an assessment of those groups' recent scientific performance. This paper suggests a method for evaluating that performance.After reviewing the literature on scientific assessment, we argue that, while there are no simple measures of the contributions to scientific knowledge made by scientists, there are a number of ‘partial indicators’ — that is, variables determined partly by the magnitude of the particular contributions, and partly by ‘other factors’. If the partial indicators are to yield reliable results, then the influence of these ‘other factors’ must be minimised. This is the aim of the method of ‘converging partial indicators’ proposed in this paper. We argue that the method overcomes many of the problems encountered in previous work on scientific assessment by incorporating the following elements: (1) the indicators are applied to research groups rather than individual scientists; (2) the indicators based on citations are seen as reflecting the impact, rather than the quality or importance, of the research work; (3) a range of indicators are employed, each of which focusses on different aspects of a group's performance; (4) the indicators are applied to matched groups, comparing ‘like’ with ‘like’ as far as possible; (5) because of the imperfect or partial nature of the indicators, only in those cases where they yield convergent results can it be assumed that the influence of the ‘other factors’ has been kept relatively small (i.e. the matching of the groups has been largely successful), and that the indicators therefore provide a reasonably reliable estimate of the contribution to scientific progress made by different research groups.In an empirical study of four radio astronomy observatories, the method of converging partial indicators is tested, and several of the indicators (publications per researcher, citations per paper, numbers of highly cited papers, and peer evaluation) are found to give fairly consistent results. The results are of relevance to two questions: (a) can basic research be assessed? (b) more specifically, can significant differences in the research performance of radio astronomy centres be identified? We would maintain that the evidence presented in this paper is sufficient to justify a positive answer to both these questions, and hence to show that the method of converging partial indicators can yield information useful to science policy-makers.
Article
The pattern of bibliographic references indicates the nature of the scientific research front.
Problems with the “measurement” of national scientific performanceOn-line approaches to measuring national scientific output—A cautionary talePerformance figures for British science
  • L Leydesdorff
  • J Anderson
  • P M D Collins J Irvine
  • P A Isard
  • B R Martin
  • F Narin
On the Measurement of Research Performance: The Use of Bibliometric Indicators, Research Policy Unit A 32-Country Comparative Evaluation of Publishing Performance and Citation Impact
  • J M Burger
  • J G Frankfort
  • A F J Van
  • T Braun
  • W Gl~nzel
  • A Schubert
  • M Callon
  • L Leydesdorff
J. M. BURGER, J. G. FRANKFORT, A. F. J. Van RAAN, On the Measurement of Research Performance: The Use of Bibliometric Indicators, Research Policy Unit, Leiden, 1983; T. BRAUN, W. GL~NZEL, A. SCHUBERT, Seientometrie Indicators. A 32-Country Comparative Evaluation of Publishing Performance and Citation Impact. World Scientific, Singapore/Phila-delphia, 1985; M. CALLON, L. LEYDESDORFF, 'La recherche frangaise, est-elle en bonne sant~, : La Recherche, number 86, March 1987, 412-19.
A review of Techniques for Evaluating National Performance in Basic Research, with Case Studies in Genetics and Solid State Physics
  • Abrc
  • Evaluation
ABRC, Evaluation ofNationalPerformance in Basic Research. A review of Techniques for Evaluating National Performance in Basic Research, with Case Studies in Genetics and Solid State Physics. ABRC Science Policy Studies No. 1, London, 1986; B. R. MARTIN e.a., 1987. Op. cir., note 10.
An alternative quantitative approach to the assessment of national perform-ance in basic research; Ciba-Foundation Symposium on The Evaluation of Scientific Research The decline of British analytical chemistry: Fact or artifact, Analytical Proceedings
  • T Braun
  • W Glnnzel
  • A Schubert
T. BRAUN, W. GLNNZEL, A. SCHUBERT, An alternative quantitative approach to the assessment of national perform-ance in basic research; Ciba-Foundation Symposium on The Evaluation of Scientific Research, London, 6-8 June 1988; T. BRAUN, A. SCHUBERT, S. ZSINDELY, The decline of British analytical chemistry: Fact or artifact, Analytical Proceedings, 26/3 (1989) 87-91.
Assessing basic research: Some partial indicators of scientific progress in radio astronomyOn the Measurement of Research Performance: The Use of Bibliometric IndicatorsScientometric Indicators. A 32-Country Comparative Evaluation of Publishing Performance and Citation Impact
  • R Martin
  • Irvine H F Moed
  • W J M Burger
  • J G Frankfort
  • A F J Van
Problems with the “measurement” of national scientific performance,Science and Public PolicyOn-line approaches to measuring national scientific output—A cautionary talePerformance figures for British science
  • L J Leydesdorff
  • P M D Anderson
  • Collins J Irvine
  • P A Isard
  • B R Martin
  • F Narin
  • K Stevens L
  • Leydesdorff
Sage Series of Quantitative Application in the Social Sciences, Nr. 07-010
  • A L I J Langbein
  • Lichtman
  • P Inference
  • Eeden Van
  • H J M Huttner
  • Multilevel Research
Charting the decline of British scienceEvaluation of National Performance in Basic Research. A review of Techniques for Evaluating National Performance in Basic Research, with Case Studies in Genetics and Solid State Physics The continuing decline of British science
  • B J Irvine
  • T Martin
  • R Peacock
  • B R Turner
  • J Martin
  • F Iprvine
  • C Narin
  • Sterritt
Various methods for the mapping of science Quasi-correspondence analysis on square scientometric transaction matrics Structural equivalence in a journal network
  • L Leydesdorff R
  • J Tijssen
  • Leeuw De
  • A F J Van
  • Raan P Doreian
  • T J Fararo
Ecological Inference, Sage Series of Quantitative Application in the Social Sciences
  • L I Langbein
  • A J Lichtman