Raf Guns

University of Antwerp | UA · ECOOM

This study contributes to the recent discussions on indicating interdisciplinarity, that is, going beyond catch‐all metrics of interdisciplinarity. We propose a contextual framework to improve the granularity and usability of the existing methodology for interdisciplinary knowledge flow (IKF) in which scientific disciplines import and export knowle...

This study presents knowledge diffusion analyses for researchers who have been active in the social sciences and humanities. We compare a network based on switches between the main disciplinary classifications of the documents authored throughout their careers to a discipline similarity network. We find that researchers are not exclusively switchin...

This study compares citation-based and expert-based journal metrics as predictors of peer-assessed research quality based on 154,826 journal articles submitted to UK’s Research Excellence Framework (REF) 2021. The Finnish expert-based Julkaisufoorumi (JUFO) level ratings of journals determined by expert-panels per field produce scores that correlat...

Using different methods to assign disciplines to publications can influence bibliometric analyses. In this study, we test the influence of applying two different types of classification on the disciplinary collaboration rates of researchers from the Social Sciences and Humanities. Two different classification types are contrasted: organisational cl...

In this paper, we study how research topics and trends have evolved in the field of the science of team science (SciTS). Over the past 12 years, the International Science of Team Science conference has been making efforts to understand and enhance the processes and outcomes of collaborative team science. We argue that the sessions and papers in aca...

This paper investigates different uses of the Journal Impact Factor (JIF) in national journal rankings and discusses the merits of supplementing metrics with expert assessment. Our focus is national journal rankings used as evidence to support decisions about the distribution of institutional funding or career advancement. The seven countries under...

This paper investigates different uses of the Journal Impact Factor (JIF) in national journal rankings and discusses the merits of supplementing metrics with expert assessment. Our focus is national journal rankings used as evidence to support decisions about the distribution of institutional funding or career advancement. The seven countries under...

The availability of Open Access journals in the various fields of knowledge in Clarivate Analytics’ Web of Science is hypothesized to present strong inequalities, thus affecting the choice of journals by researchers wishing to publish their research results in Open Access. The first objective of this research was to contrast this hypothesis, by cro...

This study contributes to the recent discussions on indicating interdisciplinarity, i.e., going beyond mere metrics of interdisciplinarity. We propose a multi-dimensional and contextual framework to improve the granularity and usability of the existing methodology for quantifying the interdisciplinary knowledge flow (IKF) in which scientific discip...

Interdisciplinary research is widely recognized as necessary to tackle some of the grand challenges facing humanity. It is generally believed that interdisciplinarity is becoming increasingly prevalent among Science, Technology, Engineering, and Mathematics (STEM) fields. However, little is known about the evolution of interdisciplinarity in the So...

In this study we explore the disciplinary diversity present within subject specialties in the social sciences and humanities. Subject specialties are operationalized as textually coherent clusters of documents. We apply topic modelling to textual information on the individual document level (titles and abstracts) to cluster a multilingual set of ro...

This article presents an analysis of the uptake of the GPRC label (Guaranteed Peer Reviewed Content label) since its introduction in 2010 until 2019. GPRC is a label for books that have been peer reviewed introduced by the Flemish publishers association. The GPRC label allows locally published scholarly books to be included in the regional database...

In this study we explore the disciplinary diversity present within subject specialties in the social sciences and humanities. Subject specialties are operationalized as textually coherent clusters of documents. We apply topic modelling to textual information on the individual document level (titles and abstracts) to cluster a multilingual set of ro...

In this study we explore the disciplinary diversity present within subject specialties in the social sciences and humanities. Subject specialties are operationalized as textually coherent clusters of documents. We apply topic modelling to textual information on the individual document level (titles and abstracts) to cluster a multilingual set of ro...

When quantifying the level of interdisciplinarity for scientific research, most established indicators employ a three‐element diversity framework, namely variety, balance, and disparity, each of which captures a distinct but insufficient element. Among three, disparity, i.e. how different (dissimilar) the categories within a system are, is the most...

The social sciences involve multiple literatures, including journals with a more international and those with a more local orientation. Quality standards in research assessment have largely been borrowed from STEM fields and generally favour internationally oriented journals with high international visibility and scientific impact, to the detriment...

We study how different document representation techniques affect the outcomes of clustering based on textual information. Our dataset consists of titles and abstracts for 15,907 publications from the social sciences and humanities. We compare established document representation techniques such as TF-IDF and Latent Semantic Indexing with word and do...

Interdisciplinary research is widely recognized as necessary to tackle some of the grand challenges facing humanity. It is generally believed that interdisciplinarity is becoming increasingly prevalent among STEM fields. However, little is known about the evolution of interdisciplinarity in the Social Sciences. Also, how interdisciplinarity and its...

It is generally believed that the tide of interdisciplinarity is rising and becomes increasingly prevalent among various disciplines in natural and biomedical sciences. However, for the social sciences and humanities (SSH) limited evidence supports such a statement from bibliometric perspectives. Also, it has seldom been quantified how interdiscipl...

This study compares publisher ratings to the visibility and impact of individual books, based on a 2017 dataset from three Nordic PRFS systems (Denmark, Norway, and Finland). Although there are Journal Impact Factors (JIFs) for journals, there is no similar indicator for book publishers. National publisher lists are used instead to account for the...

This study compares publication pattern dynamics in the social sciences and humanities in five European countries. Three are Central and Eastern European countries that share a similar cultural and political heritage (the Czech Republic, Slovakia, and Poland). The other two are Flanders (Belgium) and Norway, representing Western Europe and the Nord...

Purpose This study investigates an approach to book metrics for research evaluation that takes into account the complexity of scholarly monographs. This approach is based on work sets – unique scholarly works and their within-work related bibliographic entities – for scholarly monographs in national databases for research output. Design/methodolog...

Open Science is an umbrella term that encompasses many recommendations for possible changes in research practices, management, and publishing with the objective to increase transparency and accessibility. This has become an important science policy issue that all disciplines should consider. Many Open Science recommendations may be valuable for the...

Despite the centrality of disciplinary classifications in bibliometric analyses, it is not well known how the choice of disciplinary classification influences bibliometric representations of research in the social sciences and humanities (SSH). This is especially crucial when using data from national databases. Therefore we examine the differences...

We compare two supervised machine learning algorithms—Multinomial Naïve Bayes and Gradient Boosting—to classify social science articles using textual data. The high level of granularity of the classification scheme used and the possibility that multiple categories are assigned to a document make this task challenging. To collect the training data,...

Purpose This paper presents an overview of different kinds of lists of scholarly publication channels and of experiences related to the construction and maintenance of national lists supporting performance-based research funding systems. It also contributes with a set of recommendations for the construction and maintenance of national lists of jour...

Open access (OA) has mostly been studied by relying on publication data from selective international databases, notably Web of Science (WoS) and Scopus. The aim of our study is to show that it is possible to achieve a national estimate of the number and share of OA based on institutional publication data providing a comprehensive coverage of the pe...

The purpose of this study is to expand the knowledge on publication patterns in Social Sciences and Humanities (SSH) in terms of the coverage of journal articles in Web of Science Core Collection (WoS) and their distribution in citation indexes. Within Journal Citation Reports (JCR), we focus deeper on quartile ranks based on the Journal Impact Fac...

We investigate the state of multilingualism across the social sciences and humanities (SSH) using a comprehensive data set of research outputs from seven European countries (Czech Republic, Denmark, Finland, Flanders [Belgium], Norway, Poland, and Slovenia). Although English tends to be the dominant language of science, SSH researchers often produc...

National bibliographic data bring numerous opportunities for science studies, especially when integrating data from multiple data sources. The use of multiple data sources, however, is hindered by the lack of interoperability. Although progress has been made in developing persistent international identifiers such as ISBN, DOI, and GRID, the interop...

The need for a comprehensive infrastructure for scholarly publication information has been on the EU’s agenda for a long time. Also, the European Commission’s open science policy highlights the necessity of a good information base to follow up open access publishing across Europe. However, an all-inclusive information infrastructure on research pub...

W tym artykule omawiamy etykiety z otwartymi danymi recenzentów, czyli praktykę ujawniania danych recenzentów w monografiach naukowych, która jest powszechna w krajach Europy Środkowej i Wschodniej. Celem badania było sprawdzenie, czy etykieta z otwartymi danymi recenzentów jest rodzajem etykiety z informacją o recenzowaniu (na podobieństwo tej sto...

In this article we discuss the five yearly screenings for publications in questionable journals which have been carried out in the context of the performance-based research funding model in Flanders, Belgium. The Flemish funding model expanded from 2010 onwards, with a comprehensive bibliographic database for research output in the social sciences...

In performance-based research funding systems evidence of peer review is often considered a requirement for publications to be included. Originating from the sciences, pre-publication peer review is very common in the publishing process, also in the social sciences and humanities. Sometimes, however, it is ambiguous whether a publication is peer-re...

The purpose of this experiment is to assess whether and to what extent it is feasible to make use of supervised machine learning to classify social science journal articles into fine-grained disciplinary categories. Classifying scientific articles according to disciplines is most commonly done by making use of a proxy such as Clarivate Analytics's...

The purpose of this paper is to provide a comprehensive picture of open access publishing in Finland. Data consists of the complete national peer-reviewed output of 48177 articles and books from 14 Finnish universities in 2016-2017 stored in the VIRTA Publication Information Service. Each publication record contains an indication if it is openly av...

This article discusses the open-identity label, i.e., the practice of disclosing reviewers’ names in published scholarly books, a common practice in Central and Eastern European countries. This study’s objective is to verify whether the open-identity label is a type of peer-review label (like those used in Finland and Flanders, i.e., the Flemish pa...

Dataset B1 (Authors) from the integrated quantitative and qualitative data. (XLSX)

Dataset B2 (Reviewers) from the integrated quantitative and qualitative data. (XLSX)

This article presents a cohort analysis to study changes in the publication patterns of scholars working at a social sciences and humanities (SSH) university department or research unit in Flanders, Belgium. Starting from a comprehensive bibliographic database, we analyze the peer review status, publication language, publication type (journal artic...

The BOF-key is the performance-based research funding system that is used in Flanders, Belgium. In this paper we describe the historical background of the system, its current design and organization, as well as its effects on the Flemish higher education landscape. The BOF-key in its current form relies on three bibliometric parameters: publication...

This report results from a comparison of the list of journals submitted to the Authoritative Panel (in Dutch ‘Gezaghebbend Panel’ or GP) of the VABB-SHW (Vlaams Academisch Bibliografisch Bestand voor de Sociale en Humane Wetenschappen). ECOOM-Antwerp submitted this list of journals to the GP in July 2018 (publications from years 2008-2017) and comp...

In May 2016, an article published in Scientometrics, titled ‘Taking scholarly books into account: current developments in five European countries’, introduced a comparison of book evaluation schemes implemented within five European countries. The present article expands upon this work by including a broader and more heterogeneous set of countries (...

In earlier research, we introduced methods by which the cognitive distance between two sets of publications can be calculated (Rahman, Guns, Rousseau, & Engels, 2017). One method involves determining the barycenters of the publication sets on a journal-based map of science, and calculating the distance between them. Among other possible uses, the r...

From 2013 – 2014 onwards, our group (ECOOM - UAntwerpen) has been monitoring Predatory Open Access publication patterns in Flemish (Belgium) SSH scholarship. In light of the Flemish Performance Based Research Funding System, these screening exercises are conducted to assist university review boards with the decision-making processes concerning what...

This article provides an overview of national bibliographic databases that include data on research output within social sciences and humanities (SSH) in Europe. We focus on the comprehensiveness of the database content. Compared to the data from commercial databases such as Web of Science and Scopus, data from national bibliographic databases (e.g...

We study the discrepancy between two ways of classifying publications in the social sciences and humanities (SSH): on the basis of the contents of publications and publication channels (cognitive classification) or on the basis of the organizational structure of departments, faculties etc. (organizational classification). Using data for the period...

This study investigates patterns in the language and type of social sciences and humanities (SSH) publications in non-English speaking European countries to demonstrate that such patterns are related not only to discipline but also to each country’s cultural and historic heritage. We investigate publication patterns that occur across SSH publicatio...

Of interest for this report is to what extent malpractices within open-access scholarly publishing is a matter of concern for Flemish academia. To address this question, we study to what extent scholarly communication of social science and humanities (SSH) scholars within Flanders resides in outlets of questionable quality. Since 2013 ECOOM-UAntwer...

This article compares six informetric approaches to determine cognitive distances between the publications of panel members and those of research groups in discipline-specific research evaluation. We used data collected in the framework of six completed research evaluations from the period 2009-2014 at the University of Antwerp as a test case. We d...

The evolution of a collaboration network is to some extent steered by the network topology itself. This is the reason behind the success of network evolution models and approaches to link prediction. At the same time, some changes are due to exogenous factors (i.e., factors external to the network itself). In this paper, we explore changes in the c...

This document provides an overview of European databases and repositories for research output within the social sciences and humanities. This overview is a result of a survey conducted within the framework of European Network for Research Evaluation in the Social Sciences and Humanities (ENRESSH, www.enressh.eu). This overview can be used to identi...

This technical report is prepared in the context of A. I. M. Jakaria Rahman’s PhD project on Determining cognitive distance between publication portfolios of evaluators and evaluees in research evaluation: Exploration of informetric methods. Similar technical reports on Biology, Biomedical Sciences, Pharmaceutical Sciences, Physics and Veterinary S...

This technical report is prepared in the context of A. I. M. Jakaria Rahman’s PhD project on Determining cognitive distance between publication portfolios of evaluators and evaluees in research evaluation: Exploration of informetric methods. Similar technical reports on Biology, Biomedical Science, Chemistry Pharmaceutical Sciences and Physics depa...

This technical report is prepared in the context of A. I. M. Jakaria Rahman’s PhD project on Determining cognitive distance between publication portfolios of evaluators and evaluees in research evaluation: Exploration of informetric methods. Similar technical reports on Biology, Biomedical Sciences, Pharmaceutical Sciences, Chemistry and Veterinary...

This technical report is prepared in the context of A. I. M. Jakaria Rahman’s PhD project on Determining cognitive distance between publication portfolios of evaluators and evaluees in research evaluation: Exploration of informetric methods. Similar technical reports on Biomedical Sciences, Chemistry Pharmaceutical Sciences, Physics and Veterinary...

This technical report is prepared in the context of A. I. M. Jakaria Rahman’s PhD project on Determining cognitive distance between publication portfolios of evaluators and evaluees in research evaluation: Exploration of informetric methods. Similar technical reports on Biology, Chemistry Pharmaceutical Sciences, Physics and Veterinary Sciences dep...

This technical report is prepared in the context of A. I. M. Jakaria Rahman’s PhD project on Determining cognitive distance between publication portfolios of evaluators and evaluees in research evaluation: Exploration of informetric methods. Similar technical reports on Biology, Biomedical Sciences, Chemistry, Physics, and Veterinary Sciences depar...

We study the problem of determining the cognitive distance between the publication portfolios of two units. In this article we provide a systematic overview of five different methods (a benchmark, Euclidean distance approach, distance between barycenters in two and in three dimensions, distance between similarity-adapted publication vectors, and we...

When research groups are evaluated by an expert panel, it is an open question how one can determine the match between panel and research groups. In this paper, we outline two quantitative approaches that determine the cognitive distance between evaluators and evaluees, based on the journals they have published in. We use example data from four rese...

In Rahman, Guns, Rousseau, and Engels (2015) we investigated, among other things, the following research question: ‘How can we quantify the overlap of expertise between two entities (e.g., a research group and a panel) using publication data?’. In order to answer this research question we considered two approaches: one based on barycenters in two d...

Purpose: This study aims to answer the question to what extent different types of networks can be used to predict future co-authorship among authors. Design/methodology/approach: We compare three types of networks: unweighted networks, in which a link represents a past collaboration; weighted networks, in which links are weighted by the number of j...

In a previous article (Rahman, Guns, Rousseau, and Engels, 2015) we described several approaches to determine the cognitive distance between two units. One of these approaches was based on what we called barycenters in N dimensions. The present note corrects this terminology and introduces the more adequate term similarity-adapted publication vecto...

The current report presents the results of this monitoring exercise in view of VABB-SHW version VI, which will contain publications from the time period 2005–2014. This report provides a detailed comparison of (1) the journals published by publishers listed on Beall’s list of POA publishers and the journals on Beall’s list of stand-alone journals a...

We explore cognitive distance between an expert panel and the unit of assessment in discipline-specific research evaluation. We investigate the usefulness of overlay mapping on a global map of science based on Web of Science subject categories, and a global Journal map to measure the overlap of expertise, and introduce a method to determine an enti...