Joeran Beel

Joeran Beel
University of Siegen · Department of electrical engineering and computer science

PhD in Computer Science

About

147
Publications
133,205
Reads
How we measure 'reads'
A 'read' is counted each time someone views a publication summary (such as the title, abstract, and list of authors), clicks on a figure, or views or downloads the full-text. Learn more
4,109
Citations
Additional affiliations
December 2016 - May 2020
Trinity College Dublin
Position
  • Professor (Assistant)
April 2016 - November 2016
National Institute of Informatics
Position
  • PostDoc Position
April 2016 - March 2017
University of Konstanz
Position
  • PostDoc Position
Education
February 2009 - December 2014
Otto-von-Guericke University Magdeburg
Field of study
  • Computer Science
October 2005 - September 2006
Lancaster University
Field of study
  • Project Management
July 2004 - December 2004
Macquarie University
Field of study
  • Business Administration

Publications

Publications (147)
Preprint
Full-text available
To combat the rising energy consumption of recommender systems we implement a novel alternative for k-fold cross validation. This alternative, named e-fold cross validation, aims to minimize the number of folds to achieve a reduction in power usage while keeping the reliability and robustness of the test results high. We tested our method on 5 reco...
Conference Paper
Full-text available
When designing recommender-systems experiments, a key question that has been largely overlooked is the choice of datasets. In a brief survey of ACM RecSys papers, we found that authors typically justified their dataset choices by labelling them as public, benchmark, or ‘real-world’ without further explanation. We propose the Algorithm Performance S...
Conference Paper
Full-text available
This paper introduces e-fold cross-validation, an energy-efficient alternative to k-fold cross-validation. It dynamically adjusts the number of folds based on a stopping criterion. The criterion checks after each fold whether the standard deviation of the evaluated folds has consistently decreased or remained stable. Once met, the process stops ear...
Preprint
This paper introduces e-fold cross-validation, an energy-efficient alternative to k-fold cross-validation. It dynamically adjusts the number of folds based on a stopping criterion. The criterion checks after each fold whether the standard deviation of the evaluated folds has consistently decreased or remained stable. Once met, the process stops ear...
Preprint
As recommender systems become increasingly prevalent, the environmental impact and energy efficiency of training large-scale models have come under scrutiny. This paper investigates the potential for energy-efficient algorithm performance by optimizing dataset sizes through downsampling techniques in the context of Green Recommender Systems. We con...
Article
Full-text available
Recommender systems are a cornerstone of many industries, from e-commerce to streaming services. Their role in shaping user choices is significant, driving customer engagement and revenue. However, the computational demands of recommender systems have surged – by a factor of 42 over the past decade [16]. As recommender systems become ubiquitous on...
Article
Full-text available
I recall vividly when more than a decade ago – I was a PhD student – Konstan & Adomavicius warned that “the recommender systems research community [...] is facing a crisis where a significant number of research papers lack the rigor and evaluation to be properly judged and, therefore, have little to contribute to collective knowledge [14]”. Similar...
Preprint
Full-text available
Recommender systems are pivotal in numerous industries, from e-commerce to streaming platforms, significantly influencing user choices and engagement. However, the computational costs of these systems have escalated sharply, contributing to growing environmental concerns. This paper advocates for the development of Green Recommender Systems—recomme...
Preprint
The evaluation practices in the field of recommender systems have long faced scrutiny for their lack of rigor and reproducibility. Despite recent improvements, such as result-blind reviews and reproducibility tracks, the community still lacks comprehensive, evidence-based best-practice guidelines. This paper echoes the call for establishing such gu...
Preprint
Full-text available
Due to recent advancements in machine learning, recommender systems use increasingly more energy for training, evaluation, and deployment. However, the recommender systems community often does not report the energy consumption of their experiments. In today's research landscape, no tools exist to easily measure the energy consumption of recommender...
Preprint
Full-text available
The recommender systems algorithm selection problem for ranking prediction on implicit feedback datasets is under-explored. Traditional approaches in recommender systems algorithm selection focus predominantly on rating prediction on explicit feedback datasets, leaving a research gap for ranking prediction on implicit feedback datasets. Algorithm s...
Conference Paper
Full-text available
s recommender systems become increasingly prevalent, the environmental impact and energy efficiency of training large-scale models have come under scrutiny. This paper investigates the potential for energy-efficient algorithm performance by optimizing dataset sizes through downsampling techniques in the context of Green Recommender Systems. We cond...
Preprint
Full-text available
As global warming soars, evaluating the environmental impact of research is more critical now than ever before. However, we find that few to no recommender systems research papers document their impact on the environment. Consequently, in this paper, we conduct a comprehensive analysis of the environmental impact of recommender system research by r...
Chapter
Full-text available
To date, there have been a large number of papers written on challenges and best practices for evaluating recommender systems [6, 9, 13, 17, 18, 36, 38, 24, 36, 48]. Still, papers written and published today often fall short of embracing the practices suggested in prior works. Hence, we aim to suggest practical methods for the recommender systems c...
Preprint
Full-text available
We present the idea of "e-fold" cross-validation. The core idea is that e is chosen ’intelligently’ and individually for each experiment and dataset. This contrasts a static k chosen by gut feeling and past experiences on what k is ’typically’ good. Our goal for e-fold cross-validation is that e is as small as possible so as not to waste timeand en...
Article
Full-text available
This report documents the program and the outcomes of Dagstuhl Seminar 23031 "Frontiers of Information Access Experimentation for Research and Education", which brought together 38 participants from 12 countries. The seminar addressed technology-enhanced information access (information retrieval, recommender systems, natural language processing) an...
Article
Full-text available
Campbell and Stanley defined experiments as “that portion of research in which variables are manipulated and their effects upon other variables observed” (p. 1 in [1]).” Scientific experiments are used in confirmatory research to test a priori hypotheses as well as in exploratory research to gain new insights and help to generate hypotheses for fut...
Preprint
Full-text available
Automated machine learning (AutoML) systems commonly ensemble models post hoc to improve predictive performance, typically via greedy ensemble selection (GES). However, we believe that GES may not always be optimal, as it performs a simple deterministic greedy search. In this work, we introduce two novel population-based ensemble selection methods,...
Preprint
Full-text available
Automated Machine Learning (AutoML) frameworks regularly use ensembles. Developers need to compare different ensemble techniques to select appropriate techniques for an AutoML framework from the many potential techniques. So far, the comparison of ensemble techniques is often computationally expensive, because many base models must be trained and e...
Preprint
Full-text available
Many state-of-the-art automated machine learning (AutoML) systems use greedy ensemble selection (GES) by Caruana et al. (2004) to ensemble models found during model selection post hoc. Thereby, boosting predictive performance and likely following Auto-Sklearn 1's insight that alternatives, like stacking or gradient-free numerical optimization, over...
Poster
Full-text available
We previously (COSEAL'22) showed that CASH is challenging in rating prediction tasks in RecSys. Recently RecSys focused more on ranking prediction tasks whose evaluation is largely different from the former. In this poster, we present a selection of our research on the CASH problem in both rating and ranking prediction tasks in RecSys.
Poster
Full-text available
Our overarching goal is to bring the power of AutoML to recommender systems. Meta-learning solves various problems in AutoML. We present the idea of a cooperative meta-learning service for recommender systems (CaMeLS). With meta-learning we: 1. enable algorithm selection 2. provide a thorough benchmark (tabular-like) 3. standardize evaluation routi...
Chapter
Color modelling and extraction is an important topic in fashion. It can help build a wide range of applications, for example, recommender systems, color-based retrieval, fashion design, etc. We aim to develop and test models that can extract the dominant colors of clothing and accessory items. The approach we propose has three stages: (1) Mask-RCNN...
Preprint
Neural Architecture Search research has been limited to fixed datasets and as such does not provide the flexibility needed to deal with real-world, constantly evolving data. This is why we propose the basis of Online Neural Architecture Search (ONAS) to deal with complex, evolving, data distributions. We formalise ONAS as a minimisation problem upo...
Preprint
Full-text available
Recommendation algorithms perform differently if the users, recommendation contexts, applications, and user interfaces vary even slightly. It is similarly observed in other fields, such as combinatorial problem solving, that algorithms perform differently for each instance presented. In those fields, meta-learning is successfully used to predict an...
Preprint
Full-text available
Games such as go, chess and checkers have multiple equivalent game states, i.e. multiple board positions where symmetrical and opposite moves should be made. These equivalences are not exploited by current state of the art neural agents which instead must relearn similar information, thereby wasting computing time. Group equivariant CNNs in existin...
Preprint
Full-text available
The fashion industry is looking forward to use artificial intelligence technologies to enhance their processes, services, and applications. Although the amount of fashion data currently in use is increasing, there is a large gap in data exchange between the fashion industry and the related AI companies, not to mention the different structure used f...
Preprint
The advances in the field of Automated Machine Learning (AutoML) have greatly reduced human effort in selecting and optimizing machine learning algorithms. These advances, however, have not yet widely made it to Recommender-Systems libraries. We introduce Auto-CaseRec, a Python framework based on the CaseRec recommender-system library. Auto-CaseRec...
Article
Full-text available
Choice overload describes a situation in which a person has difficulty in making decisions due to too many options. We examine choice overload when displaying related-article recommendations in digital libraries, and examine the effectiveness of recommendation algorithms in this domain. We first analyzed existing digital libraries, and found that o...
Preprint
Full-text available
We introduce Auto-Surprise, an Automated Recommender System library. Auto-Surprise is an extension of the Surprise recommender system library and eases the algorithm selection and configuration process. Compared to out-of-the-box Surprise library, Auto-Surprise performs better when evaluated with MovieLens, Book Crossing and Jester Datasets. It may...
Preprint
Full-text available
Automated per-instance algorithm selection often outperforms single learners. Key to algorithm selection via meta-learning is often the (meta) features, which sometimes though do not provide enough information to train a meta-learner effectively. We propose a Siamese Neural Network architecture for automated algorithm selection that focuses more on...
Preprint
Full-text available
Citation parsing, particularly with deep neural networks, suffers from a lack of training data as available datasets typically contain only a few thousand training instances. Manually labelling citation strings is very time-consuming, hence synthetically created training data could be a solution. However, as of now, it is unknown if synthetically c...
Preprint
The relatedness of research articles, patents, court rulings, webpages, and other document types is often calculated with citation or hyperlink-based approaches like co-citation (proximity) analysis. The main limitation of citation-based approaches is that they cannot be used for documents that receive little or no citations. We propose Virtual Cit...
Preprint
Full-text available
The effectiveness of recommender system algorithms varies in different real-world scenarios. It is difficult to choose a best algorithm for a scenario due to the quantity of algorithms available, and because of their varying performances. Furthermore, it is not possible to choose one single algorithm that will work optimally for all recommendation...
Preprint
Full-text available
Fantasy Premier League (FPL) performance predictors tend to base their algorithms purely on historical statistical data. The main problems with this approach is that external factors such as injuries, managerial decisions and other tournament match statistics can never be factored into the final predictions. In this paper, we present a new method f...
Preprint
Full-text available
In this study, machine learning models were constructed to predict whether judgments made by the European Court of Human Rights (ECHR) would lead to a violation of an Article in the Convention on Human Rights. The problem is framed as a binary classification task where a judgment can lead to a "violation" or "non-violation" of a particular Article....
Preprint
Full-text available
Information about the contributions of individual authors to scientific publications is important for assessing authors' achievements. Some biomedical publications have a short section that describes authors' roles and contributions. It is usually written in natural language and hence author contributions cannot be trivially extracted in a machine-...
Conference Paper
Full-text available
Content-based approaches to research paper recommendation are important when user feedback is sparse or not available. The task of content-based matching is challenging, mainly due to the problem of determining the semantic similarity of texts. Nowadays, there exist many sentence embedding models that learn deep semantic representations by being tr...
Preprint
Memory-augmented neural networks (MANNs) have been shown to outperform other recurrent neural network architectures on a series of artificial sequence learning tasks, yet they have had limited application to real-world tasks. We evaluate direct application of Neural Turing Machines (NTM) and Differentiable Neural Computers (DNC) to machine translat...
Conference Paper
Full-text available
Recommendations-as-a-Service (RaaS) ease the process for small and medium-sized enterprises (SMEs) to offer product recommendations to their customers. Current RaaS, however, suffer from a one-size-fits-all concept, i.e. they apply the same recommendation algorithm for all SMEs. We introduce Darwin & Goliath, a RaaS that features multiple recommend...
Conference Paper
Full-text available
Documents from science, technology, engineering and mathematics (STEM) often contain a large number of mathematical formulae alongside text. Semantic search, recommender, and question answering systems require the occurring formula constants and variables (identifiers) to be disambiguated. We present a first implementation of a recommender system t...
Preprint
Many recommendation algorithms are available to digital library recommender system operators. The effectiveness of algorithms is largely unreported by way of online evaluation. We compare a standard term-based recommendation approach to two promising approaches for related-article recommendation in digital libraries: document embeddings, and keyphr...
Research Proposal
Full-text available
We propose the concept of "Federated Meta-Learning", a concept to share meta-data and models for various tasks (datasets) across multiple devices, to create a global meta-learning model for algorithm selection.
Chapter
Full-text available
The algorithm selection problem describes the challenge of identifying the best algorithm for a given problem space. In many domains, particularly artificial intelligence, the algorithm selection problem is well studied, and various approaches and tools exist to tackle it in practice. Especially through meta-learning impressive performance improvem...
Chapter
Full-text available
We introduce the first ‘living lab’ for scholarly recommender systems. This lab allows recommender-system researchers to conduct online evaluations of their novel algorithms for scholarly recommendations, i.e., recommendations for research papers, citations, conferences, research grants, etc. Recommendations are delivered through the living lab’s A...
Preprint
Bibliographic reference parsers extract machine-readable metadata such as author names, title, journal, and year from bibliographic reference strings. To extract the metadata, the parsers apply heuristics or machine learning. However, no reference parser, and no algorithm, consistently gives the best results in every scenario. For instance, one too...
Preprint
Recommender systems in academia are not widely available. This may be in part due to the difficulty and cost of developing and maintaining recommender systems. Many operators of academic products such as digital libraries and reference managers avoid this effort, although a recommender system could provide significant benefits to their users. In th...
Chapter
Full-text available
Neural Turing Machines (NTMs) are an instance of Memory Augmented Neural Networks, a new class of recurrent neural networks which decouple computation from memory by introducing an external memory unit. NTMs have demonstrated superior performance over Long Short-Term Memory Cells in several sequence learning tasks. A number of open source implement...
Preprint
Full-text available
Syllabuses for curriculum learning have been developed on an ad-hoc, per task basis and little is known about the relative performance of different syllabuses. We identify a number of syllabuses used in the literature. We compare the identified syllabuses based on their effect on the speed of learning and generalization ability of a LSTM network on...
Research Proposal
Full-text available
The algorithm selection problem describes the challenge of identifying the best algorithm for a given problem space. In many domains, particularly artificial intelligence, the algorithm selection problem is well studied, and various approaches and tools exist to tackle it in practice. Especially through meta-learning impressive performance improvem...
Preprint
Bibliographic reference parsers extract metadata (e.g. author names, title, year) from bibliographic reference strings. No reference parser consistently gives the best results in every scenario. For instance, one tool may be best in extracting titles, and another tool in extracting author names. In this paper, we address the problem of reference pa...
Conference Paper
Full-text available
Neural Turing Machines (NTMs) are an instance of Memory Augmented Neural Networks, a new class of recurrent neural networks which decouple computation from memory by introducing an external memory unit. NTMs have demonstrated superior performance over Long Short-Term Memory Cells in several sequence learning tasks. A number of open source implement...
Preprint
Full-text available
Neural Turing Machines (NTMs) are an instance of Memory Augmented Neural Networks, a new class of recurrent neural networks which decouple computation from memory by introducing an external memory unit. NTMs have demonstrated superior performance over Long Short-Term Memory Cells in several sequence learning tasks. A number of open source implement...
Preprint
We introduce the first living lab for scholarly recommender systems. This lab allows recommender-system researchers to conduct online evaluations of their novel algorithms for scholarly recommendations, i.e., research papers, citations, conferences, research grants etc. Recommendations are delivered through the living lab's API in platforms such as...
Preprint
Full-text available
The main contribution of this paper is to introduce and describe a new recommender-systems dataset (RARD II). It is based on data from a recommender-system in the digital library and reference management software domain. As such, it complements datasets from other domains such as books, movies, and music. The RARD II dataset encompasses 89m recomme...
Preprint
Full-text available
In this proposal we present the idea of a "macro recommender system", and "micro recommender system". Both systems can be considered as a recommender system for recommendation algorithms. A macro recommender system recommends the best performing recommendation algorithm to an organization that wants to build a recommender system. This way, an organ...
Conference Paper
Bibliographic reference parsing refers to extracting machine-readable metadata, such as the names of the authors, the title, or journal name, from bibliographic reference strings. Many approaches to this problem have been proposed so far, including regular expressions, knowledge bases and supervised machine learning. Many open source reference pars...
Conference Paper
Creating scientific publications is a complex process. It is composed of a number of different activities, such as designing the experiments, analyzing the data, and writing the manuscript. Information about the contributions of individual authors of a paper is important for assessing authors' scientific achievements. Some biomedical publications c...
Preprint
Full-text available
"Position bias" describes the tendency of users to interact with items on top of a list with higher probability than with items at a lower position in the list, regardless of the items' actual relevance. In the domain of recommender systems, particularly recommender systems in digital libraries, position bias has received little attention. We condu...
Article
Full-text available
Bibliographic reference parsing refers to extracting machine-readable metadata, such as the names of the authors, the title, or journal name, from bibliographic reference strings. Many approaches to this problem have been proposed so far, including regular expressions, knowledge bases and supervised machine learning. Many open source reference pars...
Article
Full-text available
Creating scientific publications is a complex process, typically composed of a number of different activities, such as designing the experiments, data preparation, programming software and writing and editing the manuscript. The information about the contributions of individual authors of a paper is important in the context of assessing authors' sc...
Research Proposal
Full-text available
The new architecture is a LSTM based recurrent neural networkcontroller network coupled with two external memory units, labeled input memory and output memory. The controller would have read heads into input memory and read and write heads into output memory. An embedded representation of each word ofthe input sentencewould be preloaded into input...
Article
Full-text available
In this position paper, we question the current practice of calculating evaluation metrics for recommender systems as single numbers (e.g. precision p=.28 or mean absolute error MAE = 1.21). We argue that single numbers express only average effectiveness over a usually rather long period (e.g. a year or even longer), which provides only a vague and...
Research Proposal
Full-text available
In this proposal we present the idea of a “macro recommender system”, and “micro recommender system” respectively. Both systems can be considered as a recommender system for recommendation algorithms. A macro recommender system recommends the potentially best performing recommendation algorithm to an organization that wants to build a recommender s...
Article
Full-text available
Recommender-system datasets are used for recommender-system evaluations, training machine-learning algorithms, and exploring user behavior. While there are many datasets for recommender systems in the domains of movies, books, and music, there are rather few datasets from research-paper recommender systems. In this paper, we introduce RARD, the Rel...