Jorge Carrillo-de-Albornoz

• Senior Researcher at National University of Distance Education

In this article we present UNED-ACCESS 2024, a bilingual dataset that consists of 1003 multiple-choice questions of university entrance level exams in Spanish and English. Questions are originally formulated in Span-ish and manually translated into English, and have not ever been publicly released, ensuring minimal contamination when evaluating Lar...

Over the past decade, the proliferation of hateful and sexist content targeting women on social media has become a concerning issue, adversely affecting women's lives and freedom of expression. Previous efforts to detect online sexism have utilized monolingual ensemble transformers combined with data augmentation techniques that incorporate related...

In this article we present UNED-ACCESS 2024, a bilingual dataset that consists of 1003 multiple-choice questions of university entrance level exams in Spanish and English. Questions are originally formulated in Spanish and translated manually into English, and have not ever been publicly released. A selection of current open-source and proprietary...

With the rise of social networks, there has been a marked increase in offensive content targeting women, ranging from overt acts of hatred to subtler, often overlooked forms of sexism. The EXIST (sEXism Identification in Social neTworks) competition, initiated in 2021, aimed to advance research in automatically identifying these forms of online sex...

The paper describes the EXIST 2024 lab on Sexism identification in social networks, that is expected to take place at the CLEF 2024 conference and represents the fourth edition of the EXIST challenge. The lab comprises five tasks in two languages, English and Spanish, with the initial three tasks building upon those from EXIST 2023 (sexism identifi...

The increasing interest from research agencies, governments, and universities in understanding research funding and prioritising research efforts has highlighted the need for reliable and efficient methods for exploring research portfolios. In biomedical research, this involves exploring research across what is normally considered fundamental and a...

The detection and analysis of media bias and propaganda have become essential in the current information age. This paper presents our participation in the DIPROMATS task, which focusses on identifying and characterising propaganda techniques in text. We propose a hierarchical model that leverages both the provided DIPROMATS dataset and the SemEval'...

In recent years, the rapid increase in the dissemination of offensive and discriminatory material aimed at women through social media platforms has emerged as a significant concern. This trend has had adverse effects on women’s well-being and their ability to freely express themselves. The EXIST campaign has been promoting research in online sexism...

Detecting media bias is a challenging task due to the complexity and ambiguity of language. Current approaches are limited in their ability to generalise across regions and styles of journalism. This paper proposes a new approach that focusses on detecting rhetorical linguistic techniques rather than just analysing words or contextual representatio...

How similar is the detection of media bias to the detection of persuasive techniques? We have explored how transferring knowledge from one task to the other may help to improve the performance. This paper presents the systems developed for participating in the SemEval-2023 Task 3: Detecting the Genre, the Framing, and the Persuasion Techniques in O...

The paper describes the lab on Sexism identification in social networks (EXIST 2023) that will be hosted as a lab at the CLEF 2023 conference. The lab consists of three tasks, two of which are continuation of EXIST 2022 (sexism detection and sexism categorization) and a third and novel one on source intention identification. For this edition new te...

The paper describes the organization, goals, and results of the sEXism Identification in Social neTworks (EXIST)2022 challenge, a shared task proposed for the second year at IberLEF. EXIST 2022 consists of two challenges: sexism identification and sexism categorization of tweets and gabs, both in Spanish and English. We have received a total of 45...

Detecting and tackling sexist messages in social media is important for encouraging better behaviours in our society as well as to contribute to effective equality between men and women. In this paper we present our participation in the sEXism Identification in Social neTworks (EXIST) task at IberLEF'2021 [1]. Our approach to solve the task is base...

Online reputation management (ORM) comprises the collection of techniques that help monitoring and improving the public image of an entity (companies, products, institutions) on the Internet. The ORM experts try to minimize the negative impact of the information about an entity while maximizing the positive material for being more trustworthy to th...

In Ordinal Classification tasks, items have to be assigned to classes that have a relative ordering, such as positive, neutral, negative in sentiment analysis. Remarkably, the most popular evaluation metrics for ordinal classification tasks either ignore relevant information (for instance, precision/recall on each of the classes ignores their relat...

Producing online reputation summaries for an entity (company, brand, etc.) is a focused summarization task with a distinctive feature: issues that may affect the reputation of the entity take priority in the summary. In this paper we (i) present a new test collection of manually created (abstractive and extractive) reputation reports which summariz...

During the last decade, hateful and sexist content towards women is being increasingly spread on social networks. The exposure to sexist speech has serious consequences to women’s life and limits their freedom of speech. Previous studies have focused on identifying hatred or violence towards women. However, sexism is expressed in very different for...

Over a period of 3 years, RepLab was a CLEF initiative where computer scientists and online reputation experts worked together to identify and formalize the computational challenges in the area of online reputation monitoring. Two main results emerged from RepLab: a community of researchers engaged in the problem, and an extensive Twitter test coll...

Introduction Surveys indicate that patients, particularly those suffering from chronic conditions, strongly benefit from the information found in social networks and online forums. One challenge in accessing online health information is to differentiate between factual and more subjective information. In this work, we evaluate the feasibility of ex...

Introduction Exploiting information in health-related social media services is of great interest for patients, researchers and medical companies. The challenge is, however, to provide easy, quick and relevant access to the vast amount of information that is available. One step towards facilitating information access to online health data is opinion...

Many evaluation metrics have been defined to evaluate the effectiveness ad-hoc retrieval and search result diversification systems. However, it is often unclear which evaluation metric should be used to analyze the performance of retrieval systems given a specific task. Axiomatic analysis is an informative mechanism to understand the fundamentals o...

Many evaluation metrics have been defined to evaluate the effectiveness ad-hoc retrieval and search result diversification systems. However, it is often unclear which evaluation metric should be used to analyze the performance of retrieval systems given a specific task. Axiomatic analysis is an informative mechanism to understand the fundamentals o...

The EvALL online evaluation service aims to provide a unified evaluation framework for Information Access systems that makes results completely comparable and publicly available for the whole research community. For researchers working on a given test collection, the framework allows to: (i) evaluate results in a way compliant with measurement theo...

In this paper, the Evall framework for the automatic evaluation of information systems task is presented. With just one click and providing the system outputs of the algorithms, Evall allows researchers to automatically generate a Latex report including the results of their algorithms, statistical significance tests, measures descriptions, and refe...

Producing online reputation reports for an entity (company, brand, etc.) is a focused summarization task with a distinctive feature: issues that may affect the reputation of the entity take priority in the summary. In this paper we (i) propose a novel methodology to evaluate summaries in the context of online reputation which profits from an analog...

Background: Research in biomedical text categorization has mostly used the bag-of-words representation. Other more sophisticated representations of text based on syntactic, semantic and argumentative properties have been less studied. In this paper, we evaluate the impact of different text representations of biomedical texts as features for reprod...

This paper describes the organisation and results of RepLab 2014, the third competitive evaluation campaign for Online Reputation Management systems. This year the focus lied on two new tasks: reputation dimensions classification and author profiling, which complement the aspects of reputation analysis studied in the previous campaigns. The partici...

We present a semi-automatic tool that assists experts in their daily work of monitoring the reputation of entities—companies, organizations or public figures—in Twitter. The tool automatically annotates tweets for relevance (Is the tweet about the entity?), reputational polarity (Does the tweet convey positive or negative implications for the reput...

This paper summarizes the goals, organization, and results of the second RepLab competitive evaluation campaign for Online Rep-utation Management Systems (RepLab 2013). RepLab focused on the process of monitoring the reputation of companies and individuals, and asked participant systems to annotate different types of information on tweets containin...

Negation, intensifiers, and modality are common linguistic constructions that may modify the emotional meaning of the text and therefore need to be taken into consideration in sentiment analysis. Negation is usually considered as a polarity shifter, whereas intensifiers are regarded as amplifiers or diminishers of the strength of such polarity. Mod...

Background The position of a sentence in a document has been traditionally considered an indicator of the relevance of the sentence, and therefore it is frequently used by automatic summarization systems as an attribute for sentence selection. Sentences close to the beginning of the document are supposed to deal with the main topic and thus are sel...

This paper describes the UNED's Online Reputation Moni-toring Team participation at RepLab 2013 [3]. Several approaches were tested: First, an instance-based learning approach that uses Heterogene-ity Based Ranking to combine seven different similarity measures was applied for all the subtasks. The filtering subtask was also tackled by au-tomatical...

In this paper we propound the use of a number of entropy-based metrics and a visualization tool for the intrinsic evaluation of Sentiment and Reputation Analysis tasks. We provide a theoretical justification for their use and discuss how they complement other accuracy-based metrics. We apply the proposed techniques to the analysis of TASS-SEPLN and...

Sentiment Analysis is a novel and broad area of Natural Language Processing (NLP) aiming to understand people's sentiments and opinions about a given topic. In particular, this chapter focuses on the application of Sentiment Analysis to automatically evaluate online products and services reviews. Undoubtedly, the information in customer reviews is...

This paper presents one of the two contributions from the Universidad Complutense de Madrid to the *SEM Shared Task 2012 on Resolving the Scope and Focus of Negation. We describe a rule-based system for detecting the presence of negations and delimitating their scope. It was initially intended for processing negation in opinionated texts, and has b...

UCM-2 infers the words that are affected by negations by browsing dependency syntactic structures. It first makes use of an algorithm that detects negation cues, like no, not or nothing, and the words affected by them by traversing Minipar dependency structures. Second, the scope of these negation cues is computed by using a post-processing rule-ba...

This paper presents SentiSense, a concept-based affective lexicon. It is intended to be used in sentiment analysis-related tasks, specially in polarity and intensity classification and emotion identification. SentiSense attaches emotional meanings to concepts from the WordNet lexical database, instead of terms, thus allowing to address the word amb...

Online Reputation Management is a novel and active area in Computational Linguistics. Closely related to opinion mining and sentiment analysis, it incorporates new features to traditional tasks like polarity detection. In this paper, we study the feasibility of applying complex sentiment analysis methods to classifying polarity for reputation. We a...

Sentiment Analysis is a novel and broad area of Natural Language Processing (NLP) aiming to understand people’s sentiments and opinions about a given topic. In particular, this chapter focuses on the application of Sentiment Analysis to automatically evaluate online products and services reviews. Undoubtedly, the information in customer reviews is...

The information in customer reviews is of great interest to both companies and consumers. This information is usually presented as non-structured free-text so that automatically extracting and rating user opinions about a product is a challenging task. Moreover, this opinion highly depends on the product features on which the user judgments and imp...

Los autores declaran no tener ningún tipo de interés económico o comercial RESUMEn En un entorno como el de la medicina, caracterizado por la sobrecarga de trabajo y la escasez de tiempo, los sistemas in-teligentes de acceso a la información pueden y deben utilizarse para facilitar la labor de investigadores y profesionales. Sin embargo, sorprende...

In this paper, the authors present a new approach to sentence level sentiment analysis. The aim is to determine whether a sentence expresses a positive, negative or neutral sentiment, as well as its intensity. The method performs WSD over the words in the sentence in order to work with concepts rather than terms, and makes use of the knowledge in a...

There exist several types of generative systems which produce some artistic output not necessarily considered to be creative. However, the processes they follow can, in principle, be combined between them, possibly obtaining a creative product. Although nowadays giving a general description of how such systems must be implemented is far from being...

We present an approach for the personalized access and the students' coauthoring in repositories of learning objects (LOs). The personalized access combines content-based and collaborative filtering mechanisms. Students' coauthoring includes the incorporation of assessments of the existing LOs and new LOs which are peer reviewed.

The expressiveness and possible impact on the viewer are some of the most important characteristics in order to film a cinema scene. However, it is quite rare to see this kind of features in the different approaches of automatic Camera Management Systems in virtual environments. The diverse studies that have used in some way cinematographic techniq...