Miguel-Angel Sicilia’s research while affiliated with University of Alcalá and other places

Intent classification and sentiment analysis stand as pivotal tasks in natural language understanding (NLU), with applications ranging from virtual assistants to customer service. The advent of transformer-based models has significantly enhanced the performance of various NLP tasks, with encoder-only architectures gaining prominence for their effectiveness. More recently, there has been a surge in the development of larger and more powerful decoder-only models, traditionally employed for text generation tasks. This paper aims to answer the question of whether the colossal scale of newer decoder-only language models is essential for real-world applications. The investigation involves a performance comparison between these decoder-only models and the well-established encoder-only models specifically in the domains of intent classification and sentiment analysis. The results of our study indicate that, for tasks involving natural language understanding, encoder-only models generally outperform decoder-only models, all while demanding a fraction of the computational resources. This sheds light on the practicality and efficiency of encoder-only architectures in comparison to their decoder-only counterparts in real-world applications, providing valuable insights for the advancement of natural language processing technologies.

The objective of this paper is to describe Cryptocurrency Linguo (CryptoLin), a novel corpus containing 2683 cryptocurrency-related news articles covering more than a three-year period. CryptoLin was human-annotated with discrete values representing negative, neutral, and positive news respectively. Eighty-three people participated in the annotation process; each news title was randomly assigned and blindly annotated by three human annotators, one in each different cohort, followed by a consensus mechanism using simple voting. The selection of the annotators was intentionally made using three cohorts with students from a very diverse set of nationalities and educational backgrounds to minimize bias as much as possible. In case one of the annotators was in total disagreement with the other two (e.g., one negative vs two positive or one positive vs two negative), we considered this minority report and defaulted the labeling to neutral. Fleiss’s Kappa, Krippendorff’s Alpha, and Gwet’s AC1 inter-rater reliability coefficients demonstrate CryptoLin’s acceptable quality of inter-annotator agreement. The dataset also includes a text span with the three manual label annotations for further auditing of the annotation mechanism. To further assess the quality of the labeling and the usefulness of CryptoLin dataset, it incorporates four pretrained Sentiment Analysis models: Vader, Textblob, Flair, and FinBERT. Vader and FinBERT demonstrate reasonable performance in the CryptoLin dataset, indicating that the data was not annotated randomly and is therefore useful for further research1. FinBERT (negative) presents the best performance, indicating an advantage of being trained with financial news. Both the CryptoLin dataset and the Jupyter Notebook with the analysis, for reproducibility, are available at the project’s Github. Overall, CryptoLin aims to complement the current knowledge by providing a novel and publicly available Gadi and Ángel Sicilia (Cryptolin dataset and python jupyter notebooks reproducibility codes, 2022) cryptocurrency sentiment corpus and fostering research on the topic of cryptocurrency sentiment analysis and potential applications in behavioral science. This can be useful for businesses and policymakers who want to understand how cryptocurrencies are being used and how they might be regulated. Finally, the rules for selecting and assigning annotators make CryptoLin unique and interesting for new research in annotator selection, assignment, and biases.

Intent classification and sentiment analysis stand as pivotal tasks in natural language processing, with applications ranging from virtual assistants to customer service. The advent of transformer-based models has significantly enhanced the performance of various NLP tasks, with encoder-only architectures gaining prominence for their effectiveness. More recently, there has been a surge in the development of larger and more powerful decoder-only models, traditionally employed for text generation tasks. This paper aims to answer the question of whether the colossal scale of newer decoder-only language models is essential for real-world applications by comparing their performance to the well established encoder-only models, in the domains of intent classification and sentiment analysis. Our results shows that for such natural language understanding tasks, encoder-only models in general provide better performance than decoder-only models, at a fraction of the computational demands.

Stock market indices are pivotal tools for establishing market benchmarks, enabling investors to navigate risk and volatility while capitalizing on the stock market’s prospects through index funds. For participants in decentralized finance (DeFi), the formulation of a token index emerges as a vital resource. Nevertheless, this endeavor is complex, encompassing challenges such as transaction fees and the variable availability of tokens, attributed to their brief history or limited liquidity. This research introduces an index tailored for the Ethereum ecosystem, the leading smart contract platform, and conducts a comparative analysis of capitalization-weighted (CW) and equal-weighted (EW) index performances. The article delineates exhaustive criteria for token eligibility, intending to serve as a comprehensive guide for fellow researchers. The results indicate a consistent superior performance of CW indices over EW indices in terms of return and risk metrics, with a 30-constituent CW index outshining its counterparts with varied constituent numbers. The recommended CW30 index demonstrates substantial advantages in comparison to established benchmarks, including prominent indices like DeFi Pulse Index (DPI) and CRypto IndeX (CRIX). Additionally, the article explores the practicality of implementing the CW30 in Layer 2 networks of the Ethereum Ecosystem, advocating for the Arbitrum infrastructure as the optimal choice for the decentralized crypto index protocol herein referred to as the Ethereum Ecosystem Index (EEI). The study’s insights aspire to enrich the DeFi ecosystem, offering a nuanced understanding of network selection and a strategic framework for implementation. This research significantly enhances the existing literature on index construction and performance within the Ethereum ecosystem. To our knowledge, it represents a pioneering comprehensive analysis of an index that accurately mirrors the Ethereum market, advancing our comprehension of its intricacies and wider ramifications. Moreover, this study stands as one of the initial thorough examinations of index construction methodologies within the nascent asset class of crypto. The insights gleaned provide a pragmatic approach to index construction and introduce an index poised to serve as a benchmark for index products. In illuminating the unique facets of the Ethereum ecosystem, this research makes a substantial contribution to the current discourse on crypto, offering valuable perspectives for investors, market stakeholders, and the ongoing exploration of digital assets.

Conversational Agents (CA) are increasingly being deployed by organizations to provide round-the-clock support and to increase customer satisfaction. All CA have one thing in common despite the differences in their design: they need to be trained with users’ intents and corresponding training sentences. Access to proper data with acceptable coverage of intents and training sentences is a big challenge in CA deployment. Even with the access to the past conversations, the process of discovering intents and training sentences manually is not time and cost-effective. Here, an end to end automated framework that can discover intents and their training sentences in conversation logs to generate labeled data sets for training intent models is presented. The framework proposes different feature engineering techniques and leverages dimensionality reduction methods to assemble the features, then applies a density-based clustering algorithm iteratively to mine even the least common intents. Finally, the clustering results are automatically labeled by the final algorithm.

Well labeled natural language corpus data is essential for most natural language processing techniques, especially in specialized fields. However, cohort biases remain a significant challenge in machine learning. The narrow origin of data sampling or human annotators in cohorts is a prevalent issue for machine learning researchers due to its potential to induce bias in the final product. During the development of the CryptoLin corpus for another research project, the authors became concerned about the potential influence of cohort bias on the selection of annotators. Therefore, this paper addresses the question of whether cohort diversity improves the labeling result through the implementation of a repeated annotator process, involving two annotator cohorts and a statistically robust comparison methodology. The utilization of statistical tests, such as the Chi-Square Independence test for absolute frequency tables, and the construction of confidence intervals for Kappa point estimates, facilitates a rigorous analysis of the differences between Kappa estimates. Furthermore, the application of a two-proportion z-test to compare the accuracy scores of UTAD and IE annotators for various pre-trained models, including Vader Sentiment Analysis, TextBlob Sentiment Analysis, Flair NLP library, and FinBERT Financial Sentiment Analysis with BERT, contributes to the advancement of knowledge in this field. The paper utilizes Cryptocurrency Linguo (CryptoLin), a corpus containing 2683 cryptocurrency-related news articles spanning more than three years,and compares two different selection criteria for the annotators. CryptoLin was annotated twice with discrete values representing negative, neutral, and positive news respectively. The first annotation was done by twenty-seven annotators from the same cohort. Each news title was randomly assigned and blindly annotated by three human annotators. The second annotation was carried out by eighty-three annotators from three cohorts. Each news title was randomly assigned and blindly annotated by three human annotators, one in each different cohort. In both annotations, a consensus mechanism using simple voting was applied. The first annotation used the same cohort with students from the same nationality and background. The second used three cohorts with students from a very diverse set of nationalities and educational backgrounds. The results demonstrate that manual labeling done by both groups was acceptable according to inter-rater reliability coefficients Fleiss’s Kappa, Krippendorff’s Alpha, and Gwet’s AC1. Preliminary analysis utilizing Vader, Textblob, Flair, and FinBERT confirmed the utility of the data set labeling for further refinement of sentiment analysis algorithms. Our results also highlight that the more diverse annotator pool performed better in all measured aspects.

Cryptocurrency markets have experienced large growth in recent years, with an increase in the number and diversity of traded assets. Previous work has addressed the economic properties of Bitcoin with regards to its hedging or diversification properties. However, the surge of many alternatives, applications, and decentralized finance services on a variety of blockchain networks requires a re-examination of those properties, including indexes from outside the big economies and the inclusion of a variety of cryptocurrencies. In this paper, we report the results of studying the most representative cryptocurrency of each consensus mechanism by trading volume, forming a list of twenty-four cryptocurrencies from the 1st of January 2018 to the 30th of September 2022. Using the Baur and McDermott model, we examine hedge, safe haven, and diversifier properties of all assets for all G7 country’s major indexes as well as all BRICS major indexes breaking it down by two attributes: kind of blockchain technology and pre/during COVID health crisis. Results show that both attributes play an important role in the hedge, safe haven, and diversifier properties associated with the asset. Concretely: stablecoins appear to be the only ones to maintain hedge property in most analyzed markets pre- and during-COVID; Bitcoin investment properties shifted after the COVID crisis started; China and Russia stopped being correlated with the cryptocurrency after the COVID crisis hit.

Event studies in general rely on having a high-quality curated database of events. In this paper we introduce CryptoGDelt2022, a news event dataset extracted from the Global Database of Events, Language and Tone (GDELT) containing more than 243 thousands cryptocurrency related news events between the 31st of March 2021 and 30th of April 2022. The dataset is enriched with supervised machine learning scores for Relevance, Sentiment and Strength. Supervised Relevance Score measures how related to Cryptocurrency the topic is using news web scrapped from Yahoo in general and from the Cryptocurrency part of the site, after a comparison of approaches; Latent Dirichlet Allocation (LDA), BERT and Naive Bayes, Naive Bayes was chosen and the hyper-parameter tuned model reached accuracy: 97.84 % in the train set and 91.70% in the test set. Supervised Sentiment Score measures the negative, neutral or positive tone of the news, after hyper-parameter tuning, the retrained FinBERT model achieved accuracy of 92.63% in the train set and 86.11% in the test set. Supervised Strength Score measures how strong the news by using the abnormal return using Fama French 3-factor model as target output variable, after hyper-parameter tuning, the trained Naive Bayes model reached accuracy of 63.34%. The work concludes that GDELT is more reliable source of event when compared to news selected from cryptocurrency specialized websites as it presents a more balanced positive and negative number of news. All data sets and Python Jupyter Notebooks are available in the project's GitHub.