Figure 1 - available via license: Creative Commons Attribution 4.0 International
Content may be subject to copyright.
Source publication
Garlic-related misinformation is prevalent whenever a virus outbreak occurs. With the outbreak of COVID-19, garlic-related misinformation is spreading through social media, including Twitter. Bidirectional Encoder Representations from Transformers (BERT) can be used to classify misinformation from a vast number of tweets. This study aimed to apply...
Contexts in source publication
Context 1
... study was designed to fine-tune BERT models with a "COVID-19 rumor dataset" or a garlic-specific dataset and classify garlic-related COVID-19 misinformation from tweets. Figure 1 shows the study flow diagram. The study was exempted from Institutional Review Board review (202004-HR-012-01). ...
Citations
... The research [29] investigates the classification of garlic-related COVID-19 misinformation on Twitter using BERT-based models. The study develops a garlic-specific dataset of 5929 labelled tweets and compares the performance of fine-tuned BERT models against a COVID-19 rumor dataset. ...
The spread of health-related misinformation has become a significant global challenge, particularly during the COVID-19 pandemic. This study introduces a comprehensive framework for detecting and analyzing misinformation using advanced natural language processing techniques. The proposed classification model combines BERT embeddings with Bi-LSTM architecture and attention mechanisms, achieving high performance, including 99.47% accuracy and an F1-score of 0.9947. In addition to classification, topic modeling is employed to identify thematic clusters, providing valuable insights into misinformation narratives. The findings demonstrate the effectiveness and reliability of the proposed methodology in detecting misinformation while offering tools for understanding its underlying themes. The adaptable and scalable approach makes it applicable to various domains and datasets. This research improves public health communication and combating misinformation in digital environments.
... Transformer models exhibit a high degree of adaptability to transfer learning, a process where a pre-trained model on one task or dataset can be fine-tuned on a different, often smaller, dataset for a specific task [27]. In a previous study, ChemBERTa demonstrated its ability to identify toxic chemicals from the ClinTox dataset and p53 stress-response pathway activators from the Tox21 dataset, achieving AUC-ROC values of 0.733 and 0.728, respectively [14]. ...
Natural language processing (NLP) technology has recently used to predict substance properties based on their Simplified Molecular-Input Line-Entry System (SMILES). We aimed to develop a model predicting human skin sensitizers by integrating text features derived from SMILES with in vitro test outcomes. The dataset on SMILES, physicochemical properties, in vitro tests (DPRA, KeratinoSensTM, h-CLAT, and SENS-IS assays), and human potency categories for 122 substances sourced from the Cosmetics Europe database. The ChemBERTa model was employed to analyze the SMILES of substances. The last hidden layer embedding of ChemBERTa was tested with other features. Given the modest dataset size, we trained five XGBoost models using subsets of the training data, and subsequently employed bagging to create the final model. Notably, the features computed from SMILES played a pivotal role in the model for distinguishing sensitizers and non-sensitizers. The final model demonstrated a classification accuracy of 80% and an AUC-ROC of 0.82, effectively discriminating sensitizers from non-sensitizers. Furthermore, the model exhibited an accuracy of 82% and an AUC-ROC of 0.82 in classifying strong and weak sensitizers. In summary, we demonstrated that the integration of NLP of SMILES with in vitro test results can enhance the prediction of health hazard associated with chemicals.
... The fine-tuned models were able to perform better than the TOOD and FCOS that were trained on the visdrone and xview datasets. This may be due to the usage of pre-trained weights and further fine-tuning of the network which causes an increase in performance as also observed in a previous study using fine-tuned models [50,51]. ...
Assistive technology (AT) is invaluable to people with special educational needs and disabilities, enabling them to interact with computers more efficiently. It is able to improve the interactivity between humans and computers for learning purposes. However, there is a lack of accessibility and interactivity with touchscreen devices and visual content that AT users encounter. These issues are critical in providing much-needed assistance and enhancing the daily lives of disabled individuals. Therefore, we propose an AT framework based on neural networks using embedded systems for improving the accessibility and interactivity of AT users. The proposed framework utilizes the Jetson Nano and is mainly used with a speech-to-intent neural network to process speech and move cursors. When improved with cursor object detection, the framework could obtain the location of the cursors in external displays and move the cursors of other devices. Since cursor datasets are very limited and not many detection models are up for the task, we investigated the use of Slicing Aided Hyper Inference (SAHI) pipeline along with two fine-tuned models, Fully Convoluted One-Stage (FCOS) and Task-aligned One-stage Object Detection (TOOD), to identify the minimum data required for these models to work optimally. With less than 120 annotated images and a data multiplier of 5 and 30, both models were able to achieve ~ 52 and ~ 60mAP, respectively. These results were comparable to performance on other small object detection datasets. In addition, we also present a working proof-of-concept for our proposed embedded assistive technology framework.
... The model outperformed other text representation models such as GLoVe. BERTWeet was compared with BERT cased and uncased pretrained models for detecting fake tweets about COVID-19 in [41]. BERTweet showed the best performance among different BERT models. ...
... With respect to detecting from social media, BERT has been used along with its variations, like RoBERTa, in [14,38,41,48]. BERTweet is a fine-tuned BERT-based model trained on English tweets. ...
The spread of fake news on social media continues to be one of the main challenges facing internet users, prohibiting them from discerning authentic from fabricated pieces of information. Hence, identifying the veracity of the content in social posts becomes an important challenge, especially with more people continuing to use social media as their main channel for news consumption. Although a number of machine learning models were proposed in the literature to tackle this challenge, the majority rely on the textual content of the post to identify its veracity, which poses a limitation to the performance of such models, especially on platforms where the content of the users’ post is limited (e.g., Twitter, where each post is limited to 140 characters). In this paper, we propose a deep-learning approach for tackling the fake news detection problem that incorporates the content of both the social post and the associated news article as well as the context of the social post, coined TChecker. Throughout the experiments, we use the benchmark dataset FakeNewsNet to illustrate that our proposed model (TChecker) is able to achieve higher performance across all metrics against a number of baseline models that utilize the social content only as well as models combining both social and news content.
... In recent years, BERT has also been used in the field of rumor detection on social media [22][23] . Some researchers implemented BERT to detect rumors on during the COVID-19 pandemic [24][25] . These studies demonstrate the effectiveness of BERT in detecting rumors on social media platforms. ...
The abundance of information on social media has increased the necessity of accurate real-time rumour detection. Manual techniques of identifying and verifying fake news generated by AI tools are impracticable and time-consuming given the enormous volume of information generated every day. This has sparked an increase in interest in creating automated systems to find fake news on the Internet. The studies in this research demonstrate that the BERT and RobertA models with fine-tuning had the best success in detecting AI generated news. With a score of 98%, tweaked RobertA in particular showed excellent precision. In conclusion, this study has shown that neural networks can be used to identify bogus news AI generation news created by ChatGPT. The RobertA and BERT models' excellent performance indicates that these models can play a critical role in the fight against misinformation.
... Recent works have shown that fine-tuning BERT has been very successful in many NLP tasks, including sentiment analysis [29], text classification [30], named-entity recognition (NER) [31], intent recognition [32], etc. For instance, Agrawal et al. in Ref. [31] attempted to solve the nested-NER problem using transfer learning through fine-tuning BERT. ...
BERT, the most popular deep learning language model, has yielded breakthrough results in various NLP tasks. However, the semantic representation space learned by BERT has the property of anisotropy. Therefore, BERT needs to be fine-tuned for certain downstream tasks such as Semantic Textual Similarity (STS). To overcome this problem and improve the sentence representation space, some contrastive learning methods have been proposed for fine-tuning BERT. However, existing contrastive learning models do not consider the importance of input triplets in terms of easy and hard negatives during training. In this paper, we propose the SelfCCL: Curriculum Contrastive Learning model by Transferring Self-taught Knowledge for Fine-Tuning BERT, which mimics the two ways that humans learn about the world around them, namely contrastive learning and curriculum learning. The former learns by contrasting similar and dissimilar samples. The latter is inspired by the way humans learn from the simplest concepts to the most complex concepts. Our model also performs this training by transferring self-taught knowledge. That is, the model figures out which triplets are easy or difficult based on previously learned knowledge, and then learns based on those triplets in the order of curriculum using a contrastive objective. We apply our proposed model to the BERT and Sentence BERT(SBERT) frameworks. The evaluation results of SelfCCL on the standard STS and SentEval transfer learning tasks show that using curriculum learning together with contrastive learning increases average performance to some extent.
... Some recent studies have conducted more in-depth analyses of misinformation associated with specific products or substances that were rumored to be effective against COVID-19. For example, Kim et al [19] fine-tuned transformer-based models to automatically classify misinformation related to garlic. Quinn et al [20] analyzed misinformation related to vitamin D and COVID-19 on YouTube. ...
Background
Social media has served as a lucrative platform for spreading misinformation and for promoting fraudulent products for the treatment, testing, and prevention of COVID-19. This has resulted in the issuance of many warning letters by the US Food and Drug Administration (FDA). While social media continues to serve as the primary platform for the promotion of such fraudulent products, it also presents the opportunity to identify these products early by using effective social media mining methods.
Objective
Our objectives were to (1) create a data set of fraudulent COVID-19 products that can be used for future research and (2) propose a method using data from Twitter for automatically detecting heavily promoted COVID-19 products early.
Methods
We created a data set from FDA-issued warnings during the early months of the COVID-19 pandemic. We used natural language processing and time-series anomaly detection methods for automatically detecting fraudulent COVID-19 products early from Twitter. Our approach is based on the intuition that increases in the popularity of fraudulent products lead to corresponding anomalous increases in the volume of chatter regarding them. We compared the anomaly signal generation date for each product with the corresponding FDA letter issuance date. We also performed a brief manual analysis of chatter associated with 2 products to characterize their contents.
Results
FDA warning issue dates ranged from March 6, 2020, to June 22, 2021, and 44 key phrases representing fraudulent products were included. From 577,872,350 posts made between February 19 and December 31, 2020, which are all publicly available, our unsupervised approach detected 34 out of 44 (77.3%) signals about fraudulent products earlier than the FDA letter issuance dates, and an additional 6 (13.6%) within a week following the corresponding FDA letters. Content analysis revealed misinformation, information, political, and conspiracy theories to be prominent topics.
Conclusions
Our proposed method is simple, effective, easy to deploy, and does not require high-performance computing machinery unlike deep neural network–based methods. The method can be easily extended to other types of signal detection from social media data. The data set may be used for future research and the development of more advanced methods.
... BERT pre-trains a deep bidirectional language representation model (Devlin et al., 2018), and then the pretrained BERT model can be fine-tuned with an additional output layer to create state-of-the-art models for a wide range of tasks without major architectural modifications (Zhang et al., 2021). There are currently different versions of pre-trained models of BERT, in our research, we used BERT_base which is pre-trained by English Wikipedia and Book-Corpus (Kim et al., 2022). ProtBERT, as one of the ProtTrans models (Elnaggar et al., 2020) which is pre-trained with 217 million protein sequences and is providing state-of-the-art pre-trained models for proteins, was used as our base pre-trained model. ...
As a potential and effective substitute for the drugs of antihypertension, the food-derived antihypertensive peptides have arisen great interest in scholars recently. However, the traditional screening methods for antihypertensive peptides are at considerable expense and laborious, which blocks the exploration of available antihypertensive peptides. In our study, we reported the use of a protein-specific deep learning model called ProtBERT to screen for antihypertensive peptides. Compared to other deep learning models, ProrBERT reached the highest the area under the receiver operating characteristic curve (AUC) value of 0.9785. In addition, we used ProtBERT to screen candidate peptides in soybean protein isolate (SPI), followed by molecular docking and in vitro validation, and eventually found that peptides LVPFGW (IC50 = 20.63 μM), VSFPVL (2.57 μM), and VLPF (5.78 μM) demonstrated the good antihypertensive activity. Deep learning such as ProtBERT will be a useful tool for the rapid screening and identification of antihypertensive peptides.
... Some recent studies have conducted more in-depth analyses of misinformation associated with specific products or substances that were rumored to be effective against COVID-19. For example, Kim et al [19] fine-tuned transformer-based models to automatically classify misinformation related to garlic. Quinn et al [20] analyzed misinformation related to vitamin D and COVID-19 on YouTube. ...
Background
Social media have served as lucrative platforms for spreading misinformation and for promoting fraudulent products for the treatment, testing, and prevention of COVID-19. This has resulted in the issuance of many warning letters by the United States Food and Drug Administration (FDA). While social media continue to serve as the primary platform for the promotion of such fraudulent products, they also present the opportunity to identify these products early by employing effective social media mining methods. In this study, we employ natural language processing and time series anomaly detection methods for automatically detecting fraudulent COVID-19 products early from Twitter. Our approach is based on the intuition that increases in the popularity of fraudulent products lead to corresponding anomalous increases in the volume of chatter regarding them.
Results
We utilized an anomaly detection method on streaming COVID-19-related Twitter data to detect potentially anomalous increases in mentions of fraudulent products. We compared the anomaly signal generation date for each product with the corresponding FDA letter issuance date. Issue dates ranged from March 6, 2020 to June 22, 2021, and 44 key phrases representing fraudulent products were included. From 577,872,350 posts made between February 19, 2020 to December 31, 2020, our unsupervised approach detected 34/44 (77.3%) signals about fraudulent products earlier than the FDA letter issuance dates, and an additional 6/44 (13.6%) within a week following the corresponding FDA letters.
Conclusions
Our proposed method is simple, effective, and easy to deploy, and does not require high-performance computing machinery, unlike deep neural network-based methods. The method can be easily extended to other types of signal detection from social media data.
... For example, [30] used social media posts from Twitter and Facebook with BERT and deep learning models for analyzing attitudes towards COVID-19 vaccines. In another study, the authors classified garlic-related misinformation in the context of COVID-19 using different BERTbased pre-trained model variants using a large Twitter corpus [31]. The study [32] proposed a model named DICE which uses deep contextualized embedding of BERT in addition to a bidirectional LSTM network for sentiment detection. ...
Depression detection from social media texts such as Tweets or Facebook comments could be very beneficial as early detection of depression may even avoid extreme consequences of long-term depression i.e. suicide. In this study, depression intensity classification is performed using a labeled Twitter dataset. Further, this study makes a detailed performance evaluation of four transformer-based pre-trained small language models, particularly those having less than 15 million tunable parameters i.e. Electra Small Generator (ESG), Electra Small Discriminator (ESD), XtremeDistil-L6 (XDL) and Albert Base V2 (ABV) for classification of depression intensity using Tweets. The models are fine-tuned to get the best performance by applying different hyperparameters. The models are tested by classification of depression intensity of labeled tweets for three label classes i.e. ’severe’, ’moderate’, and ’mild’ by downstream fine-tuning the parameters. Evaluation metrics such as accuracy, F1, precision, recall, and specificity are calculated to evaluate the performance of the models. Comparative analysis of these models is also done with a moderately larger model i.e. DistilBert which has 67 million tunable parameters for the same task with the same experimental settings. Results indicate that ESG outperforms all other models including DistilBert due to its better deep contextualized text representation as it gets the best F1 score of 89% with comparatively less training time. This study helps to achieve better classification performance of depression detection as well as to choose the best language model in terms of performance and less training time for Twitter-related downstream NLP tasks.
