Proposed rumor detection framework for unknown rumors.

Proposed rumor detection framework for unknown rumors.

Source publication
Article
Full-text available
There have been many efforts to detect rumors using various machine learning (ML) models, but there is still a lack of understanding of their performance against different rumor topics and available features, resulting in a significant performance degrade against completely new and unseen (unknown) rumors. To address this issue, we investigate the...

Contexts in source publication

Context 1
... it is of paramount importance to ensure the performance and accuracy of the rumor detection component are high. Rumor detection using ML models normally has four main steps: (1) data collection, (2) feature extraction, (3) ML model training, and (4) performance evaluation, as depicted in Figure 1. Further details of each step are described as follows. ...
Context 2
... we can see, not all features used for Twitter are present. However, better performance was still observed using ES TOP3 in comparison to single ML models and eventAI, as shown in Figure 10. We suspect that the top three best performing ML models when used for ES can compliment and identify rumors that were otherwise would have been undetected. ...
Context 3
... users may believe that regardless of the ES configuration, using any combinations of the ML models (classifiers) as an ensemble could still be better at detecting unknown rumors. So, we carried out ES configuration comparisons as shown in Figure 11 using both PHEME and RE2019 datasets together. We setup three ES configurations: (1) Worst, (2) Random, and (3) Best. ...

Similar publications

Article
Full-text available
In the two recent decades various security authorities around the world acknowledged the importance of exploiting the ever-growing amount of information published on the web on various types of events for early detection of certain threats, situation monitoring and risk analysis. Since the information related to a particular real-world event might...

Citations

Article
With the increased popularity of social media platforms, people are increasingly depending on them for news and updates. Even official media channels post news on social media platforms such as Twitter and Facebook. However, with the vast amount of user-generated content, the credibility of shared information must be verified, and this process should be performed automatically and efficiently to accommodate the huge rate of generated posts. Current technology provides powerful methods and tools to solve the issue of rumor spreading on social networks. In this study, the aim is to investigate the use of state-of-the-art machine learning and deep learning models to detect rumors in a collection of Arabic tweets using the ArCOV19-Rumors dataset. A comprehensive comparison of the performance of the models was conducted. In deep learning experiments, the performances of seven optimizers were compared. The results demonstrated that using over-sampled data did not enhance classical and deep learning models. By contrast, using stacking classifiers increased the predictive model’s performance. As a result, the model became more logical and realistic in predicting rumors, non-rumors, and other classes than using classical machine learning without the stacking technique. Additionally, both long short-term memory (LSTM) and bidirectional-LSTM (Bi-LSTM) with the Root mean square propagation (RMSprop) optimizer obtained the best results. Finally, the results were analyzed to explain and interpret the low performance.
Conference Paper
Full-text available
News on social media can significantly influence users, manipulating them for political or economic reasons. Adversarial manipulations in the text have proven to create vulnerabilities in classifiers, and the current research is towards finding classifier models that are not susceptible to such manipulations. In this paper, we present a novel technique called ConTheModel, which slightly modifies social media news to confuse machine learning (ML)-based classifiers under the black-box setting. ConTheModel replaces a word in the original tweet with its synonym or antonym to generate tweets that confuse classifiers. We evaluate our technique on three different scenarios of the dataset and perform a comparison between five well-known machine learning algorithms, which includes Support Vector Machine (SVM), Naive Bayes (NB), Random Forest (RF), eXtreme Gradient Boosting (XGBoost), and Multilayer Perceptron (MLP) to demonstrate the performance of classifiers on the modifications done by ConTheModel. Our results show that the classifiers are confused after modification with the utmost drop of 16.36%. We additionally conducted a human study with 25 participants to validate the effectiveness of ConTheModel and found that the majority of participants (65%) found it challenging to classify the tweets correctly. We hope our work will help in finding robust ML models against adversarial examples.