Figure - uploaded by Ioannis A Kakadiaris
Content may be subject to copyright.
Statistics on the training and test datasets of Task 4b. All the numbers for the documents, snippets, concepts and triples refer to averages.

Statistics on the training and test datasets of Task 4b. All the numbers for the documents, snippets, concepts and triples refer to averages.

Context in source publication

Context 1
... A. Table 4 presents the statistics of the training and test data provided to the participants. The evaluation included five test batches. ...

Citations

... In this experiment, we present a systematic evaluation on biomedical questions provided by the BioASQ challenge so as to compare with BioASQ participant systems. As we previously noted, the BioASQ challenges in phase B (i.e., exact an ideal answers) of Task b provide the test set of biomedical questions along with their golden documents, golden snippets, and questions types [61,62,56] and participant systems [29,31,30,28,32] were asked to answer with exact answers and ideal answers using the golden documents, golden snippets, and golden questions types. For each question, each participating system may return an ideal answer, i.e., a paragraph-sized summary of relevant information. ...
Article
Background and objective Question answering (QA), the identification of short accurate answers to users questions written in natural language expressions, is a longstanding issue widely studied over the last decades in the open-domain. However, it still remains a real challenge in the biomedical domain as the most of the existing systems support a limited amount of question and answer types as well as still require further efforts in order to improve their performance in terms of precision for the supported questions. Here, we present a semantic biomedical QA system named SemBioNLQA which has the ability to handle the kinds of yes/no, factoid, list, and summary natural language questions. Methods This paper describes the system architecture and an evaluation of the developed end-to-end biomedical QA system named SemBioNLQA, which consists of question classification, document retrieval, passage retrieval and answer extraction modules. It takes natural language questions as input, and outputs both short precise answers and summaries as results. The SemBioNLQA system, dealing with four types of questions, is based on (1) handcrafted lexico-syntactic patterns and a machine learning algorithm for question classification, (2) PubMed search engine and UMLS similarity for document retrieval, (3) the BM25 model, stemmed words and UMLS concepts for passage retrieval, and (4) UMLS metathesaurus, BioPortal synonyms, sentiment analysis and term frequency metric for answer extraction. Results and conclusion Compared with the current state-of-the-art biomedical QA systems, SemBioNLQA, a fully automated system, has the potential to deal with a large amount of question and answer types. SemBioNLQA retrieves quickly users’ information needs by returning exact answers (e.g., “yes”, “no”, a biomedical entity name, etc.) and ideal answers (i.e., paragraph-sized summaries of relevant information) for yes/no, factoid and list questions, whereas it provides only the ideal answers for summary questions. Moreover, experimental evaluations performed on biomedical questions and answers provided by the BioASQ challenge especially in 2015, 2016 and 2017 (as part of our participation), show that SemBioNLQA achieves good performances compared with the most current state-of-the-art systems and allows a practical and competitive alternative to help information seekers find exact and ideal answers to their biomedical questions. The SemBioNLQA source code is publicly available at https://github.com/sarrouti/sembionlqa.
... An exception is the framework proposed by NCBI (Mao et al., 2014), which directly computes the cosine similarities between the questions and the sentences. Another team (Yang et al., 2016) introduced a unified classification interface for judging the relevance of each retrieved concept, document and snippet, which can combine the relevant scores evidenced by various sources (Krithara et al., 2016). ...
Article
Motivation: With the abundant medical resources, especially literature available online, it is possible for people to understand their own health status and relevant problems autonomously. However, how to obtain the most appropriate answer from the increasingly large-scale database, remains a great challenge. Here, we present a biomedical question answering framework and implement a system, Health Assistant, to enable the search process. Methods: In Health Assistant, a search engine is firstly designed to rank biomedical documents based on contents. Then various query processing and search techniques are utilized to find the relevant documents. Afterwards, the titles and abstracts of top-N documents are extracted to generate candidate snippets. Finally, our own designed query processing and retrieval approaches for short text are applied to locate the relevant snippets to answer the questions. Results: Our system is evaluated on the BioASQ benchmark datasets, and experimental results demonstrate the effectiveness and robustness of our system, compared to BioASQ participant systems and some state-of-the-art methods on both document retrieval and snippet retrieval tasks. Availability and implementation: A demo of our system is available at https://github.com/jinzanxia/biomedical-QA.
... In this paper, we investigate the effectiveness of BioBERT in biomedical question answering and report our results from the 7th BioASQ Challenge [7,10,11,21]. Biomedical question answering has its own unique challenges. First, the size of datasets is often very small (e.g., few thousands of samples in BioASQ) as the creation of biomedical question answering datasets is very expensive. ...
Preprint
Full-text available
The recent success of question answering systems is largely attributed to pre-trained language models. However, as language models are mostly pre-trained on general domain corpora such as Wikipedia, they often have difficulty in understanding biomedical questions. In this paper, we investigate the performance of BioBERT, a pre-trained biomedical language model, in answering biomedical questions including factoid, list, and yes/no type questions. BioBERT uses almost the same structure across various question types and achieved the best performance in the 7th BioASQ Challenge (Task 7b, Phase B). BioBERT pre-trained on SQuAD or SQuAD 2.0 easily outperformed previous state-of-the-art models. BioBERT obtains the best performance when it uses the appropriate pre-/post-processing strategies for questions, passages, and answers.
... Additionally, the CNN is trained with different pre-trained word embedding models and compared with the random initialization. First, the different word embedding models using the toolkit Word2vec (Mikolov, Sutskever, Chen, Corrado and Dean, 2013) are trained on the BioASQ 2016 dataset (Krithara et al., 2016), which contains more than 12 million MedLine abstracts. Skip-gram and continuous bag-of-words (CBOW) architectures of Word2vec are applied with the default parameters used in the C version of the Word2vec toolkit (i.e. ...
Thesis
Full-text available
The main hypothesis of this PhD dissertation is that novel Deep Learning algorithms can outperform classical Machine Learning methods for the task of Information Extraction in the Biomedical Domain. Contrary to classical systems, Deep Learning models can learn the representation of the data automatically without an expert domain knowledge and avoid the tedious and time-consuming task of defining relevant features. A Drug-Drug Interaction (DDI), which is an essential subset of Adverse Drug Reaction (ADR), represents the alterations in the effects of drugs that were taken simultaneously. The early recognition of interacting drugs is a vital process that prevents serious health problems that can cause death in the worst cases. Health-care professionals and researchers in this domain find the task of discovering information about these incidents very challenging due to the vast number of pharmacovigilance documents. For this reason, several shared tasks and datasets have been developed in order to solve this issue with automated annotation systems with the capability to extract this information. In the present document, the DDI corpus, which is an annotated dataset of DDIs, is used with Deep Learning architectures without any external information for the tasks of Name Entity Recognition and Relation Extraction in order to validate the hypothesis. Furthermore, some other datasets are tested to evidence the performance of these systems. To sum up, the results suggest that the most common Deep Learning methods like Convolutional Neural Networks and Recurrent Neural Networks overcome the traditional algorithms concluding that Deep Learning is a real alternative for a specific and complex scenario like the Information Extraction in the Biomedical domain. As a final goal, a complete architecture that covers the two tasks is developed to structure the named entities and their relationships from raw pharmacological texts.
... The BioASQ challenge was continuously conducted each year until today with many participants and a variety of approaches in both tasks A and B [8,5,19,25]. In Task A, MeSHLabeler won the challenge in 2014, 2015 and 2016 [21] using an ensemble approach of k-NN, the MTI itself as well as further MeSH classification approaches. ...
Preprint
Full-text available
Official MeSH annotations are provided from curators at the National Library of Medicine (NLM). Efforts to automatically assign such headings to Medline abstracts have proven difficult. Trained solutions , i.e. machine learning solutions, achieve promising results, however even these successes leave the open question, which features from the text best support the identification of MeSH terms from a given Medline abstract. This manuscript lays out specific approaches for the identification and use of contextual features for the Multi-Label Classification (BioASQ Task6a). In particular, the use of different approaches for the identification of compound terms have been tested. Furthermore, the used system has been extended to better rank selected labels for the BioASQ Task7a challenge. The tested solutions improved recall measures (see Task6a) whereas the second system did boost both performance for both precision-measures and recall-measures. Our presented work gives insights into the use of contextual features from text that would reduce the performance gap given to purely trained solutions in the respective tasks. Nevertheless, we still recognize that lexical features based on the MeSH thesaurus have a high discrepancy towards the actual annotation of MeSH Heading to Medline citations by human curators, another gap that requires explanations to improve the automatic annotation of Medline abstracts with MeSH Headings.
... BioASQ (Krithara et al., 2016), i2b2 (Sun et al., 2013), and ShARe/CLEF eHealth (Kelly et al., 2014). Huang and Lu (2016) provides an overview of the community challenges organized over a period of 12 years. ...
Chapter
Full-text available
Biomedical literature has become a rich source of information for various applications. Automatic text mining methods can make the processing of extracting information from a large set of documents more efficient. However, since natural language is not easily processed by computer programs, it is necessary to develop algorithms to transform text into a structured representation. Scientific texts present a challenge to text mining methods since the language used is formal and highly specialized. This article presents an overview of the current biomedical text mining tools and bioinformatics applications using them.
... BioNLP-ST has organized various biomedical IE tasks, usually focused on a specific biological system such as seed development [24], epigenetics and post-translational modifications [80], and cancer genetics [81]. Other community challenges relevant to biomedical text mining include JNLPBA [82], BioASQ [83], i2b2 [84], and ShARe/CLEF eHealth [85]. ...
... Here, we will briefly introduce other participants' methods of document retrieval employed in the 2016 [37] and 2017 BioASQ [38] challenge. Papagiannopoulou et al. [39] built their system on Indri search engine and a variety of libraries had been used, such as the StAX Parser, the Stanford Parser and the GSON library. ...
Article
The conventional Sequential Dependence Model (SDM) has been proved to perform better than the Bag of Words (BoW) model for biomedical article search because it pays attention to the sequence information within queries. Meanwhile, introducing lexical semantic relations into query expansion becomes a hot topic in IR research. However, few researches have been conducted on combining semantic and sequence information together. Hence, we propose the Semantic Sequential Dependence Model (SSDM) in this paper, which provides an innovative combination of semantic information and the conventional SDM. Specifically, our synonyms are obtained automatically through the word embeddings which are trained on the domain-specific corpus by selecting an appropriate language model. Then, these synonyms are utilized to generate possible sequences with the same semantics as the original query and these sequences are fed into SDM to obtain the final retrieval results. The proposed approach is evaluated on 2016 and 2017 BioASQ benchmark test sets and the experimental results show that our query expansion approach outperforms the baseline and other participants in the BioASQ competitions. OAPA
... In phase B, the released questions contained the golden answers for the required elements (documents and snippets) of the first phase. The participants had to answer with exact answers (e.g., biomedical entity, number, list of biomedical entities, yes, no, etc.) as well as with ideal answers (i.e., paragraph-sized summaries) (Krithara et al., 2016). In this paper, we describe our participation in the phase B (i.e., exact and ideal answers) of task5b in the 2017 BioASQ challenge. ...
Conference Paper
Full-text available
Question answering, the identification of short accurate answers to users questions, is a longstanding challenge widely studied over the last decades in the opendomain. However, it still requires further efforts in the biomedical domain. In this paper, we describe our participation in phase B of task 5b in the 2017 BioASQ challenge using our biomedical question answering system. Our system, dealing with four types of questions (i.e., yes/no, factoid, list, and summary), is based on (1) a dictionary-based approach for generating the exact answers of yes/no questions, (2) UMLS metathesaurus and term frequency metric for extracting the exact answers of factoid and list questions, and (3) the BM25 model and UMLS concepts for retrieving the ideal answers (i.e., paragraph-sized summaries). Preliminary results show that our system achieves good and competitive results in both exact and ideal answers extraction tasks as compared with the participating systems.
... Thus, in addition to the CNN with a random initialization, we trained a CNN with different pre-trained word embedding models. First, we pre-trained different word embedding models using the toolkit word2vec (27) on the BioASQ 2016 dataset (28), which contains more than 12 million MedLine abstracts. We used both architectures of word2vec, skip-gram and continuous bagof-words (CBOW), and applied the default parameters used in the C version of the word2vec toolkit (i.e. ...
Article
Full-text available
Drug–drug interaction (DDI), which is a specific type of adverse drug reaction, occurs when a drug influences the level or activity of another drug. Natural language processing techniques can provide health-care professionals with a novel way of reducing the time spent reviewing the literature for potential DDIs. The current state-of-the-art for the extraction of DDIs is based on feature-engineering algorithms (such as support vector machines), which usually require considerable time and effort. One possible alternative to these approaches includes deep learning. This technique aims to automatically learn the best feature representation from the input data for a given task. The purpose of this paper is to examine whether a convolutional neural network (CNN), which only uses word embeddings as input features, can be applied successfully to classify DDIs from biomedical texts. Proposed herein, is a CNN architecture with only one hidden layer, thus making the model more computationally efficient, and we perform detailed experiments in order to determine the best settings of the model. The goal is to determine the best parameter of this basic CNN that should be considered for future research. The experimental results show that the proposed approach is promising because it attained the second position in the 2013 rankings of the DDI extraction challenge. However, it obtained worse results than previous works using neural networks with more complex architectures.