Article

Mixture of experts models to exploit global sequence similarity on biomolecular sequence labeling

Artificial Intelligence Research Laboratory, Computer Science Department, Iowa State University, Ames, IA 50010, USA.
BMC Bioinformatics (Impact Factor: 2.58). 02/2009; 10 Suppl 4(Suppl 4):S4. DOI: 10.1186/1471-2105-10-S4-S4
Source: PubMed

ABSTRACT

Identification of functionally important sites in biomolecular sequences has broad applications ranging from rational drug design to the analysis of metabolic and signal transduction networks. Experimental determination of such sites lags far behind the number of known biomolecular sequences. Hence, there is a need to develop reliable computational methods for identifying functionally important sites from biomolecular sequences.
We present a mixture of experts approach to biomolecular sequence labeling that takes into account the global similarity between biomolecular sequences. Our approach combines unsupervised and supervised learning techniques. Given a set of sequences and a similarity measure defined on pairs of sequences, we learn a mixture of experts model by using spectral clustering to learn the hierarchical structure of the model and by using bayesian techniques to combine the predictions of the experts. We evaluate our approach on two biomolecular sequence labeling problems: RNA-protein and DNA-protein interface prediction problems. The results of our experiments show that global sequence similarity can be exploited to improve the performance of classifiers trained to label biomolecular sequence data.
The mixture of experts model helps improve the performance of machine learning methods for identifying functionally important sites in biomolecular sequences.

Download full-text

Full-text

Available from: Jivko Sinapov
  • Source
    • "In general, the objective of the Mixture-of-Experts (MoE) [12] approach is to replace a single complex computational model that might be applicable to only a subset of the possible inputs, by multiple simpler and more tailored models (each corresponding to an expert) that can, individually or in some combination, generate equally good or improved solutions for a larger range of input cases. MoE systems find application in many domains, including classification, image processing, time-series prediction, data mining, fault-tolerance, modeling, etc [4] [6] [11] [15] [16]. This paper considers the challenge of scheduling the execution of experts in MoE systems with real-time constraints and limited resources. "
    [Show abstract] [Hide abstract]
    ABSTRACT: Mixture-of-Experts (MoE) systems solve intricate problems by combining results generated independently by multiple computational models (the "experts"). Given an instance of a problem, the responsibility of an expert measures the degree to which the expert's output contributes to the final solution. Brain Machine Interfaces are examples of applications where an MoE system needs to run periodically and expert responsibilities can vary across execution cycles. When resources are insufficient to run all experts in every cycle, it becomes necessary to execute the most responsible experts within each cycle. The problem of adaptively scheduling experts with dynamic responsibilities can be formulated as a succession of optimization problems. Each of these problems can be solved by a known technique called "task compression" using explicit mappings described in this paper to relate expert responsibilities to task elasticities. A novel heuristic is proposed to enable real-time execution rate adaptation in MoE systems with insufficient resources. In any given cycle, the heuristic uses sensitivity analysis to test whether one of two pre-computed schedules is the optimal solution of the optimization problem to avoid re-optimization when the test result is positive. These two candidate schedules are the schedule used in the previous cycle and the schedule pre-computed by the heuristic during the previous cycle, using future responsibilities predicted by the heuristic's responsibility predictor. Our heuristic significantly reduces the scheduling delay in the execution of experts when re-execution of the task-compression algorithm is not needed from O(N2) time, where N denotes the number of experts, to O(N) time. Experimental evaluation of the heuristic on a test case in motor control shows that these time savings occur and scheduled experts' deadlines are met in up to 90% of all cycles. For the test scenario considered in the paper, the average output error of a real-time MoE system due to the use of limited resources is less than 7%.
    Full-text · Conference Paper · Jan 2010
  • Source
    [Show abstract] [Hide abstract]
    ABSTRACT: Several experiments were conducted in order to investigate the usefulness of mixture of experts (ME) approach to an online internet system assisting in real estate appraisal. All experiments were performed using 28 real-world datasets composed of data taken from a cadastral system and GIS data derived from a cadastral map. The analysis of the results was performed using recently proposed statistical methodology including nonparametric tests followed by post-hoc procedures designed especially for multiple 1×n and n×n comparisons. GLM (general linear model) architectures of mixture of experts achieved better results for ME with an adaptive variance parameter for each expert, whereas MLP (multilayer perceptron) architectures - for standard mixtures of experts.
    Full-text · Conference Paper · Jun 2010
  • Source
    [Show abstract] [Hide abstract]
    ABSTRACT: Several experiments were conducted in order to investigate the usefulness of mixture of experts approach to an online internet system assisting in real estate appraisal. All experiments were performed using real-world datasets taken from a cadastral system. The analysis of the results was performed using statistical methodology including nonparametric tests followed by post-hoc procedures designed especially for multiple 1×N and N×N comparisons. The mixture of experts architectures studied in the paper comprised: four algorithms used as expert networks: glm --- general linear model, mlp --- multilayer perceptron and two support vector regression ε-SVR and ν-SVR as well as and three algorithms glm, mlp, and gmm --- gaussian mixture model employed as gating networks.
    Full-text · Conference Paper · Mar 2013
Show more