Mixture of experts models to exploit global sequence similarity on biomolecular sequence labeling

Artificial Intelligence Research Laboratory, Computer Science Department, Iowa State University, Ames, IA 50010, USA.
BMC Bioinformatics (Impact Factor: 2.67). 02/2009; 10 Suppl 4(Suppl 4):S4. DOI: 10.1186/1471-2105-10-S4-S4
Source: PubMed

ABSTRACT Identification of functionally important sites in biomolecular sequences has broad applications ranging from rational drug design to the analysis of metabolic and signal transduction networks. Experimental determination of such sites lags far behind the number of known biomolecular sequences. Hence, there is a need to develop reliable computational methods for identifying functionally important sites from biomolecular sequences.
We present a mixture of experts approach to biomolecular sequence labeling that takes into account the global similarity between biomolecular sequences. Our approach combines unsupervised and supervised learning techniques. Given a set of sequences and a similarity measure defined on pairs of sequences, we learn a mixture of experts model by using spectral clustering to learn the hierarchical structure of the model and by using bayesian techniques to combine the predictions of the experts. We evaluate our approach on two biomolecular sequence labeling problems: RNA-protein and DNA-protein interface prediction problems. The results of our experiments show that global sequence similarity can be exploited to improve the performance of classifiers trained to label biomolecular sequence data.
The mixture of experts model helps improve the performance of machine learning methods for identifying functionally important sites in biomolecular sequences.

  • [Show abstract] [Hide abstract]
    ABSTRACT: A Mixture-of-Experts (MoE) system generates an output in each operating cycle by combining results of multiple models (the “experts”). The contribution of any given expert to a final solution depends on a parameter called responsibility, which can vary from cycle to cycle. When resources are insufficient to run all experts, two problems arise: 1) how much utilization is to be allocated to experts and 2) how can a schedule be created based on these allocations. Problem (1) can be formulated as a succession of optimization problems, each of which calculates experts’ allocations in a cycle. Explicit mappings from responsibilities to allocation weights are needed to solve each of these problems in every cycle using a technique called “task compression (TC).” We refer to this baseline approach as TT-TC. Two other proposed heuristics ${ssr TT}hbox{-}{ssr TC}^ast$ and TT-Top reduce TC’s execution time to ${ssr O}({mbi{N}})$ for ${mbi{N}}$ experts. To address (2), the proposed EPOC scheduler converts the heuristics’ allocations into schedules that satisfy capacity, execution, and learning constraints across cycles. Simulations demonstrate that our approaches enable real-time computation and significantly decrease the average percentage error of limited-resource outputs (i.e., 0.2%–40% and 0.3%–0.5% when scheduled with ${ssr TT}hbox{-}{ssr TC}^ast$ and TT-Top, respectively, versus 0.2%–97% when using TT-TC).
    IEEE Transactions on Computers 07/2014; 63(7):1751-1764. DOI:10.1109/TC.2013.50 · 1.47 Impact Factor
  • Source
    [Show abstract] [Hide abstract]
    ABSTRACT: Mixture-of-Experts (MoE) systems solve intricate problems by combining results generated independently by multiple computational models (the "experts"). Given an instance of a problem, the responsibility of an expert measures the degree to which the expert's output contributes to the final solution. Brain Machine Interfaces are examples of applications where an MoE system needs to run periodically and expert responsibilities can vary across execution cycles. When resources are insufficient to run all experts in every cycle, it becomes necessary to execute the most responsible experts within each cycle. The problem of adaptively scheduling experts with dynamic responsibilities can be formulated as a succession of optimization problems. Each of these problems can be solved by a known technique called "task compression" using explicit mappings described in this paper to relate expert responsibilities to task elasticities. A novel heuristic is proposed to enable real-time execution rate adaptation in MoE systems with insufficient resources. In any given cycle, the heuristic uses sensitivity analysis to test whether one of two pre-computed schedules is the optimal solution of the optimization problem to avoid re-optimization when the test result is positive. These two candidate schedules are the schedule used in the previous cycle and the schedule pre-computed by the heuristic during the previous cycle, using future responsibilities predicted by the heuristic's responsibility predictor. Our heuristic significantly reduces the scheduling delay in the execution of experts when re-execution of the task-compression algorithm is not needed from O(N2) time, where N denotes the number of experts, to O(N) time. Experimental evaluation of the heuristic on a test case in motor control shows that these time savings occur and scheduled experts' deadlines are met in up to 90% of all cycles. For the test scenario considered in the paper, the average output error of a real-time MoE system due to the use of limited resources is less than 7%.
    Proceedings of the 13th ACM International Conference on Hybrid Systems: Computation and Control, HSCC 2010, Stockholm, Sweden, April 12-15, 2010; 01/2010

Full-text (3 Sources)

Available from
Jun 2, 2014