Fernando Pérez-Cruz

University Carlos III de Madrid, Getafe, Madrid, Spain

Are you Fernando Pérez-Cruz?

Claim your profile

Publications (103)127.94 Total impact

  • Isabel Valera · Francisco Ruiz · Fernando Perez-Cruz ·
    [Show abstract] [Hide abstract]
    ABSTRACT: There are many scenarios in artificial intelligence, signal processing or medicine, in which a temporal sequence consists of several unknown overlapping independent causes, and we are interested in accurately recovering those canonical causes. Factorial hidden Markov models (FHMMs) present the versatility to provide a good fit to these scenarios. However, in some scenarios, the number of causes or the number of states of the FHMM cannot be known or limited a priori. In this paper, we propose an infinite factorial unbounded-state hidden Markov model (IFUHMM), in which the number of parallel hidden Markov models (HMMs) and states in each HMM are potentially unbounded. We rely on a Bayesian nonparametric (BNP) prior over integer-valued matrices, in which the columns represent the Markov chains, the rows the time indexes, and the integers the state for each chain and time instant. First, we extend the existent infinite factorial binary-state HMM to allow for any number of states. Then, we modify this model to allow for an unbounded number of states and derive an MCMC-based inference algorithm that properly deals with the trade-off between the unbounded number of states and chains. We illustrate the performance of our proposed models in the power disaggregation problem.
    IEEE Transactions on Pattern Analysis and Machine Intelligence 11/2015; DOI:10.1109/TPAMI.2015.2498931 · 5.78 Impact Factor
  • [Show abstract] [Hide abstract]
    ABSTRACT: In this paper we propose an Iterative Re-Weighted Least Square procedure in order to solve the Support Vector Machines for regression and function estimation. Furthermore, we include a new algorithm to train Support Vector Machines, covering both the proposed approach instead of the quadratic programming part and the most advanced methods to deal with large training data sets. Finally, the performance of the method is assessed by selected examples which show that the training time is much shorter and the memory requirements much less than the employed ones by current methods.
  • Source
    Francisco J. R. Ruiz · Fernando Perez-Cruz ·
    [Show abstract] [Hide abstract]
    ABSTRACT: We show that a classical model for soccer can also provide competitive results in predicting basketball outcomes. We modify the classical model in two ways in order to capture both the specific behavior of each National collegiate athletic association (NCAA) conference and different strategies of teams and conferences. Through simulated bets on six online betting houses, we show that this extension leads to better predictive performance in terms of profit we make. We compare our estimates with the probabilities predicted by the winner of the recent Kaggle competition on the 2014 NCAA tournament, and conclude that our model tends to provide results that differ more from the implicit probabilities of the betting houses and, therefore, has the potential to provide higher benefits.
    Journal of Quantitative Analysis in Sports 01/2015; DOI:10.1515/jqas-2014-0055
  • Javier Cespedes · P.M. Olmos · Matilde Sanchez-Fernandez · Fernando Perez-Cruz ·
    [Show abstract] [Hide abstract]
    ABSTRACT: Modern communications systems use multiple-input multiple-output (MIMO) and high-order QAM constellations for maximizing spectral efficiency. However, as the number of antennas and the order of the constellation grow, the design of efficient and low-complexity MIMO receivers possesses big technical challenges. For example, symbol detection can no longer rely on maximum likelihood detection or sphere-decoding methods, as their complexity increases exponentially with the number of transmitters/receivers. In this paper, we propose a low-complexity high-accuracy MIMO symbol detector based on the Expectation Propagation (EP) algorithm. EP allows approximating iteratively at polynomial-time the posterior distribution of the transmitted symbols. We also show that our EP MIMO detector outperforms classic and state-of-the-art solutions reducing the symbol error rate at a reduced computational complexity.
    IEEE Transactions on Communications 08/2014; 62(8):2840-2849. DOI:10.1109/TCOMM.2014.2332349 · 1.99 Impact Factor
  • Source
    Pablo G. Moreno · Yee Whye Teh · Fernando Perez-Cruz · Antonio Artés-Rodríguez ·
    [Show abstract] [Hide abstract]
    ABSTRACT: Crowdsourcing has been proven to be an effective and efficient tool to annotate large datasets. User annotations are often noisy, so methods to combine the annotations to produce reliable estimates of the ground truth are necessary. We claim that considering the existence of clusters of users in this combination step can improve the performance. This is especially important in early stages of crowdsourcing implementations, where the number of annotations is low. At this stage there is not enough information to accurately estimate the bias introduced by each annotator separately, so we have to resort to models that consider the statistical links among them. In addition, finding these clusters is interesting in itself as knowing the behavior of the pool of annotators allows implementing efficient active learning strategies. Based on this, we propose in this paper two new fully unsupervised models based on a Chinese Restaurant Process (CRP) prior and a hierarchical structure that allows inferring these groups jointly with the ground truth and the properties of the users. Efficient inference algorithms based on Gibbs sampling with auxiliary variables are proposed. Finally, we perform experiments, both on synthetic and real databases, to show the advantages of our models over state-of-the-art algorithms.
  • Camilo G. Taborda · Fernando Perez-Cruz · Dongning Guo ·
    [Show abstract] [Hide abstract]
    ABSTRACT: In recent years, a number of mathematical relationships have been established between information measures and estimation measures for various models, including Gaussian, Poisson and binomial models. In this paper, it is shown that the second derivative of the input-output mutual information with respect to the input scaling can be expressed as the expectation of a certain Bregman divergence pertaining to the conditional expectations of the input and the input power. This result is similar to that found for the Gaussian model where the Bregman divergence therein is the square distance. In addition, the Poisson, binomial and negative binomial models are shown to be similar in the small scaling regime in the sense that the derivative of the mutual information and the derivative of the relative entropy converge to the same value.
    2014 IEEE International Symposium on Information Theory (ISIT); 06/2014
  • Javier Cespedes · Pablo M. Olmos · Matilde Sanchez-Fernandez · Fernando Perez-Cruz ·
    [Show abstract] [Hide abstract]
    ABSTRACT: Modern communications systems use efficient encoding schemes, multiple-input multiple-output (MIMO) and high-order QAM constellations for maximizing spectral efficiency. However, as the dimensions of the system grow, the design of efficient and low-complexity MIMO receivers possesses technical challenges. Symbol detection can no longer rely on conventional approaches for posterior probability computation due to complexity. Marginalization of this posterior to obtain per-antenna soft-bit probabilities to be fed to a channel decoder is computationally challenging when realistic signaling is used. In this work, we propose to use Expectation Propagation (EP) algorithm to provide an accurate low-complexity Gaussian approximation to the posterior, easily solving the posterior marginalization problem. EP soft-bit probabilities are used in an LDPC-coded MIMO system, achieving outstanding performance improvement compared to similar approaches in the literature for low-complexity LDPC MIMO decoding.
    2014 IEEE International Symposium on Information Theory (ISIT); 06/2014
  • Isabel Valera · Francisco J. R. Ruiz · Fernando Perez-Cruz ·
    [Show abstract] [Hide abstract]
    ABSTRACT: Bayesian nonparametric models allow solving estimation and detection problems with an unbounded number of degrees of freedom. In multiuser multiple-input multiple-output (MIMO) communication systems we might not know the number of active users and the channel they face, and assuming maximal scenarios (maximum number of transmitters and maximum channel length) might degrade the receiver performance. In this paper, we propose a Bayesian nonparametric prior and its associated inference algorithm, which is able to detect an unbounded number of users with an unbounded channel length. This generative model provides the dispersive channel model for each user and a probabilistic estimate for each transmitted symbol in a fully blind manner, i.e., without the need of pilot (training) symbols.
    2014 4th International Workshop on Cognitive Information Processing (CIP); 05/2014
  • Camilo G. Taborda · Dongning Guo · Fernando Perez-Cruz ·
    [Show abstract] [Hide abstract]
    ABSTRACT: In recent years, a number of new connections between information measures and estimation have been found under various models, including, predominantly, Gaussian and Poisson models. This paper develops similar results for the binomial and negative binomial models. In particular, it is shown that the derivative of the relative entropy and the derivative of the mutual information for the binomial and negative binomial models can be expressed through the expectation of closed-form expressions that have conditional estimates as the main argument. Under mild conditions, those derivatives take the form of an expected Bregman divergence.
    IEEE Transactions on Information Theory 05/2014; 60(5):2630-2646. DOI:10.1109/TIT.2014.2307070 · 2.33 Impact Factor
  • Source
    Francisco J. R. Ruiz · Isabel Valera · Carlos Blanco · Fernando Perez-Cruz ·
    [Show abstract] [Hide abstract]
    ABSTRACT: The analysis of comorbidity is an open and complex research field in the branch of psychiatry, where clinical experience and several studies suggest that the relation among the psychiatric disorders may have etiological and treatment implications. In this paper, we are interested in applying latent feature modeling to find the latent structure behind the psychiatric disorders that can help to examine and explain the relationships among them. To this end, we use the large amount of information collected in the National Epidemiologic Survey on Alcohol and Related Conditions (NESARC) database and propose to model these data using a nonparametric latent model based on the Indian Buffet Process (IBP). Due to the discrete nature of the data, we first need to adapt the observation model for discrete random variables. We propose a generative model in which the observations are drawn from a multinomial-logit distribution given the IBP matrix. The implementation of an efficient Gibbs sampler is accomplished using the Laplace approximation, which allows integrating out the weighting factors of the multinomial-logit likelihood model. We also provide a variational inference algorithm for this model, which provides a complementary (and less expensive in terms of computational complexity) alternative to the Gibbs sampler allowing us to deal with a larger number of data. Finally, we use the model to analyze comorbidity among the psychiatric disorders diagnosed by experts from the NESARC database.
    Journal of Machine Learning Research 01/2014; 15. · 2.47 Impact Factor
  • [Show abstract] [Hide abstract]
    ABSTRACT: The dependence-testing (DT) bound is one of the strongest achievability bounds for the binary erasure channel (BEC) in the finite block length regime. In this paper, we show that maximum likelihood decoded regular low-density parity-check (LDPC) codes with at least 5 ones per column almost achieve the DT bound. Specifically, using quasi-regular LDPC codes with block length of 256 bits, we achieve a rate that is less than 1% away from the rate predicted by the DT bound for a word error rate below $10^{-3} $. The results also indicate that the maximum-likelihood solution is computationally feasible for decoding block codes over the BEC with several hundred bits.
    IEEE Communications Letters 01/2014; 19(2):1-1. DOI:10.1109/LCOMM.2014.2371032 · 1.27 Impact Factor
  • Source
    [Show abstract] [Hide abstract]
    ABSTRACT: Automated screening systems are commonly used to detect some agent in a sample and take a global decision about the subject (e.g. ill/healthy) based on these detections. We propose a Bayesian methodology for taking decisions in (sequential) screening systems that considers the false alarm rate of the detector. Our approach assesses the quality of its decisions and provides lower bounds on the achievable performance of the screening system from the training data. In addition, we develop a complete screening system for sputum smears in tuberculosis diagnosis, and show, using a real-world database, the advantages of the proposed framework when compared to the commonly used count detections and threshold approach.
    10/2013; 18(3). DOI:10.1109/JBHI.2013.2282874
  • [Show abstract] [Hide abstract]
    ABSTRACT: Gaussian processes (GPs) are versatile tools that have been successfully employed to solve nonlinear estimation problems in machine learning but are rarely used in signal processing. In this tutorial, we present GPs for regression as a natural nonlinear extension to optimal Wiener filtering. After establishing their basic formulation, we discuss several important aspects and extensions, including recursive and adaptive algorithms for dealing with nonstationarity, low-complexity solutions, non-Gaussian noise models, and classification scenarios. Furthermore, we provide a selection of relevant applications to wireless digital communications.
    IEEE Signal Processing Magazine 07/2013; 30(4):40-50. DOI:10.1109/MSP.2013.2250352 · 5.85 Impact Factor
  • L. Salamanca · J.-J. Murillo-Fuentes · P.M. Olmos · F. Perez-Cruz ·
    [Show abstract] [Hide abstract]
    ABSTRACT: In this paper, we propose the tree-structured expectation propagation (TEP) algorithm for low-density parity-check (LDPC) decoding over the binary additive white Gaussian noise (BI-AWGN) channel. By approximating the posterior distribution by a tree-structure factorization, the TEP has been proven to improve belief propagation (BP) decoding over the binary erasure channel (BEC). We show for the AWGN channel how the TEP decoder is also able to capture additional information disregarded by the BP solution, which leads to a noticeable reduction of the error rate for finite-length codes. We show that for the range of codes of interest, the TEP gain is obtained with a slight increase in complexity over that of the BP algorithm. An efficient way of constructing the tree-like structure is also described.
    Information Theory Proceedings (ISIT), 2013 IEEE International Symposium on; 01/2013
  • Pablo M Olmos · Fernando Pérez-Cruz · Luis Salamanca · Juan José Murillo-Fuentes ·
    [Show abstract] [Hide abstract]
    ABSTRACT: In this work, we analyze the finite-length performance of low-density parity check (LDPC) ensembles decoded over the binary erasure channel (BEC) using the tree-expectation propagation (TEP) algorithm. In a previous paper, we showed that the TEP improves the BP performance for decoding regular and irregular short LDPC codes, but the perspective was mainly empirical. In this work, given the degree-distribution of an LDPC ensemble, we explain and predict the range of code lengths for which the TEP improves the BP solution. In addition, for LDPC ensembles that present a single critical point, we propose a scaling law to accurately predict the performance in the waterfall region. These results are of critical importance to design practical LDPC codes for the TEP decoder.
    IEEE International Symposium on Information Theory Proceedings (ISIT); 07/2012
  • Camilo G. Taborda · Fernando Perez-Cruz ·
    [Show abstract] [Hide abstract]
    ABSTRACT: We study the relation of the mutual information and relative entropy over the Binomial and Negative Binomial channels with estimation theoretical quantities, in which we extend already known results for Gaussian and Poisson channels. We establish general expressions for these information theory concepts with a direct connection with estimation theory through the conditional mean estimation and a particular loss function.
    Information Theory Proceedings (ISIT), 2012 IEEE International Symposium on; 07/2012
  • Pablo M. Olmos · Luis Salamanca · Juan José Murillo-Fuentes · Fernando Pérez-Cruz ·
    [Show abstract] [Hide abstract]
    ABSTRACT: Low-density parity-check convolutional (LDPCC) codes asymptotically achieve channel capacity under belief propagation (BP) decoding. In this paper, we decode LDPCC codes using the Tree-Expectation Propagation (TEP) decoder, recently proposed as an alternative decoding method to the BP algorithm for the binary erasure channel (BEC). We show that, for LDPCC codes, the TEP decoder improves the BP solution with a comparable complexity or, alternatively, it allows using shorter codes to achieve similar error rates. We also propose a window-sliding scheme for the TEP decoder to reduce the decoding latency.
    IEEE Communications Letters 05/2012; 16(5-5):726-729. DOI:10.1109/LCOMM.2012.030912.120092 · 1.27 Impact Factor
  • Luis Salamanca · Juan José Murillo-Fuentes · Fernando Pérez-Cruz ·
    [Show abstract] [Hide abstract]
    ABSTRACT: We describe the channel equalization problem, and its prior estimate of the channel state information (CSI), as a joint Bayesian estimation problem to improve each symbol posterior estimates at the input of the channel decoder. Our approach takes into consideration not only the uncertainty due to the noise in the channel, but also the uncertainty in the CSI estimate. However, this solution cannot be computed in linear time, because it depends on all the transmitted symbols. Hence, we also put forward an approximation for each symbol's posterior, using the expectation propagation algorithm, which is optimal from the Kullback–Leibler divergence viewpoint and yields an equalization with a complexity identical to the BCJR algorithm. We also use a graphical model representation of the full posterior, in which the proposed approximation can be readily understood. The proposed posterior estimates are more accurate than those computed using the ML estimate for the CSI. In order to illustrate this point, we measure the error rate at the output of a low-density parity-check decoder, which needs the exact posterior for each symbol to detect the incoming word and it is sensitive to a mismatch in those posterior estimates. For example, for QPSK modulation and a channel with three taps, we can expect gains over 0.5 dB with same computational complexity as the ML receiver.
    IEEE Transactions on Signal Processing 05/2012; 60(5-5):2672-2676. DOI:10.1109/TSP.2012.2184098 · 2.79 Impact Factor
  • Source
    [Show abstract] [Hide abstract]
    ABSTRACT: Strategies for generating knowledge in medicine have included observation of associations in clinical or research settings and more recently, development of pathophysiological models based on molecular biology. Although critically important, they limit hypothesis generation to an incremental pace. Machine learning and data mining are alternative approaches to identifying new vistas to pursue, as is already evident in the literature. In concert with these analytic strategies, novel approaches to data collection can enhance the hypothesis pipeline as well. In data farming, data are obtained in an 'organic' way, in the sense that it is entered by patients themselves and available for harvesting. In contrast, in evidence farming (EF), it is the provider who enters medical data about individual patients. EF differs from regular electronic medical record systems because frontline providers can use it to learn from their own past experience. In addition to the possibility of generating large databases with farming approaches, it is likely that we can further harness the power of large data sets collected using either farming or more standard techniques through implementation of data-mining and machine-learning strategies. Exploiting large databases to develop new hypotheses regarding neurobiological and genetic underpinnings of psychiatric illness is useful in itself, but also affords the opportunity to identify novel mechanisms to be targeted in drug discovery and development.
    Molecular Psychiatry 01/2012; 17(10):956-9. DOI:10.1038/mp.2011.173 · 14.50 Impact Factor
  • Source
    Pablo M. Olmos · Juan José Murillo-Fuentes · Fernando Pérez-Cruz ·
    [Show abstract] [Hide abstract]
    ABSTRACT: We present the tree-structure expectation propagation (Tree-EP) algorithm to decode low-density parity-check (LDPC) codes over discrete memoryless channels (DMCs). EP generalizes belief propagation (BP) in two ways. First, it can be used with any exponential family distribution over the cliques in the graph. Second, it can impose additional constraints on the marginal distributions. We use this second property to impose pair-wise marginal constraints over pairs of variables connected to a check node of the LDPC code's Tanner graph. Thanks to these additional constraints, the Tree-EP marginal estimates for each variable in the graph are more accurate than those provided by BP. We also reformulate the Tree-EP algorithm for the binary erasure channel (BEC) as a peeling-type algorithm (TEP) and we show that the algorithm has the same computational complexity as BP and it decodes a higher fraction of errors. We describe the TEP decoding process by a set of differential equations that represents the expected residual graph evolution as a function of the code parameters. The solution of these equations is used to predict the TEP decoder performance in both the asymptotic regime and the finite-length regime over the BEC. While the asymptotic threshold of the TEP decoder is the same as the BP decoder for regular and optimized codes, we propose a scaling law (SL) for finite-length LDPC codes, which accurately approximates the TEP improved performance and facilitates its optimization.
    IEEE Transactions on Information Theory 01/2012; 59(6). DOI:10.1109/TIT.2013.2245494 · 2.33 Impact Factor

Publication Stats

1k Citations
127.94 Total Impact Points


  • 2001-2015
    • University Carlos III de Madrid
      • • Department of Signal Theory and Communications
      • • Department of Electrical Engineering
      Getafe, Madrid, Spain
  • 2007-2010
    • Princeton University
      • Department of Electrical Engineering
      Princeton, New Jersey, United States
  • 2004-2006
    • University College London
      • Gatsby Computational Neuroscience Unit
      Londinium, England, United Kingdom
  • 2000-2001
    • University of Alcalá
      • Departamento de Teoría de la Señal y Comunicaciones
      Alcalá de Henares, Madrid, Spain
  • 1998
    • University of Vigo
      Vigo, Galicia, Spain