Illustration of maxout layer upper bound

Data · February 2018with33 Reads
All content in this shaded area was uploaded by Guido Montufar on Feb 09, 2018

1 File
PDF · 149.26 KB · Available from Guido Montufar

All content in this shaded area was uploaded by Guido Montufar on Feb 09, 2018
This research doesn't cite any other publications.

Recommended publications

March 2015
    We establish upper bounds for the minimal number of hidden units for which a binary stochastic feedforward network with sigmoid activation probabilities and a single hidden layer is a universal approximator of Markov kernels. We show that each possible probabilistic assignment of the states of $n$ output units, given the states of $k\geq1$ input units, can be approximated arbitrarily well by a... [Show full abstract]
    Conference Paper
    May 2015
      We show that deep narrow Boltzmann machines are universal approximators of probability distributions on the activities of their visible units, provided they have sufficiently many hidden layers, each containing the same number of units as the visible layer. We show that, within certain parameter domains, deep Boltzmann machines can be studied as feedforward networks. We provide upper and lower... [Show full abstract]
      March 2013 · Neural Computation
        We generalize recent theoretical work on the minimal number of layers of narrow deep belief networks that can approximate any probability distribution on the states of their visible units arbitrarily well, from the setting of binary units (Sutskever and Hinton, 2008; Le Roux and Bengio, 2008, 2010; Montúfar and Ay, 2011) to the setting of units with finite state spaces. In particular we show... [Show full abstract]
        Conference Paper
        March 2017
          We follow up on previous work addressing the number of response regions of the functions representable by feedforward neural networks with piecewise linear activation functions. We discuss upper bounds on the maximum number of linear regions for deep networks with rectified linear units. We elaborate on the identification of input regions as an analysis tool, and how it implies exponential... [Show full abstract]
          Discover more