[Show abstract][Hide abstract] ABSTRACT: Bridging cultures that have often been distant, Julia combines expertise from the diverse fields of computer science and computational science to create a new approach to numerical computing. Julia is designed to be easy and fast. Julia questions notions generally held as “laws of nature” by practitioners of numerical computing:
1. High-level dynamic programs have to be slow,
2. One must prototype in one language and then rewrite in another language for speed or deploy- ment, and
3. There are parts of a system for the programmer, and other parts best left untouched as they are built by the experts.
We introduce the Julia programming language and its design — a dance between specialization and abstraction. Specialization allows for custom treatment. Multiple dispatch, a technique from computer science, picks the right algorithm for the right circumstance. Abstraction, what good computation is really about, recognizes what remains the same after differences are stripped away. Abstractions in mathematics are captured as code through another technique from computer science, generic programming.
Julia shows that one can have machine performance without sacrificing human convenience.
[Show abstract][Hide abstract] ABSTRACT: We introduce a method that uses the Cauchy-Crofton formula and a new
curvature formula from integral geometry to reweight the sampling probabilities
of Metropolis-within-Gibbs algorithms in order to increase their convergence
speed. We consider algorithms that sample from a probability density
conditioned on a manifold $\mathcal{M}$. Our method exploits the symmetries of
the algorithms' isotropic random search-direction subspaces to analytically
average out the variance in the intersection volume caused by the orientation
of the search-subspace with respect to the manifold $\mathcal{M}$ it
intersects. This variance can grow exponentially with the dimension of the
search-subspace, greatly slowing down the algorithm. Eliminating this variance
allows us to use search-subspaces of dimensions many times greater than would
otherwise be possible, allowing us to sample very rare events that a
lower-dimensional search-subspace would be unlikely to intersect.
To extend this method to events that are rare for reasons other than their
support $\mathcal{M}$ having a lower dimension, we formulate and prove a new
theorem in integral geometry that makes use of the curvature form of the
Chern-Gauss-Bonnet theorem to reweight sampling probabilities. On the side, we
also apply our theorem to obtain new theoretical bounds for the volumes of real
algebraic manifolds.
Finally, we demonstrate the computational effectiveness and speedup of our
method by numerically applying it to the conditional stochastic Airy operator
sampling problem in random matrix theory.
[Show abstract][Hide abstract] ABSTRACT: The four major asymptotic level density laws of random matrix theory may all
be showcased though their Jacobi parameter representation as having a bordered
Toeplitz form. We compare and contrast these laws, completing and exploring
their representations in one place. Inspired by the bordered Toeplitz form, we
propose an algorithm for the finite moment problem by proposing a solution
whose density has a bordered Toeplitz form.
Linear Algebra and its Applications 02/2015; 467. DOI:10.1016/j.laa.2014.11.006 · 0.94 Impact Factor
[Show abstract][Hide abstract] ABSTRACT: What is the probability that a random triangle is acute? We explore this old
question from a modern viewpoint, taking into account linear algebra, shape
theory, numerical analysis, random matrix theory, the Hopf fibration, and much
much more. One of the best distributions of random triangles takes all six
vertex coordinates as independent standard Gaussians. Six can be reduced to
four by translation of the center to $(0,0)$ or reformulation as a 2x2 matrix
problem.
In this note, we develop shape theory in its historical context for a wide
audience. We hope to encourage other to look again (and differently) at
triangles.
We provide a new constructive proof, using the geometry of parallelians, of a
central result of shape theory: Triangle shapes naturally fall on a hemisphere.
We give several proofs of the key random result: that triangles are uniformly
distributed when the normal distribution is transferred to the hemisphere. A
new proof connects to the distribution of random condition numbers.
Generalizing to higher dimensions, we obtain the "square root ellipticity
statistic" of random matrix theory.
Another proof connects the Hopf map to the SVD of 2 by 2 matrices. A new
theorem describes three similar triangles hidden in the hemisphere. Many
triangle properties are reformulated as matrix theorems, providing insight to
both. This paper argues for a shift of viewpoint to the modern approaches of
random matrix theory. As one example, we propose that the smallest singular
value is an effective test for uniformity. New software is developed and
applications are proposed.
[Show abstract][Hide abstract] ABSTRACT: Some properties that nominally involve the eigenvalues of Gaussian Unitary
Ensemble (GUE) can instead be phrased in terms of singular values. By
discarding the signs of the eigenvalues, we gain access to a surprising
decomposition: the singular values of the GUE are distributed as the union of
the singular values of two independent ensembles of Laguerre type. This
independence is remarkable given the well known phenomenon of eigenvalue
repulsion.
The structure of this decomposition reveals that several existing
observations about large $n$ limits of the GUE are in fact manifestations of
phenomena that are already present for finite random matrices. We relate the
semicircle law to the quarter-circle law by connecting Hermite polynomials to
generalized Laguerre polynomials with parameter $\pm$1/2. Similarly, we write
the absolute value of the determinant of the $n\times{}n$ GUE as a product n
independent random variables to gain new insight into its asymptotic
log-normality. The decomposition also provides a description of the
distribution of the smallest singular value of the GUE, which in turn permits
the study of the leading order behavior of the condition number of GUE
matrices.
The study is motivated by questions involving the enumeration of orientable
maps, and is related to questions involving powers of complex Ginibre matrices.
The inescapable conclusion of this work is that the singular values of the GUE
play an unpredictably important role that had gone unnoticed for decades even
though, in hindsight, so many clues had been around.
[Show abstract][Hide abstract] ABSTRACT: Polymorphism in programming languages enables code reuse. Here, we show that
polymorphism has broad applicability far beyond computations for technical
computing: parallelism in distributed computing, presentation of visualizations
of runtime data flow, and proofs for formal verification of correctness. The
ability to reuse a single codebase for all these purposes provides new ways to
understand and verify parallel programs.
[Show abstract][Hide abstract] ABSTRACT: Arrays are such a rich and fundamental data type that they tend to be built into a language, either in the compiler or in a large low-level library. Defining this functionality at the user level instead provides greater flexibility for application domains not envisioned by the language designer. Only a few languages, such as C++ and Haskell, provide the necessary power to define n-dimensional arrays, but these systems rely on compile-time abstraction, sacrificing some flexibility. In contrast, dynamic languages make it straightforward for the user to define any behavior they might want, but at the possible expense of performance.
As part of the Julia language project, we have developed an approach that yields a novel trade-off between flexibility and compile-time analysis. The core abstraction we use is multiple dispatch. We have come to believe that while multiple dispatch has not been especially popular in most kinds of programming, technical computing is its killer application. By expressing key functions such as array indexing using multi-method signatures, a surprising range of behaviors can be obtained, in a way that is both relatively easy to write and amenable to compiler analysis. The compact factoring of concerns provided by these methods makes it easier for user-defined types to behave consistently with types in the standard library.
[Show abstract][Hide abstract] ABSTRACT: We derive explicit expressions for the distributions of the extreme eigenvalues of the beta-Wishart random matrices in terms of the hypergeometric function of a matrix argument. These results generalize the classical results for the real (β = 1), complex (β = 2), and quaternion (β = 4) Wishart matrices to any β > 0.
[Show abstract][Hide abstract] ABSTRACT: We find the joint generalized singular value distribution and largest
generalized singular value distributions of the $\beta$-MANOVA ensemble with
positive diagonal covariance, which is general. This has been done for the
continuous $\beta > 0$ case for identity covariance (in eigenvalue form), and
by setting the covariance to $I$ in our model we get another version. For the
diagonal covariance case, it has only been done for $\beta = 1,2,4$ cases
(real, complex, and quaternion matrix entries). This is in a way the first
second-order $\beta$-ensemble, since the sampler for the generalized singular
values of the $\beta$-MANOVA with diagonal covariance calls the sampler for the
eigenvalues of the $\beta$-Wishart with diagonal covariance of Forrester and
Dubbs-Edelman-Koev-Venkataramana. We use a conjecture of MacDonald proven by
Baker and Forrester concerning an integral of a hypergeometric function and a
theorem of Kaneko concerning an integral of Jack polynomials to derive our
generalized singular value distributions. In addition we use many identities
from Forrester's {\it Log-Gases and Random Matrices}. We supply numerical
evidence that our theorems are correct.
[Show abstract][Hide abstract] ABSTRACT: We introduce a “broken-arrow” matrix model for the β-Wishart ensemble, which improves on the traditional bidiagonal model by generalizing to non-identity covariance parameters. We prove that its joint eigenvalue density involves the correct hypergeometric function of two matrix arguments, and a continuous parameter β > 0. If we choose β = 1, 2, 4, we recover the classical Wishart ensembles of general covariance over the reals, complexes, and quaternions. Jack polynomials are often defined as the eigenfunctions of the Laplace-Beltrami operator. We prove that Jack polynomials are in addition eigenfunctions of an integral operator defined as an average over a β-dependent measure on the sphere. When combined with an identity due to Stanley, we derive a definition of Jack polynomials. An efficient numerical algorithm is also presented for simulations. The algorithm makes use of secular equation software for broken arrow matrices currently unavailable in the popular technical computing languages. The simulations are matched against the cdfs for the extreme eigenvalues. The techniques here suggest that arrow and broken arrow matrices can play an important role in theoretical and computational random matrix theory including the study of corners processes. We provide a number of simulations illustrating the extreme eigenvalue distributions that are likely to be useful for applications. We also compare the n → ∞ answer for all β with the free-probability prediction.
[Show abstract][Hide abstract] ABSTRACT: A linear algebraic approach to graph algorithms that exploits the sparse adjacency matrix representation of graphs can provide a variety of benefits. These benefits include syntactic simplicity, easier implementation, and higher performance. One way to employ linear algebra techniques for graph algorithms is to use a broader definition of matrix and vector multiplication. We demonstrate through the use of the Julia language system how easy it is to explore semirings using linear algebraic methodologies.
High Performance Extreme Computing Conference (HPEC), 2013 IEEE; 01/2013
[Show abstract][Hide abstract] ABSTRACT: Dynamic languages have become popular for scientific computing. They are
generally considered highly productive, but lacking in performance. This paper
presents Julia, a new dynamic language for technical computing, designed for
performance from the beginning by adapting and extending modern programming
language techniques. A design based on generic functions and a rich type system
simultaneously enables an expressive programming model and successful type
inference, leading to good performance for a wide range of programs. This makes
it possible for much of the Julia library to be written in Julia itself, while
also incorporating best-of-breed C and Fortran libraries.
[Show abstract][Hide abstract] ABSTRACT: Theoretical studies of localization, anomalous diffusion and ergodicity breaking require solving the electronic structure of disordered systems. We use free probability to approximate the ensemble-averaged density of states without exact diagonalization. We present an error analysis that quantifies the accuracy using a generalized moment expansion, allowing us to distinguish between different approximations. We identify an approximation that is accurate to the eighth moment across all noise strengths, and contrast this with perturbation theory and isotropic entanglement theory.
[Show abstract][Hide abstract] ABSTRACT: We define an indefinite Wishart matrix as a matrix of the form
A=W^{T}W\Sigma, where \Sigma is an indefinite diagonal matrix and W is a matrix
of independent standard normals. We focus on the case where W is L by 2 which
has engineering applications. We obtain the distribution of the ratio of the
eigenvalues of A. This distribution can be "folded" to give the distribution of
the condition number. We calculate formulas for W real (\beta=1), complex
(\beta=2), quaternionic (\beta=4) or any ghost 0<\beta<\infty. We then
corroborate our work by comparing them against numerical experiments.
Linear Algebra and its Applications 07/2012; 483. DOI:10.1016/j.laa.2015.05.027 · 0.94 Impact Factor
[Show abstract][Hide abstract] ABSTRACT: We investigate the implications of free probability for random matrices. From
rules for calculating all possible joint moments of two free random matrices,
we develop a notion of partial freeness which is quantified by the breakdown of
these rules. We provide a combinatorial interpretation for partial freeness as
the presence of closed paths in Hilbert space defined by particular joint
moments. We also discuss how asymptotic moment expansions provide an error term
on the density of states. We present MATLAB code for the calculation of moments
and free cumulants of arbitrary random matrices.
[Show abstract][Hide abstract] ABSTRACT: We approximate the density of states in disordered systems by
decomposing the Hamiltonian into two random matrices and constructing
their free convolution. The error in this approximation is determined
using asymptotic moment expansions. Each moment can be decomposed into
contributions from specific joint moments of the random matrices; each
of which has a combinatorial interpretation as the weighted sum of
returning trajectories. We show how the error, like the free convolution
itself, can be calculated without explicit diagonalization of the
Hamiltonian. We apply our theory to Hamiltonians for one-dimensional
tight binding models with Gaussian and semicircular site disorder. We
find that the particular choice of decomposition crucially determines
the accuracy of the resultant density of states. From a partitioning of
the Hamiltonian into diagonal and off-diagonal components, free
convolution produces an approximate density of states which is correct
to the eighth moment. This allows us to explain the accuracy of mean
field theories such as the coherent potential approximation, as well as
the results of isotropic entanglement theory.
[Show abstract][Hide abstract] ABSTRACT: We propose a method that we call isotropic entanglement (IE), which predicts the eigenvalue distribution of quantum many body (spin) systems with generic interactions. We interpolate between two known approximations by matching fourth moments. Though such problems can be QMA-complete, our examples show that isotropic entanglement provides an accurate picture of the spectra well beyond what one expects from the first four moments alone. We further show that the interpolation is universal, i.e., independent of the choice of local terms.
[Show abstract][Hide abstract] ABSTRACT: Partitioning oracles were introduced by Hassidim et al. (FOCS 2009) as a generic tool for constant-time algorithms. For any ε > 0, a partitioning oracle provides query access to a fixed partition of the input bounded-degree minor-free graph, in which every component has size poly(1/ε), and the number of edges removed is at most εn, where n is the number of vertices in the graph.
However, the oracle of Hassidim et al. makes an exponential number of queries to the input graph to answer every query about the partition. In this paper, we construct an efficient partitioning oracle for graphs with constant treewidth. The oracle makes only O(poly(1/ε)) queries to the input graph to answer each query about the partition.
Examples of bounded-treewidth graph classes include k-outerplanar graphs for fixed k, series-parallel graphs, cactus graphs, and pseudoforests. Our oracle yields poly(1/ε)-time property testing algorithms for membership in these classes of graphs. Another application of the oracle is a poly(1/ε)-time algorithm that approximates the maximum matching size, the minimum vertex cover size, and the minimum dominating set size up to an additive εn in graphs with bounded treewidth. Finally, the oracle can be used to test in poly(1/ε) time whether the input bounded-treewidth graph is k-colorable or perfect.
[Show abstract][Hide abstract] ABSTRACT: Approximating ideal program outputs is a common technique for solving computationally difficult problems, for adhering to processing or timing constraints, and for performance optimization in situations where perfect precision is not necessary. To this end, programmers often use approximation algorithms, iterative methods, data resampling, and other heuristics. However, programming such variable accuracy algorithms presents difficult challenges since the optimal algorithms and parameters may change with different accuracy requirements and usage environments. This problem is further compounded when multiple variable accuracy algorithms are nested together due to the complex way that accuracy requirements can propagate across algorithms and because of the size of the set of allowable compositions. As a result, programmers often deal with this issue in an ad-hoc manner that can sometimes violate sound programming practices such as maintaining library abstractions. In this paper, we propose language extensions that expose trade-offs between time and accuracy to the compiler. The compiler performs fully automatic compile-time and installtime autotuning and analyses in order to construct optimized algorithms to achieve any given target accuracy. We present novel compiler techniques and a structured genetic tuning algorithm to search the space of candidate algorithms and accuracies in the presence of recursion and sub-calls to other variable accuracy code. These techniques benefit both the library writer, by providing an easy way to describe and search the parameter and algorithmic choice space, and the library user, by allowing high level specification of accuracy requirements which are then met automatically without the need for the user to understand any algorithm-specific parameters. Additionally, we present a new suite of benchmarks, written in our language, to examine the efficacy of our techniques. Our experimental results show that by relaxing accuracy requirements , we can easily obtain performance improvements ranging from 1.1× to orders of magnitude of speedup.
Proceedings of the CGO 2011, The 9th International Symposium on Code Generation and Optimization, Chamonix, France, April 2-6, 2011; 04/2011