Yuesheng Xu

Old Dominion University | ODU · Department of Mathematics and Statistics

We consider deep neural networks with a Lipschitz continuous activation function and with weight matrices of variable widths. We establish a uniform convergence analysis framework in which sufficient conditions on weight matrices and bias vectors together with the Lipschitz constant are provided to ensure uniform convergence of the deep neural netw...

Sparsity of a learning solution is a desirable feature in machine learning. Certain reproducing kernel Banach spaces (RKBSs) are appropriate hypothesis spaces for sparse learning methods. The goal of this paper is to understand what kind of RKBSs can promote sparsity for learning solutions. We consider two typical learning models in an RKBS: the mi...

This paper introduces a successive affine learning (SAL) model for constructing deep neural networks (DNNs). Traditionally, a DNN is built by solving a non-convex optimization problem. It is often challenging to solve such a problem numerically due to its non-convexity and having a large number of layers. To address this challenge, inspired by the...

L^1$ based optimization is widely used in image denoising, machine learning and related applications. One of the main features of such approach is that it naturally provide a sparse structure in the numerical solutions. In this paper, we study an $L^1$ based mixed DG method for second-order elliptic equations in the non-divergence form. The ellipti...

The aim of this expository paper is to explain to graduate students and beginning researchers in the field of mathematics, statistics and engineering the fundamental concept of sparse machine learning in Banach spaces. In particular, we use binary classification as an example to explain the essence of learning in a reproducing kernel Hilbert space...

The current deep learning model is of a single-grade, that is, it learns a deep neural network by solving a single nonconvex optimization problem. When the layer number of the neural network is large, it is computationally challenging to carry out such a task efficiently. Inspired by the human education process which arranges learning in grades, we...

We consider a regularization problem whose objective function consists of a convex fidelity term and a regularization term determined by the l 1 norm composed with a linear transform. Empirical results show that the regularization with the l 1 norm can promote sparsity of a regularized solution. The goal of this paper is to understand theoretically...

Electrocardiogram (EMG) signals play a significant role in decoding muscle contraction information for robotic hand prosthesis controllers. Widely applied decoders require large amount of EMG signals sensors, resulting in complicated calculations and unsatisfactory predictions. By the biomechanical process of single degree-of-freedom human hand mov...

We study the use of deep learning techniques to reconstruct the kinematics of the neutral current deep inelastic scattering (DIS) process in electron–proton collisions. In particular, we use simulated data from the ZEUS experiment at the HERA accelerator facility, and train deep neural networks to reconstruct the kinematic variables $$Q^2$$ Q 2 and...

Purpose: Synthetic digital mammogram (SDM) is a 2D image generated from digital breast tomosynthesis (DBT) and used as a substitute for a full-field digital mammogram (FFDM) to reduce the radiation dose for breast cancer screening. The previous deep learning-based method used FFDM images as the ground truth, and trained a single neural network to...

We estimate convergence rates for fixed-point iterations of a class of nonlinear operators which are partially motivated by convex optimization problems. We introduce the notion of the generalized averaged nonexpansive (GAN) operator with a positive exponent, and provide convergence rate analysis of the fixed-point iteration of the GAN operator. Th...

We propose a sparse regularization model for inversion of incomplete Fourier transforms and apply it to seismic wavefield modeling. The objective function of the proposed model employs the Moreau envelope of the \(\ell _0\) norm under a tight framelet system as a regularization to promote sparsity. This model leads to a non-smooth, non-convex optim...

More competent learning models are demanded for data processing due to increasingly greater amounts of data available in applications. Data that we encounter often have certain embedded sparsity structures. That is, if they are represented in an appropriate basis, their energies can concentrate on a small number of basis functions. This paper is de...

We consider a minimization problem whose objective function is the sum of a fidelity term, not necessarily convex, and a regularization term defined by a positive regularization parameter [Formula: see text] multiple of the [Formula: see text] norm composed with a linear transform. This problem has wide applications in compressed sensing, sparse ma...

Convergence of deep neural networks as the depth of the networks tends to infinity is fundamental in building the mathematical foundation for deep learning. In a previous study, we investigated this question for deep networks with the Rectified Linear Unit (ReLU) activation function and with a fixed width. This does not cover the important convolut...

We propose a sparse regularization model for inversion of incomplete Fourier transforms and apply it to seismic wavefield modeling. The objective function of the proposed model employs the Moreau envelope of the $\ell_0$ norm under a tight framelet system as a regularization to promote sparsity. This model leads to a non-smooth, non-convex optimiza...

We investigated the imaging performance of a fast convergent ordered-subsets algorithm with subiteration-dependent preconditioners (SDPs) for positron emission tomography (PET) image reconstruction. In particular, we considered the use of SDP with the block sequential regularized expectation maximization (BSREM) approach with the relative differenc...

We consider a regularization problem whose objective function consists of a convex fidelity term and a regularization term determined by the $\ell_1$ norm composed with a linear transform. Empirical results show that the regularization with the $\ell_1$ norm can promote sparsity of a regularized solution. It is the goal of this paper to understand...

Deep neural networks, as a powerful system to represent high dimensional complex functions, play a key role in deep learning. Convergence of deep neural networks is a fundamental issue in building the mathematical foundation for deep learning. We investigated the convergence of deep ReLU networks and deep convolutional neural networks in two recent...

Sample entropy, an approximation of the Kolmogorov entropy, was proposed to characterize complexity of a time series, which is essentially defined as −log(B/A), where B denotes the number of matched template pairs with length m and A denotes the number of matched template pairs with m+1, for a predetermined positive integer m. It has been widely us...

Positron emission tomography (PET) is traditionally modeled as discrete systems. Such models may be viewed as piecewise constant approximations of the underlying continuous model for the physical processes and geometry of the PET imaging. Due to the low accuracy of piecewise constant approximations, discrete models introduce an irreducible modeling...

The goal of this study is to develop a new computed tomography (CT) image reconstruction method, aiming at improving the quality of the reconstructed images of existing methods while reducing computational costs. Existing CT reconstruction is modeled by pixel-based piecewise constant approximations of the integral equation that describes the CT pro...

We consider a minimization problem whose objective function is the sum of a fidelity term, not necessarily convex, and a regularization term defined by a positive regularization parameter $\lambda$ multiple of the $\ell_0$ norm composed with a linear transform. This problem has wide applications in compressed sensing, sparse machine learning and im...

We investigated the imaging performance of a fast convergent ordered-subsets algorithm with subiteration-dependent preconditioners (SDPs) for positron emission tomography (PET) image reconstruction. In particular, we considered the use of SDP with the block sequential regularized expectation maximization (BSREM) approach with the relative differenc...

Convergence of deep neural networks as the depth of the networks tends to infinity is fundamental in building the mathematical foundation for deep learning. In a previous study, we investigated this question for deep ReLU networks with a fixed width. This does not cover the important convolutional neural networks where the widths are increasing fro...

We study the use of deep learning techniques to reconstruct the kinematics of the deep inelastic scattering (DIS) process in electron-proton collisions. In particular, we use simulated data from the ZEUS experiment at the HERA accelerator facility, and train deep neural networks to reconstruct the kinematic variables $Q^2$ and $x$. Our approach is...

We estimate convergence rates for fixed-point iterations of a class of nonlinear operators which are partially motivated from solving convex optimization problems. We introduce the notion of the generalized averaged nonexpansive (GAN) operator with a positive exponent, and provide a convergence rate analysis of the fixed-point iteration of the GAN...

We explore convergence of deep neural networks with the popular ReLU activation function, as the depth of the networks tends to infinity. To this end, we introduce the notion of activation domains and activation matrices of a ReLU network. By replacing applications of the ReLU activation function by multiplications with activation matrices on activ...

This article presents a new primal-dual weak Galerkin method for second order elliptic equations in non-divergence form. The new method is devised as a constrained $L^p$-optimization problem with constraints that mimic the second order elliptic equation by using the discrete weak Hessian locally on each element. An equivalent min-max characterizati...

Synthetic digital mammography (SDM), a 2D image generated from digital breast tomosynthesis (DBT), is used as a potential substitute for full-field digital mammography (FFDM) in clinic to reduce the radiation dose for breast cancer screening. Previous studies exploited projection geometry and fused projection data and DBT volume, with different pos...

Seismic wavefield modeling is an important tool for the seismic interpretation. We consider modeling the wavefield in the frequency domain. This requires to solve a sequence of Helmholtz equations of wave numbers governed by the Nyquist sampling theorem. Inevitably, we have to solve Helmholtz equations of large wave numbers, which is a challenging...

We consider solving a system of semi-discrete first kind integral equations with a right-hand-side being a finite dimensional vector of sampling values and propose a regularization method for the system in a functional reproducing kernel Hilbert space (FRKHS), where the linear functionals that define the semi-discrete integral operator are continuo...

Constructing or learning a function from a finite number of sampled data points (measurements) is a fundamental problem in science and engineering. This is often formulated as a minimum norm interpolation problem, regularized learning problem or, in general, a semi-discrete inverse problem, in certain functional spaces. The choice of an appropriate...

We consider the minimum norm interpolation problem in the [Formula: see text] space, aiming at constructing a sparse interpolation solution. The original problem is reformulated in the pre-dual space, thereby inducing a norm in a related finite-dimensional Euclidean space. The dual problem is then transformed into a linear programming problem, whic...

The goal of this paper is to develop a novel numerical method for efficient multiplicative noise removal. The nonlocal self-similarity of natural images implies that the matrices formed by their nonlocal similar patches are low-rank. By exploiting this low-rank prior with application to multiplicative noise removal, we propose a nonlocal low-rank m...

Existing reconstruction methods for single photon emission computed tomography (SPECT) are most based on discrete models, leading to low accuracy in reconstruction. Reconstruction methods based on integral equation models (IEMs) with a higher order piecewise polynomial discretization on the pixel grid for SEPCT imaging were recently proposed to ove...

Our goal in the paper is to address the following problem: From an unknown matrix, we are given inner products of that matrix with a set of prescribed matrices, and wish to find the unknown matrix. We shall consider this problem by using the notion of minimal norm interpolation. A fixed-point proximity algorithm for solving this problem will be dev...

We develop a numerical method for construction of an adaptive display image from a given display image which is an artificial scene displayed in a computer screen. The adaptive display image is encoded on an adaptive pixel mesh obtained by a merging scheme from the original pixel mesh. The cardinality of the adaptive pixel mesh is significantly les...

Sparse learning models are popular in many application areas. Objective functions in sparse learning models are usually non-smooth, which makes it difficult to solve them numerically. We develop a fast and convergent two-step iteration scheme for solving a class of non-differentiable optimization models motivated from sparse learning. To overcome t...

Compared to mammographic screening, digital breast tomosynthesis (DBT) as an adjunct to full-field digital mammography (FFDM) so called combo-mode has been shown to improve sensitivity and reduce false positive rates in breast cancer detection. However, combo-mode screening increases the radiation dose to the patient. In this study, our purpose is...

The low rank matrix completion problem which aims to recover a matrix from that having missing entries has received much attention in many fields such as image processing and machine learning. The rank of a matrix may be measured by the [Formula: see text] norm of the vector of its singular values. Due to the nonconvexity and discontinuity of the [...

Our aim was to enhance visual quality and quantitative accuracy of dynamic positron emission tomography (PET)uptake images by improved image reconstruction, using sophisticated sparse penalty models that incorporate both 2D spatial+1D temporal (3DT) information. We developed two new 3DT PET reconstruction algorithms, incorporating different tempora...

The purpose of this research is to develop an advanced reconstruction method for low-count, hence high-noise, Single-Photon Emission Computed Tomography (SPECT) image reconstruction. It consists of a novel reconstruction model to suppress noise while conducting reconstruction and an efficient algorithm to solve the model. A novel regularizer is int...

We develop efficient numerical integration methods for computing an integral whose integrand is a product of a smooth function and the Gaussian function with a small standard deviation. Traditional numerical integration methods applied to the integral normally lead to poor accuracy due to the rapid change in high order derivatives of its integrand...

This article studies constructions of reproducing kernel Banach spaces (RKBSs) which may be viewed as a generalization of reproducing kernel Hilbert spaces (RKHSs). A key point is to endow Banach spaces with reproducing kernels such that machine learning in RKBSs can be well-posed and of easy implementation. First we verify many advanced properties...

This paper presents a preconditioned Krasnoselskii-Mann (KM) algorithm with an improved EM preconditioner (IEM-PKMA) for higher-order total variation (HOTV) regularized positron emission tomography (PET) image reconstruction. The PET reconstruction problem can be formulated as a threeterm convex optimization model consisting of the Kullback-Leibler...

Existing single-photon emission computed tomography (SPECT) reconstruction methods are mostly based on discrete models that may be viewed as piecewise constant approximations of a continuous data acquisition process. Due to low accuracy order of piecewise constant approximations, a traditional discrete model introduces irreducible model errors whic...

We develop a fully discrete fast Fourier–Galerkin method for solving a boundary integral equation for the biharmonic equation by introducing a quadrature scheme for computing the integrals of non-smooth functions that appear in the Fourier–Galerkin method. A key step in developing the fully discrete fast Fourier–Galerkin method is the design of a f...

We present a novel approach for support vector machines in reproducing kernel Banach spaces induced by a finite basis. In particular, we show that the support vector classification in the 1-norm reproducing kernel Banach space is mathematically equivalent to the sparse support vector machine. Finally, we develop fixed-point proximity algorithms for...

We introduce an optimization model of the support vector regression with the group lasso regularization and develop a class of efficient two-step fixed-point proximity algorithms to solve it numerically. To overcome the difficulty brought by the non-differentiability of the group lasso regularization term and the loss function in the proposed model...

We propose an approximation model of the original ℓ0 minimization model arising from various sparse signal recovery problems. The objective function of the proposed model uses the Moreau envelope of the ℓ0 norm to promote the sparsity of the signal in a tight framelet system . This leads to a non-convex optimization problem involved the ℓ0 norm. We...

The goal of this paper is to construct an efficient numerical algorithm for computing the coefficient matrix and the right hand side of the linear system resulting from the spectral Galerkin approximation of a stochastic elliptic partial differential equation. We establish that the proposed algorithm achieves an exponential convergence with requiri...

Restoration of images contaminated by multiplicative noise (also known as speckle noise) is a key issue in coherent image processing. Notice that images under consideration are often highly compressible in certain suitably chosen transform domains. By exploring this intrinsic feature embedded in images, this paper introduces a variational restorati...

Motivated by the need of processing non-point-evaluation functional data, we introduce the notion of functional reproducing kernel Hilbert spaces (FRKHSs). This space admits a unique functional reproducing kernel which reproduces a family of continuous linear functionals on the space. The theory of FRKHSs and the associated functional reproducing k...

Purpose The authors recently developed a preconditioned alternating projection algorithm (PAPA) for solving the penalized-likelihood SPECT reconstruction problem. The proposed algorithm can solve a wide variety of non-differentiable optimization models. This work is dedicated to comparing the performance of PAPA with total variation (TV) regulariza...

This paper presents a second-order hybrid finite volume method for solving the Stokes equation on a two dimensional domain. The trial function space of the method for velocity is chosen to be a quadratic conforming finite element space with a hierarchical decomposition technique on triangular meshes, and its corresponding test function space consis...

Multi-block separable convex problems recently received considerable attention. This class of optimization problems minimizes a separable convex objective function with linear constraints. The algorithmic challenges come from the fact that the classic alternating direction method of multipliers (ADMM) for the problem is not necessarily convergent....

Motivation: The sequence alignment is a fundamental problem in bioinformatics. BLAST is a routinely used tool for this purpose with over 118 000 citations in the past two decades. As the size of bio-sequence databases grows exponentially, the computational speed of alignment softwares must be improved. Results: We develop the heterogeneous BLAST...

High-resolution image reconstruction obtains one high-resolution image from multiple low-resolution, shifted, degraded samples of a true scene. This is a typical ill-posed problem and optimization models such as the ℓ²/TV model are previously studied for solving this problem. It is based on the assumption that during acquisition digital images are...

The purpose of this paper is to investigate the ability of the infimal convolution regularization in curing the staircasing artifacts of the TV model in the SPECT reconstruction. We formulate the problem of SPECT reconstruction with the infimal convolution regularization as a convex three-block optimization problem and characterize its solution by...

We consider minimization of functions that are compositions of functions having closed-form proximity operators with linear transforms. A wide range of image processing problems including image deblurring can be formulated in this way. We develop proximity algorithms based on the fixed point characterization of the solution to the minimization prob...

We propose a collocation method for solv- ing integral equations which model image restoration from out-of-focus images. Restoration of images from out-of-focus images can be formulated as an integral equation of the first kind, which is an ill-posed problem. We employ the Tikhonov regularization to treat the ill-posedness and obtain results of a w...

Motivated by the need of processing functional-valued data, or more general, operatorvalued data, we introduce the notion of the operator reproducing kernel Hilbert space (ORKHS). This space admits a unique operator reproducing kernel which reproduces a family of continuous linear operators on the space. The theory of ORKHSs and the associated oper...

We study the oscillatory structures of solutions of Volterra integral and integro-differential equations (VIEs, VIDEs) with highly oscillatory kernels. Based on the structured oscillatory spaces introduced in Wang and Xu [28], we first analyze the degree of oscillation of the solution of VIEs associated with the oscillatory kernels belonging to a c...

An accurate and efficient algorithm for solving the constrained ℓ 1-norm minimization problem is highly needed and is crucial for the success of sparse signal recovery in compressive sampling. We tackle the constrained ℓ 1-norm minimization problem by reformulating it via an indicator function which describes the constraints. The resulting model is...

We propose a variational model for restoration of images corrupted by multiplicative noise. The proposed model formulated in the logarithm transform domain of the desirable images consists of a data fitting term, a quadratic term, and a total variation regularizer. The data fitting term results directly from the presence of the multiplicative noise...

Sequence alignment is a long standing problem in bioinformatics. The Basic Local Alignment Search Tool (BLAST) is one of the most popular and fundamental alignment tools. The explosive growth of biological sequences calls for speedup of sequence alignment tools such as BLAST. To this end, we develop high speed BLASTN (HS-BLASTN), a parallel and fas...

Purpose: The authors have recently developed a preconditioned alternating projection algorithm (PAPA) with total variation (TV) regularizer for solving the penalized-likelihood optimization model for single-photon emission computed tomography (SPECT) reconstruction. This algorithm belongs to a novel class of fixed-point proximity methods. The goal...

We develop two classes of composite moment-free numerical quadratures for computing highly oscillatory integrals having integrable singularities and stationary points. The first class of the quadrature rules has a polynomial order of convergence and the second class has an exponential order of convergence. We first modify the moment-free Filon-type...

In this paper, we develop composite moment-free numerical quadratures for computing highly os- cillatory integrals with singularities and stationary points. The composite quadrature rules for computing highly oscillatory integrals with a smooth integrand and without a stationary point are developed based on partitioning the integration domain accor...

Wavelet transforms have been successfully applied in many fields of image processing. Yet, to our knowledge, they have never been directly incorporated to the objective function in Emission Computed Tomography (ECT) image reconstruction. Our aim has been to investigate if the 1-norm of non-decimated discrete cosine transform (DCT) coefficients of t...

We propose a constrained inpainting model to recover an image from its incomplete and/or inaccurate wavelet coefficients. The objective functional of the proposed model uses the norm to promote the sparsity of the resulting image in a tight framelet system. To overcome the algorithmic difficulty caused by the use of the norm, we approximate the nor...

The compressive sensing (CS) method allows us to recover a sparse signal from a small number of its linear measurements relative to the dimension of the signal space. The classic CS method assumes the measurements to have infinite bit precisions, which cannot be satisfied in practice. Quantization of the measurements is a crucial issue in CS. An ex...

We propose a fast discrete Fourier transform for a given data set which may be generated from sampling a function of d-variables on a sparse grid and a fast discrete backward Fourier transform on a hyperbolic cross index set. Computation of these transforms can be formulated as evaluation of dimension-reducible sums on sparse grids. We introduce a...

The main purpose of this paper is to study the construction of higher-order finite volume methods (FVMs) of triangle meshes. We investigate the relationship of the three theoretical notions crucial in the construction of FVMs: the uniform ellipticity of the family of its discrete bilinear forms, its inf–sup condition and its uniform local elliptici...

We develop efficient algorithms for solving the compressed sensing problem. We modify the standard ℓ1 regularization model for compressed sensing by adding a quadratic term to its objective function so that the objective function of the dual formulation of the modified model is Lipschitz continuous. In this way, we can apply the well-known Nesterov...

We provide a method for the construction of higher-order finite volume methods (FVMs) for solving boundary value problems of the two dimensional elliptic equations. Specifically, when the trial space of the FVM is chosen to be a conforming triangle mesh finite element space, we describe a construction of the associated test space that guarantees th...

Existing sparse inpainting models often suffer from their over-constraints on the sparsity of the transformed recovered images. Due to the fact that a transformed image of a wavelet or framelet transform is not truly sparse, but approximately sparse, we introduce an approximate sparsity model for inpainting. We formulate the model as minimizing the...

This paper discusses some mat hematical issues related to empirical mode decom-position (EMD). A B-spline EMD algorithm is introduced and developed for the convenience of mathematical studies. The numerical analysis using both simulated and practical signals and application examples from vibration analysis indicate that the B-spline algorithm has a...

Purpose: To investigate the performance of a new penalized-likelihood PET image reconstruction algorithm using the 11-norm total-variation (TV) sum of the 1st through 4th-order gradients as the penalty. Simulated and brain patient data sets were analyzed. Methods: This work represents an extension of the preconditioned alternating projection algori...

We study the orthogonal polynomial expansion on sparse grids for a function of dd variables in a weighted L2L2 space. Two Fast algorithms are developed for computing the orthogonal polynomial expansion and evaluating a linear combination of orthogonal polynomials on sparse grids by combining the fast cosine transform, the fast transforms between th...

We introduce in this paper a class of multi-step fixed-point proximity algorithms for solving optimization problems in the context of image processing. The objective functions of such optimization problems are the sum of two convex functions having one composed with an affine transformation which is often the regularization term. We are particularl...

This paper presents the Moreau envelope viewpoint for the L1/TV image denoising model. The main algorithmic difficulty for the numerical treatment of the L1/TV model lies in the non-differentiability of both the fidelity and regularization terms of the model. To overcome this difficulty, we propose five modified L1/TV models by replacing one or two...

This paper presents the Moreau envelope viewpoint for the L1/TV image denoising model. The main algorithmic difficulty for the numerical treatment of the L1/TV model lies in the non-differentiability of both the fidelity and regularization terms of the model. To overcome this difficulty, we propose five modified L1/TV models by replacing one or two...

We develop a fast Fourier-Galerkin method for solving a boundary integral equation which is a reformulation of the Dirichlet problem of the biharmonic equation. The proposed method is based on a splitting of the resulting boundary integral operator. That is, we write the operator as a sum of two integral operators, one having the Fourier basis func...