Splines in Higher Order TV Regularization

International Journal of Computer Vision (Impact Factor: 3.81). 12/2006; 70(3):241-255. DOI: 10.1007/s11263-006-8066-7
Source: DBLP


Splines play an important role as solutions of various interpolation and approximation problems that minimize special functionals in some smoothness spaces. In this paper, we show in a strictly discrete setting that splines of degree m−1 solve also a minimization problem with quadratic data term and m-th order total variation (TV) regularization term. In contrast to problems with quadratic regularization terms involving m-th order derivatives, the spline knots are not known in advance but depend on the input data and the regularization parameter λ. More precisely, the spline knots are determined by the contact points of the m–th discrete antiderivative of the solution with the tube of width 2λ around the m-th discrete antiderivative of the input data. We point out that the dual formulation of our minimization problem can be considered as support vector regression problem in the discrete counterpart of the Sobolev space W
. From this point of view, the solution of our minimization problem has a sparse representation in terms of discrete fundamental splines.

Full-text preview

Available from:
  • Source
    • "The 1 trend filtering method is a special case of the generalized lasso method studied in [5]. It is also related to spline approximations [6] [7]. "
    [Show abstract] [Hide abstract]
    ABSTRACT: In this paper we study the estimation of changing trends in time-series using $\ell_1$ trend filtering. This method generalizes 1D Total Variation (TV) denoising for detection of step changes in means to detecting changes in trends, and it relies on a convex optimization problem for which there are very efficient numerical algorithms. It is known that TV denoising suffers from the so-called stair-case effect, which leads to detecting false change points. The objective of this paper is to show that $\ell_1$ trend filtering also suffers from a certain stair-case problem. The analysis is based on an interpretation of the dual variables of the optimization problem in the method as integrated random walk. We discuss consistency conditions for $\ell_1$ trend filtering, how to monitor their fulfillment, and how to modify the algorithm to avoid the stair-case false detection problem.
    Preview · Article · Dec 2014
  • Source
    • "Nonparametric regression has a rich history in statistics, carrying well over 50 years of associated literature . The goal of this paper is to port a successful idea in univariate nonparametric regression, trend filtering [28] [13] [30] [34], to the setting of estimation on graphs. The proposed estimator, graph trend filtering, shares three key properties of trend filtering in the univariate setting. "
    [Show abstract] [Hide abstract]
    ABSTRACT: We introduce a family of adaptive estimators on graphs, based on penalizing the $\ell_1$ norm of discrete graph differences. This generalizes the idea of trend filtering [Kim et al. (2009), Tibshirani (2014)], used for univariate nonparametric regression, to graphs. Analogous to the univariate case, graph trend filtering exhibits a level of local adaptivity unmatched by the usual $\ell_2$-based graph smoothers. It is also defined by a convex minimization problem that is readily solved (e.g., by fast ADMM or Newton algorithms). We demonstrate the merits of graph trend filtering through examples and theory.
    Preview · Article · Oct 2014
  • Source
    • "Trend filtering is a relatively new method for nonparametric regression, proposed independently by Steidl et al. (2006), Kim et al. (2009). Suppose that we are given output points y = (y 1 , . . . "
    [Show abstract] [Hide abstract]
    ABSTRACT: This paper focuses on computational approaches for trend filtering, using the alternating direction method of multipliers (ADMM). We argue that, under an appropriate parametrization, ADMM is highly efficient for trend filtering, competitive with the specialized interior point methods that are currently in use. Furthermore, the proposed ADMM implementation is very simple, and importantly, it is flexible enough to extend to many interesting related problems, such as sparse trend filtering and isotonic trend filtering. Software for our method will be made freely available, written in C++ (see \url{}), and also in R (see the {\tt trendfilter} function in the R package {\tt genlasso}).
    Preview · Article · Jun 2014 · Journal of Computational and Graphical Statistics
Show more