Chapter

Chapter

# On the Sensitivity of Least Squares Data Fitting by Nonnegative Second Divided Differences

If you want to read the PDF, try requesting it from the authors.

## Abstract

Let measurements of a real function of one variable be given. If the function is convex but convexity has been lost due to errors of measurement, then we make the least sum of squares change to the data so that the second divided differences of the smoothed values are nonnegative. The underlying calculation is a quadratic programming algorithm and the piecewise linear interpolant to the solution components is a convex curve. Problems of this structure arise in various contexts in research and applications in science, engineering and social sciences. The sensitivity of the solution is investigated when the data are slightly altered. The sensitivity question arises in the utilization of the method. First some theory is presented and then an illustrative example shows the effect of specific as well as random changes of the data to the solution. As an application to real data, an experiment on the sensitivity of the convex estimate to the Gini coefficient in the USA for the time period 1947–1996 is presented. The measurements of the Gini coefficient are considered uncertain, with a uniform probability distribution over a certain interval. Some consequences of this uncertainty are investigated with the aid of a simulation technique.

## No full-text available

Article
Full-text available
Let n measurements of a process be provided sequentially, where the process follows a sigmoid shape, but the data have lost sigmoidicity due to measuring errors. If we smooth the data by making least the sum of squares of errors subject to one sign change in the second divided differences, then we obtain a sigmoid approximation. It is known that the optimal fit of this calculation is composed of two separate sections, one best convex and one best concave. We propose a method that starts at the beginning of the data and proceeds systematically to construct the two sections of the fit for the current data, step by step as n is increased. Although the minimization calculation at each step may have many local minima, it can be solved in about O(n2) operations, because of properties of the join between the convex and the concave section. We apply this method to data of daily Covid-19 cases and deaths of Greece, the United States of America and the United Kingdom. These data provide substantial differences in the final approximations. Thus, we evaluate the performance of the method in terms of its capabilities as both constructing a sigmoid-type approximant to the data and a trend detector. Our results clarify the optimization calculation both in a systematic manner and to a good extent. At the same time, they reveal some features of the method to be considered in scenaria that may involve predictions, and as a tool to support policy-making. The results also expose some limitations of the method that may be useful to future research on convex-concave data fitting.
Article
If the data (x i ,y i )∈ℝ 2 , i=1,2,⋯,n, include substantial random errors, then the number of sign changes in the sequence of second differences y[x i-1 ,x i ,x i+1 ], i=2,3,⋯,n-1, may be unacceptably large, where we are assuming x 1 <x 2 <⋯<x n . Therefore we address the problem of calculating the fit ϕ i , i=1,2,⋯,n, that minimizes ∑ i=1 n (y i -ϕ i ) 2 , subject to the condition that there are at most k-1 sign changes in the sequence ϕ[x i-1 ,x i ,x i+1 ], i=2,3,⋯,n-1, where k is a prescribed positive integer. It is proved that there exists a partition of the data into k disjoint subsets, such that the required fit can be calculated by solving a convex quadratic programming problem on each subset. Further, the piecewise linear interpolant to the fit is either convex or concave on each partition, and there is alternation between the convex and concave partitions. Thus, about twelve years ago, the authors [IMA J. Numer. Anal. 11, No. 3, 433-448 (1991; Zbl 0726.65009)] developed some dynamic programming methods that calculate the required fit efficiently, assuming that, if the right hand end of a partition is available, then the left hand end can be calculated without using the data beyond the right hand end. The present paper, however, gives a counter-example to this conjecture. Then a new dynamic programming algorithm is proposed that does not depend on any assumptions, and whose total work should not exceed O(kn 3 ) for all values of k, but the work before was only about O(kn 2 ).
Article
A method is developed for obtaining maximum likelihood estimates of points on a surface of unspecified algebraic form when ordinates of the points are required to satisfy a set of linear inequalities. A production function with one variable input is considered in some detail. In this case the restrictions follow from the assumption of non-increasing returns. An illustrative computation is worked out using a procedure based on equivalence between the estimation problem and a certain saddle point problem. Alternative procedures for production functions with two variable inputs are sketched.
Article
Abstract Abstract: We study nonparametric estimation of convex regression and density functions by methods of least squares (in the regression and density cases) and maximum,likelihood (in the density estimation case). We provide characterizations of these estimators, prove that they are consistent, and establish their asymptotic distributions at a xed point of positive curvature of the functions estimated. The asymptotic distribution theory relies on the existence of a \invelope function" for integrated two-sided Brownian motion +tResearch supported in part by National Science Foundation grant DMS-95-32039 and NIAID grant 2R01
Article
A regression procedure has been developed to correlate scanning capacitance microscope (SCM) data with dopant concentration in three dimensions. The inverse problem (calculation of the dopant profile from SCM data) is formulated in two dimensions as a regularized nonlinear least-squares optimization problem. For each iteration of the regression procedure, Poisson’s equation is numerically solved within the quasistatic approximation. For a given type model ion-implanted dopant profile, two cases are considered; the background doping is either the same or the opposite type as that ion-implanted. Due to the long-range nature of the interactions in the sample, the regression is done using two spatial meshes: a coarse mesh and a dense mesh. The coarse mesh stepsize is of the order of the probe-tip size. The dense mesh stepsize is a fraction of the coarse mesh stepsize. The regression starts and proceeds with the coarse mesh until the spatial wavelength of the error or noise in the estimated dopant density profile is of the order of the coarse mesh stepsize. The regression then proceeds in like manner with the dense mesh. Regularization and filtering are found to be important to the convergence of the regression procedure.
Article
A Fortran subroutine applies the method of Demetriou and Powell [1991] to restore convexity in n measurements of a convex function contaminated by random errors. The method minimizes the sum of the squares of the errors, subject to nonnegativity of second divided differences, in two phases. First, an approximation close to the optimum is derived in O(n) operations. Then, this approximation is used as the starting point of a dual-feasible quadratic programming algorithm that completes the calculation of the optimum. The constraints allow B-splines to be used, which reduce the problem to an equivalent one with fewer variables where the knots of the splines are determined automatically from the data points due to the constraint equations. The subroutine benefits from this reduction, since common submatrices that occur during the calculation are updated suitably. Iterative refinement improves the accuracy of some calculations when round-off errors accumulate. The subroutine has been applied to a variety of data having substantial differences and has performed fast and stably even for small data spacing, large n, and single-precision arithmetic. Driver programs and examples with output are provided to demonstrate the use of the subroutine.
Article
An efficient and numerically stable dual algorithm for positive definite quadratic programming is described which takes advantage of the fact that the unconstrained minimum of the objective function can be used as a starting point. Its implementation utilizes the Cholesky and QR factorizations and procedures for updating them. The performance of the dual algorithm is compared against that of primal algorithms when used to solve randomly generated test problems and quadratic programs generated in the course of solving nonlinear programming problems by a successive quadratic programming code (the principal motivation for the development of the algorithm). These computational results indicate that the dual algorithm is superior to primal algorithms when a primal feasible point is not readily available. The algorithm is also compared theoretically to the modified-simplex type dual methods of Lemke and Van de Panne and Whinston and it is illustrated by a numerical example.
Article
Methods are presented for least squares data smoothing by using the signs of divided differences of the smoothed values. Professor M.J.D. Powell initiated the subject in the early 1980s and since then, theory, algorithms and FORTRAN software make it applicable to several disciplines in various ways.Let us consider n data measurements of a univariate function which have been altered by random errors. Then it is usual for the divided differences of the measurements to show sign alterations, which are probably due to data errors. We make the least sum of squares change to the measurements, by requiring the sequence of divided differences of order m to have at most q sign changes for some prescribed integer q. The positions of the sign changes are integer variables of the optimization calculation, which implies a combinatorial problem whose solution can require about O(nq) quadratic programming calculations in n variables and n−m constraints.Suitable methods have been developed for the following cases. It has been found that a dynamic programming procedure can calculate the global minimum for the important cases of piecewise monotonicity m=1,q⩾1 and piecewise convexity/concavity m=2,q⩾1 of the smoothed values. The complexity of the procedure in the case of m=1 is computer operations, while it is reduced to only O(n) when q=0 (monotonicity) and q=1 (increasing/decreasing monotonicity). The case m=2,q⩾1 requires O(qn2) computer operations and n2 quadratic programming calculations, which is reduced to one and n−2 quadratic programming calculations when m=2,q=0, i.e. convexity, and m=2,q=1, i.e. convexity/concavity, respectively.Unfortunately, the technique that receives this efficiency cannot generalize for the highly nonlinear case m⩾3,q⩾2. However, the case m⩾3,q=0 is solved by a special strictly convex quadratic programming calculation, and the case m⩾3,q=1 can be solved by at most 2(n−m) applications of the previous algorithm. Also, as m gets larger, large sets of active constraints usually occur at the optimal approximation, which makes the calculation for higher values of q less expensive than what it seems to. Further, attention is given to the sensitivity of the solution with respect to changes in the constraints and the data.The smoothing technique is an active research topic and there seems to be room for further developments. One strong reason for studying methods that make use of divided differences for data smoothing is that, whereas only the data are provided, the achieved accuracy goes much beyond the accuracy of the data at an order determined automatically by the chosen divided differences.
Article
For each $t$ in some subinterval $T$ of the real line let $F_t$ be a distribution function with mean $m(t)$. Suppose $m(t)$ is concave. Let $t_1, t_2, \cdots$ be a sequence of points in $T$ and let $Y_1, Y_2, \cdots$ be an independent sequence of random variables such that the distribution function of $Y_k$ is $F_{t_k}$. We consider estimators $m_n(t) = m_n(t; Y_1, \cdots, Y_n)$ which are concave in $t$ and which minimize $\sum^n_{i=1} \lbrack m_n(t_i; Y_1, \cdots, Y_n) - Y_i\rbrack^2$ over the class of concave functions. We investigate their consistency and the convergence of $\{m_n'(t)\}$ to $m'(t)$.
Article
Let n measurements of a real valued function of one variable be given. If the function is convex but the data have lost convexity due to the errors of the measuring process, then the least sum of squares change to the data that provides nonnegative second divided differences may be required. An algorithm is proposed for this highly structured quadratic programming calculation. First a procedure that requires only O ( n ) computer operations generates a starting point for the main calculation, and then a version of the iterative method of Goldfarb & Idnani (1983) is applied. It is proved that the algorithm converges, the analysis being a special case of the theory of Goldfarb & Idnani. The algorithm is efficient because the matrices that occur are banded due to representing the required fit as a linear combination of B-splines. Some numerical results illustrate the method. They suggest that the algorithm can be used when n is very large, because the O ( n ) starting procedure identifies most of the convexity constraints that are active at the solution.
Article
The problem of convexity runs deeply in economic theory. For example, increasing returns or upward slopes (convexity) and diminishing returns or downward slopes (concavity) of certain supply, demand, production and utility relations are often assumed in economics. Quite frequently, however, the observations have lost convexity (or concavity) due to errors of the measuring process. We derive the Karush- Kuhn-Tucker test statistic of convexity, when the convex estimator of the data minimizes the sum of squares of residuals subject to the assumption of non-decreasing returns. Testing convexity is a linear regression problem with linear inequality constraints on the regression coefficients, so generally the work of Gouriéroux, Holly and Monfort (1982) as well as Hartigan (1967) apply. Convex estimation is a highly structured quadratic programming calculation that is solved very efficiently by the Demetriou and Powell (1991) algorithm. Certain applications that test the convexity assumption of real economic data are considered, the results are briefly analyzed and the interpretation capability of the test is demonstrated. Some numerical results illustrate the computation and present the efficacy of the test in small, medium and large data sets. They suggest that the test is suitable when the number of observations is very large.
A Practical Guide to Splines, Revised Edition
• C De Boor