Code

Stochastic Gradient Descent (SGA) Algorithm - Code

February 2022

Authors:

Shakira Musa Baig

COMSATS University Islamabad

ResearchGate has not been able to resolve any citations for this publication.

ResearchGate has not been able to resolve any references for this publication.

Conference Paper

Full-text available

Scilab code of reduced gradient method for solving linear convex optimal control

January 2012

In this paper, the optimization of a objective function under linear constraints is given for optimal control of servomotor, a Scilab code of reduced gradient is proposed and modified for this type problems. Some numerical examples of linear optimal control are also given.

View full-text

Article

New Fr\'echet features for random distributions and associated sensitivity indices

March 2015

In this article we define new Fr\`Echet features for random cumulative distribution functions using contrast. These contrasts allow to construct Wasserstein costs and our new features minimize the average costs as the Fr\`Echet mean minimizes the mean square Wasserstein$_2$ distance. An example of new features is the median, and more generally the quantiles. From these definitions, we are able to ... [Show full abstract] define sensitivity indices when the random distribution is the output of a stochastic code. Associated to the Fr\`Echet mean we extend the Sobol indices, and in general the indices associated to a contrast that we previously proposed.

Article

A new three-term conjugate gradient method

November 1985 · Journal of Optimization Theory and Applications

In this paper, we describe an implementation and give performance results for a conjugate gradient algorithm for unconstrained optimization. The algorithm is based upon the Nazareth three-term formula and incorporates Allwright preconditioning matrices and restart tests. The performance results for this combination compare favorably with existing codes.

Preprint

Provably Faster Algorithms for Bilevel Optimization

June 2021

Bilevel optimization has been widely applied in many important machine learning applications such as hyperparameter optimization and meta-learning. Recently, several momentum-based algorithms have been proposed to solve bilevel optimization problems faster. However, those momentum-based algorithms do not achieve provably better computational complexity than $\mathcal{O}(\epsilon^{-2})$ of the ... [Show full abstract] SGD-based algorithm. In this paper, we propose two new algorithms for bilevel optimization, where the first algorithm adopts momentum-based recursive iterations, and the second algorithm adopts recursive gradient estimations in nested loops to decrease the variance. We show that both algorithms achieve the complexity of $\mathcal{O}(\epsilon^{-1.5})$, which outperforms all existing algorithms by the order of magnitude. Our experiments validate our theoretical results and demonstrate the superior empirical performance of our algorithms in hyperparameter applications. Our codes for MRBO, VRBO and other benchmarks are available $\text{online}^1$.

Article

Numerical experience with conjugate direction methods in constrained minimization

February 1989 · European Journal of Operational Research

In this paper, the numerical experience gained in solving constrained minimization problems using Conjugate Direction Methods is presented. For this purpose, Dennemeyer's Least Square Algorithm, Sequential Conjugate Gradient-Restoration Algorithm, Reduced Conjugate Gradient Method (RCGM) and an accelerated computer code mikki2 based on RCGM are described. Numerical implementation of these ... [Show full abstract] techniques is presented by solving certain test problems. The results obtained clearly indicate superiority and efficiency of the developed code mikki2.

Conference Paper

Convergence and Numerical Sensitivity of Backward-Adaptive LPC Predictor

February 1993

First Page of the Article

Preprint

Trade-offs Between Error Exponents and Excess-Rate Exponents of Typical Slepian-Wolf Codes

May 2020

Typical random codes (TRC) in a communication scenario of source coding with side information at the decoder is the main subject of this work. We derive the TRC error exponent for fixed-rate random binning and show that at relatively high rates, the TRC deviates significantly from the optimal code. We discuss the trade-offs between the error exponent and the excess-rate exponent for the typical ... [Show full abstract] random variable-rate code and characterize its optimal rate function. We show that the error exponent of the typical random variable-rate code may be strictly higher than in fixed-rate coding. We propose a new code, the semi-deterministic ensemble, which is a certain variant of the variable-rate code, and show that it dramatically improves upon the later: it is proved that the trade-off function between the error exponent and the excess-rate exponent for the typical random semi-deterministic code may be strictly higher than the same trade-off for the variable-rate code. Moreover, we show that the performance under optimal decoding can be attained also by different universal decoders: the minimum empirical entropy decoder and the generalized (stochastic) likelihood decoder with an empirical entropy metric.

Data

Full-text available

Figure S2

August 2011

Annotated Sample R Code to Perform the Stochastic Brass PF Ratio Method (DOCX)

View full-text

Conference Paper

Error Correction Coding of Stochastic Numbers Using BER Measurement

July 2019

Article

INTERACTIVE UBASIC CODE FOR ROSEN'S GRADIENT PROJECTION METHOD.

June 1975

An interactive computer program for J. Rosen's Gradient Projection Method is presented. This interactive algorithm approach is contrasted with both manual computation and ″canned″ program execution for the implementation of complex mathematical programming techniques. An example problem along with its solution has been included for illustrative purposes.

Article

Validating a six-degrees-of-freedom maneuvering in irregular waves code for extreme conditions—The p...

July 2020 · Probabilistic Engineering Mechanics

Arthur M. Reed

This paper focuses on issues related to Validation, i.e., determining for a computational tool that predicts the motions of a vessel in extreme seas if the theory and the code that implements the theory accurately models the relevant physical problem of interest. The challenge of this validation is the fact that the excitation process— waves—are stochastic and that as a consequence, the responses ... [Show full abstract] are in turn stochastic. Therefore, there are three aspects of the problem that must be dealt with: how much data is sufficient; what statistical techniques should be used to analyse the data; and how should the data be compared? Possible answers to these questions are provided.

Article

Asymptotics of Predictive Stochastic Complexity

January 1993

The purpose of this expository paper is to give a survey of rigorous asymptotic results in the theory of stochastic complexity. We provide a brief survey of the early developments and the ideas which culminated in a basic inequality on universal codes. The rest of the paper is devoted to results showing that the lower bound given by the basic inequality is achieved in many important cases. The ... [Show full abstract] asymptotic behaviour of three kinds of predictive stochastic complexities associated with ARMA processes will be established. Finally, extensions of the ARMA results for multivariable systems, for Ljung’s scheme, and for nonparametric problems will be considered.

Stochastic Gradient Descent (SGA) Algorithm - Code

No file available

Recommended publications

Scilab code of reduced gradient method for solving linear convex optimal control

New Fr\'echet features for random distributions and associated sensitivity indices

A new three-term conjugate gradient method

Provably Faster Algorithms for Bilevel Optimization

Numerical experience with conjugate direction methods in constrained minimization

Convergence and Numerical Sensitivity of Backward-Adaptive LPC Predictor

Trade-offs Between Error Exponents and Excess-Rate Exponents of Typical Slepian-Wolf Codes

Figure S2

Error Correction Coding of Stochastic Numbers Using BER Measurement

INTERACTIVE UBASIC CODE FOR ROSEN'S GRADIENT PROJECTION METHOD.

Validating a six-degrees-of-freedom maneuvering in irregular waves code for extreme conditions—The p...

Asymptotics of Predictive Stochastic Complexity