Article

# The generalization performance of ERM algorithm with strongly mixing observations

Machine Learning (impact factor: 1.59). 04/2012; 75(3):275-295. DOI:10.1007/s10994-009-5104-z pp.275-295
Source: DBLP

ABSTRACT The generalization performance is the main concern of machine learning theoretical research. The previous main bounds describing
the generalization ability of the Empirical Risk Minimization (ERM) algorithm are based on independent and identically distributed
(i.i.d.) samples. In order to study the generalization performance of the ERM algorithm with dependent observations, we first
establish the exponential bound on the rate of relative uniform convergence of the ERM algorithm with exponentially strongly
mixing observations, and then we obtain the generalization bounds and prove that the ERM algorithm with exponentially strongly
mixing observations is consistent. The main results obtained in this paper not only extend the previously known results for
i.i.d. observations to the case of exponentially strongly mixing observations, but also improve the previous results for strongly
mixing samples. Because the ERM algorithm is usually very time-consuming and overfitting may happen when the complexity of
the hypothesis space is high, as an application of our main results we also explore a new strategy to implement the ERM algorithm
in high complexity hypothesis space.

0 0
·
0 Bookmarks
·
44 Views

### Keywords

complexity hypothesis space

dependent observations

Empirical Risk Minimization

ERM algorithm

exponential

exponentially

generalization ability

generalization bounds

generalization performance

hypothesis space

i.i.d. observations

independent

known results

new strategy

observations

overfitting

previous results

relative uniform convergence

samples

theoretical research