A theoretical framework for transfer of knowledge across modalities in artificial and cognitive systems

Source: OAI
  • Source
    [Show abstract] [Hide abstract]
    ABSTRACT: We study on-line learning in the linear regression framework. Most of the performance bounds for on-line algorithms in this framework assume a constant learning rate. To achieve these bounds the learning rate must be optimized based on a posteriori information. This information depends on the whole sequence of examples and thus it is not available to any strictly on-line algorithm. We introduce new techniques for adaptively tuning the learning rate as the data sequence is progressively revealed. Our techniques allow us to prove essentially the same bounds as if we knew the optimal learning rate in advance. Moreover, such techniques apply to a wide class of on-line algorithms, including p-norm algorithms for generalized linear regression and Weighted Majority for linear regression with absolute loss. Our adaptive tunings are radically different from previous techniques, such as the so-called doubling trick. Whereas the doubling trick restarts the on-line algorithm several times using a constant learning rate for each run, our methods save information by changing the value of the learning rate very smoothly. In fact, for Weighted Majority over a finite set of experts our analysis provides a better leading constant than the doubling trick.
    Journal of Computer and System Sciences 01/2000; DOI:10.1006/jcss.2001.1795 · 1.09 Impact Factor
  • Source
    [Show abstract] [Hide abstract]
    ABSTRACT: 1. Spatial ensemble averages were computed for 64 traces of electroencephalograms (EEGs) simultaneously recorded from 8 x 8 arrays over the epidural surfaces of the prepyriform cortex (PPC) and visual, somatic, and auditory cortices. They revealed a common waveform across each array. Examination of the spatial amplitude modulation (AM) of the waveform revealed classifiable spatial pattern in short time segments. The AM patterns varied within trials after presentation of identical conditioned stimuli, and also between trials with differing stimuli. 2. PPC EEGs revealed strong correlates with the respiratory rhythm; neocortical EEGs did not. 3. Time ensemble averaging of the PPC EEG attenuated the oscillatory bursts, indicating that olfactory gamma oscillations (20-80 Hz) were not phase-locked to the times of stimulus delivery but instead to inhalations. Time ensemble averages of neocortical recordings across trials revealed average evoked potentials starting 30-50 ms after the arrival of the stimulus. 4. Average temporal fast Fourier transform (FFT) power spectral densities (PSDs) from pre- and poststimulus PPC EEG segments revealed a peak of gamma activity in olfactory bursts. 5. The logarithm of the average temporal FFT PSDs from pre- and poststimulus neocortical EEG segments, when plotted against log frequency, revealed 1/f-type spectra in both pre- and poststimulus segments for negative/aversive conditioned stimuli (CS-) and positive/rewarding conditioned stimuli (CS+). The alpha'- and beta'-coefficients from the regression of Eq. 2 onto the average PSDs were significantly different between pre- and poststimulus segments, owing to the evoked potentials, but not between CS- and CS+ stimulus segments. 6. Spatiotemporal patterns were invariant over all frequency bins in the 1/f domain (20-100 Hz). Spatiotemporal patterns in the 2- to 20-Hz domain progressively differed from the invariant patterns with decreasing frequency. 7. In the spatial frequency domain, the logarithm of the average spatial FFT power spectra from pre- and poststimulus neocortical EEG segments, when plotted against the log spatial frequency, fell monotonically from the maximum at the lowest spatial frequency, downwardly curving to a linear 1/f spectral domain. This curve in the 1/f spectral domain extended from 0.133 to 0.880 cycles/mm in the PPC and from 0.095 to 0.624 cycles/mm in the neocortices. 8. Methods of FFT and principal component analysis (PCA) EEG decomposition were used to extract the broad-spectrum waveform common to all 64 EEGs from an array. AM patterns for the FFT and PCA components were derived by regression. They were shown by cross-correlation to yield spatial patterns that were equivalent to each other and to AM patterns from calculation of the 64 root-mean-square amplitudes of the segments. 9. Each spatial AM pattern was expressed by a 1 x 64 column vector and a point in 64-space. Similar patterns formed clusters, and dissimilar patterns gave multiple clusters. A statistical test was devised to evaluate dissimilarity by a Euclidean distance metric in 64-space. 10. Significant spatial pattern classification of CS- versus CS+ trials (below the 1% confidence limit for 20 of each) was found in discrete temporal segments of poststimulus data after digital temporal and spatial filter optimization. 11. Varying the analysis window duration from 10 to 500 ms yielded a window length of 120 ms as optimal for pattern classification. A 120-ms window was subsequently stepped across each record in overlapping intervals of 20 ms. Windows in which episodic, significant CS+/CS- differences occurred lasted 50-200 ms and were separated by 100-200 ms in the poststimulus period. 12. Neocortical spatial patterns changed under reinforcement contingency reversal, showing a lack of invariance in respect to stimuli and a dependence on context and learning, as previously found for the olfactory bulb and PPC.
    Journal of Neurophysiology 08/1996; 76(1):520-39. · 3.04 Impact Factor
  • Source
    [Show abstract] [Hide abstract]
    ABSTRACT: We present a family of scale-invariant local shape features formed by chains of k connected, roughly straight contour segments (kAS), and their use for object class detection. kAS are able to cleanly encode pure fragments of an object boundary, without including nearby clutter. Moreover, they offer an attractive compromise between information content and repeatability, and encompass a wide variety of local shape structures. We also define a translation and scale invariant descriptor encoding the geometric configuration of the segments within a kAS, making kAS easy to reuse in other frameworks, for example as a replacement or addition to interest points. Software for detecting and describing kAS is released on We demonstrate the high performance of kAS within a simple but powerful sliding-window object detection scheme. Through extensive evaluations, involving eight diverse object classes and more than 1400 images, we 1) study the evolution of performance as the degree of feature complexity k varies and determine the best degree; 2) show that kAS substantially outperform interest points for detecting shape-based classes; 3) compare our object detector to the recent, state-of-the-art system by Dalal and Triggs [4].
    IEEE Transactions on Pattern Analysis and Machine Intelligence 02/2008; 30(1):36-51. DOI:10.1109/TPAMI.2007.1144 · 5.69 Impact Factor

Full-text (2 Sources)

Available from
May 27, 2014