John Denker

Single-molecule-sensitive microscopy and spectroscopy are transforming biophysics and materials science laboratories. Techniques such as fluorescence correlation spectroscopy (FCS) and single-molecule sensitive fluorescence resonance energy transfer (FRET) are now commonly available in research laboratories but are as yet infrequently available in...

Single-molecule sensitive microscopies and spectroscopies are transforming the biophysics and materials science laboratory. Techniques such as fluorescence correlation spectroscopy (FCS) and single-molecule sensitive fluorescence resonance energy transfer (FRET) are now commonly available in research laboratories but are as yet infrequently availab...

The one-dimensional motion of a particle is analyzed when the force on it is inversely proportional to its displacement and directly proportional to the elapsed time. Such a force law describes a projectile in a musket barrel that is propelled by a hot ideal gas where either the number of moles or the temperature increases linearly with time due to...

We describe a method, “Shortest Path Segmentation” (SPS), which combines dynamic programming and a neural net recognizer for segmenting and recognizing character strings. We describe the application of this method to two problems: recognition of handwritten ZIP Codes, and recognition of handwritten words. For the ZIP Codes, we also used the method...

The Physics Teacher 48(3), 207 (2010) DOI: http://doi.org/10.1119/1.3317468

The Physics Teacher 48(2), 143 (2010) DOI: http://doi.org/10.1119/1.3293672

We describe a novel method of generating monodisperse subfemtoliter aqueous droplets on demand by means of piezoelectric injection. Droplets with volumes down to 200 aL are generated by this technique. The droplets are injected into a low refractive index perfluorocarbon so that they can be optically trapped. We demonstrate the use of optical tweez...

We have developed and describe a method to generate monodisperse optically-trappable aqueous emulsion droplets (“hydrosomes”) on demand by piezoelectric injection. The droplets have been measured to have radii as small as 368 nm ± 16 nm, corresponding to a volume of 212 aL ± 27 aL. The hydrosomes are injected into a perfluorocarbon continuous phase...

The Physics Teacher 46(2), 126 (2008) DOI: http://doi.org/10.1119/1.2834542

The Physics Teacher 46(1), 63 (2008) DOI: http://doi.org/10.1119/1.2824009

:(1) Theoutputsof a typicalmulti-outputclassificationnetworkdo not satisfythe axiomsof probability;probabilitiesshouldbepositiveandsumto one.

We have developeda neural-network architecture for recognizing

Memory-based classification algorithms such as Radial Basis Functions or K-nearest neighbors often rely on simple distances (Euclidean distance, Hamming distance, etc.), which are rarely meaningful on pattern vectors.

In pattern recognition, statistical modeling, or regression, the amount of data is a critical factor affecting the performance. If the amount of data and computational resources are unlimited, even trivial algorithms will converge to the optimal solution. However, in the practical case, given limited data and other resources, satisfactory performan...

This paper compares the performance of several classifier algorithms on a standard database of handwritten digits. We consider not only raw accuracy, but also training time, recognition time, and memory requirements. When available, we report measurements of the fraction of patterns that must be rejected so that the remaining patterns have misclass...

This paper compares the performance of several classifier algorithms on a standard database of handwritten digits. We consider not only raw accuracy, but also training time, recognition time, and memory requirements. When available, we report measurements of the fraction of patterns that must be rejected so that the remaining patterns have misclass...

We have used information-theoretic ideas to derive a class of practical and nearly optimal schemes for adapting the size of a neural network. By removing unimportant weights from a network, several improvements can be expected: better generalization, fewer training examples required, and improved speed of learning and/or classification. The basic i...

. In pattern recognition, statistical modeling, or regression, the amount of data is a critical factor affecting the performance. If the amount of data and computational resources are unlimited, even trivial algorithms will converge to the optimal solution. However, in the practical case, given limited data and other resources, satisfactory perform...

This paper compares the performance of classifier algorithms on a standard database of handwritten digits. We consider not raw accuracy, but rejection, training time, recognition time, and memory requirements. "Comparison of Leaning for Handwritten Digit Recognition", International Conference on Neural F. and P. Cie Publishers, 1995 Y. Le L. Bottou...

This paper compare the performance of classifier algorithms on a stan- dard database of handwritten digits. We consider not only raw accuracy, but also training time, recognition time, and memory requirements. When available, we report measurements of the fraction of patterns that must be rejected so that the remaining patterns have misclassificati...

In pattern recognition, statistical modeling, or regression, the amount of data is a critical factor affecting the performance. If the amount of data and computational resources are unlimited, even trivial algorithms will converge to the optimal solution. However, in the practical case, given limited data and other resources, satisfactory performan...

This paper compares the performance of several classi#er algorithms on a standard database of handwritten digits. We consider not only raw accuracy, but also rejection, training time, recognition time, and memory requirements. 1 COMPARISON OF LEARNING ALGORITHMS FOR HANDWRITTEN DIGIT RECOGNITION Y. LeCun, L. Jackel, L. Bottou, A. Brunot, C. Cortes,...

We have implemented a system for virtual private networking, with special attention to the needs of telecommuters. In particular, we used off-the-shelf hardware and open-source software to create a platform to provide IP security and other services for in-home networks.Our experience has taught us a number of things about the scalability of the Fre...

: (1) The outputs of a typical multi-output classification network do not satisfy the axioms of probability; probabilities should be positive and sum to one. This problem can be solved by treating the trained network as a preprocessor that produces a feature vector that can be further processed, for instance by classical statistical estimation tech...

We present an application of back-propagation networks to handwritten digit recognition. Minimal preprocessing of the data was required, but architecture of the network was highly constrained and specifically designed for the task. The input of the network consists of normalized images of isolated digits. The method has 1% error rate and about a 9%...

We present a feed-forward network architecture for recognizing an unconstrained handwritten multi-digit string. This is an extension of previous work on recognizing isolated digits. In this architecture a single digit recognizer is replicated over the input. The output layer of the network is coupled to a Viterbi alignment module that chooses the b...

This paper compares the performance of classifier algorithmson a standard database of handwritten digits. We consider not rawaccuracy, but rejection, training time, recognition time, and memoryrequirements."Comparison of Leaning for Handwritten Digit Recognition", International Conference onNeural F. and P. Cie Publishers, 1995Y. Le L. Bottou, C. J...

Character Recognition has served as one of the principal proving grounds for neural-net methods and has emerged as one of the most successful applications of this technology. This chapter outlines optical character recognition document analysis systems developed at AT&T Bell Labs that combine the strengths of machine-learning algorithms with high-s...

A new class of adiabatic computing circuits is presented. One advantage of this new kind of circuits over existing approaches is that switching energy can be reduced because no diodes are used. These new circuits also have non-floating output levels over most of the data valid time which is important for restoring logic levels and minimizing proble...

We explain (a) why people want a low-energy computer; (b) under what conditions there is-or is not-an irreducible energy per computation for CMOS circuits; (c) partial versus full adiabatic computation, and their relationship to logically reversible computation; (d) various schemes for achieving adiabatic operation

Memory-based classification algorithms such as radial basis functions or K-nearest neighbors often rely on simple distances (Euclidean distance, Hamming distance, etc.), which are rarely meaningful on pattern vectors. More complex better suited distance measures are often expensive and rather ad-hoc. We propose a new distance measure which: 1) can...

This paper compares the performance of several classifier algorithms on a standard database of handwritten digits. We consider not only raw accuracy, but also training time, recognition time, and memory requirements. When available, we report measurements of the fraction of patterns that must be rejected so that the remaining patterns have misclass...

First Page of the Article

With adiabatic techniques for capacitor charging, theory suggests that it should be possible to build gates with arbitrarily small energy dissipation. In practice the complexity of adiabatic approaches has made them impractical. We describe a new CMOS logic family-Adiabatic Dynamic Logic (ADL)-that is the result of combining adiabatic theory with c...

We have constructed a system for recognizing multi-character images 1. This is a nontrivial extension of our previous work on single-character im- ages. It is somewhat surprising that a very good single-character recognizer does not in general form a good basis for a multi-character recognizer. The correct solution depends on three key ideas: 1) A...

The authors have designed a writer-adaptable character recognition system for online characters entered on a touch terminal. It is based on a Time Delay Neural Network (TDNN) that is pre-trained on examples from many writers to recognize digits and uppercase letters. The TDNN without its last layer serves as a preprocessor for an optimal hyperplane...

A mathematical model for the formation and maintenance of synaptic contacts at the motor endplate is proposed. It is based on diffusion between sarcoplasmic nuclei of limiting amounts of a morphogen substance. The morphogen is postulated to act on genetic ...

We describe a method, “Shortest Path Segmentation” (SPS), which combines dynamic programming and a neural net recognizer for segmenting and recognizing character strings. We describe the application of this method to two problems: recognition of handwritten ZIP Codes, and recognition of handwritten words. For the ZIP Codes, we also used the method...

First Page of the Article

In many machine learning applications, one has not only training data but also some high-level information about certain invariances that the system should exhibit. In character recognition, for example, the answer should be invariant with respect to small spatial distortions in the input images (translations, rotations, scale changes, etcetera). T...

A neural network algorithm-based system that reads handwritten ZIP codes appearing on real US mail is described. The system uses a recognition-based segmenter, that is a hybrid of connected-components analysis (CCA), vertical cuts, and a neural network recognizer. Connected components that are single digits are handled by CCA. CCs that are combined...

We describe two neural-net approaches to digit recognition. One method uses a neural-network chip to perform line thinning and local feature extraction. This preprocessing stage was designed by hand and did not involve any learning. However, automatic learning was used in the final classification step. The chip can process about 100 characters/sec,...

The authors describe a method which combines dynamic programming and a neural network recognizer for segmenting and recognizing character strings. The method selects the optimal consistent combination of cuts from a set of candidate cuts generated using heuristics. The optimal segmentation is found by representing the image, the candidate segments,...

We present a feed-forward network architecture for recognizing an unconstrainedhandwritten multi-digit string. This is an extension of previouswork on recognizing isolated digits. In this architecture a single digit recognizeris replicated over the input. The output layer of the network iscoupled to a Viterbi alignment module that chooses the best...

A method for computer-aided cleaning of undesirable patterns in large training databases has been developed. The method uses the trainable classifier itself, to point out patterns that are suspicious, and should be checked by the human supervisor. While suspicious patterns that are meaningless or mislabeled are considered garbage, and removed from...

We describe a system which can recognize digits and uppercase letters handprinted on a touch terminal. A character is input as a sequence of [x(t), y(t)] coordinates, subjected to very simple preprocessing, and then classified by a trainable neural network. The classifier is analogous to “time delay neural networks” previously applied to speech rec...

: (1) The outputs of a typical multi-output classification network do not satisfy theaxioms of probability; probabilities should be positive and sum to one. This problem canbe solved by treating the trained network as a preprocessor that produces a feature vectorthat can be further processed, for instance by classical statistical estimation techniq...

The authors outline OCR (optical character recognition) technology developed at AT&T Bell Laboratories, including a recognition network that learns feature extraction kernels and a custom VLSI chip that is designed for neural-net image processing. It is concluded that both high speed and high accuracy can be obtained using neural-net methods for ch...

Hardware architectures for character recognition are discussed, and choices for possible circuits are outlined. An advanced (and working) reconfigurable neural-net chip that mixes analog and digital processing is described. It is found that different approaches to image recognition often lead to neural-net architectures that have limited connectivi...

A large class of applications where theoretical considerations that promote high-accuracy classification result in constrained network architectures have been identified through a series of experiments in pattern recognition using neural net algorithms. These constrained nets can map onto appropriately designed hardware. The concepts learned from t...

Exhaustive exploration of an ensemble of networks is used to model learning and generalization in layered neural networks. A simple Boolean learning problem involving networks with binary weights is numerically solved to obtain the entropy Sm and the average generalization ability Gm as a function of the size m of the training set. Learning curves...

It is shown that a neural net can perform handwritten digit recognition with state-of-the-art accuracy. The solution required automatic learning and generalization from thousands of training examples and also required designing into the system considerable knowledge about the task-neither engineering nor learning from examples alone would have suff...

Neural Network research has always interested hardware designers, theoreticians, and application engineers. But until recently, the common ground between these groups was limited: the neural-net chips were too small to implement any full-size application, and the algorithms were too complicated (or the applications not interesting enough) to be imp...

An application of back-propagation networks to handwritten zip code recognition is presented. Minimal preprocessing of the data is required, but the architecture of the network is highly constrained and specifically designed for the task. The input of the network consists of size-normalized images of isolated digits. The performance on zip code dig...

The ability of learning networks to generalize can be greatly enhanced by providing constraints from the task domain. This paper demonstrates how such constraints can be integrated into a backpropagation network through the architecture of the network. This approach has been successfully applied to the recognition of handwritten zip code digits pro...

Two novel methods for achieving handwritten digit recognition are described. The first method is based on a neural network chip that performs line thinning and feature extraction using local template matching. The second method is implemented on a digital signal processor and makes extensive use of constrained automatic learning. Experimental resul...

An evaluation is made of several neural network classifiers, comparing their performance on a typical problem, namely handwritten digit recognition. For this purpose, the authors use a database of handwritten digits, with relatively uniform handwriting styles. The authors propose a novel way of organizing the network architectures by training sever...

We present an application of back-propagation networks to handwritten digit recognition. Minimal preprocessing of the data was required, but architecture of the network was highly constrained and specifically designed for the task. The input of the network consists of normalized images of isolated digits. The method has 1% error rate and about a 9%...

An abstract is not available.

Early results from exploring alternative computer architectures based on hints from neurobiology suggest that networks of highly-interconnected, simple, low-precision processors can give tools for tackling problems that have been hard or impossible to do on standard computers. The authors describe an electronic neural model and show how this model...

Summary form only given. The author discusses neural networks, devices that have the potential to learn from examples, form generalizations, and discover the rules behind patterns in data. He has studied the limits on such devices and has shown that learning from examples is possible, as long as plenty of training data are provided and the network...

A general-purpose, fully interconnected neural-net chip was used to perform computationally intensive tasks for handwritten digit recognition. The chip has nearly 3000 programmable connections, which can be set for template matching. The templates can be reprogrammed as needed during the recognition sequence. The recognition process proceeds in fou...

A discussion is presented as to why special-purpose chips are needed for useful implementations of algorithms in such applications as pattern recognition and classification. The authors describe one design in particular, the key feature of which is simple arithmetic processors distributed among data storage to minimize data movement. The algorithm...

High‐resolution patterning of InP is gaining importance for electronic and photonic applications such as distributed feedback lasers and high‐speed transistors. Electron‐beam lithography is the highest resolution and most flexible technology for these applications. Unfortunately, backscattered electron effects become an imposing limitation when bot...

Electronic neural networks can perform the function of associative memory. Given an input pattern, the network searches through its stored memories to find which of them best matches the input. Thus the network does a combination of content-addressable search and error correction. The number of random memories that a network can store is limited to...

A high-density matrix of α-Si resistors was made to demonstrate a new type of parallel-processing associative memory consisting of an interconnected array of analog amplifiers. The 22 × 22 resistor matrix was made using a technology compatible with conventional VLSI processing. This demonstration circuit can recall up to four 22- bit memories in 1...

We have studied the dynamics of a totally interconnected network of nonlinear amplifiers by building model electronic circuits using dense arrays of resistors and discrete amplifiers. Such models have been discussed recently in the context of spin glasses and neural networks. Even without optimization for speed, these circuits easily reproduce and...

We report on the measurement of the longitudinal spin diffusion coefficient in doubly spin polarized atomic hydrogen. Measurements were made using pulsed nuclear magnetic resonance to probe the transport of magnetization between a sealed container and a large reservoir. Results are presented for densities between 1.4×10 ¹⁵ and 4.8×10 ¹⁶ cm ⁻³ at te...

Recent work has applied ideas from many fields including biology, physics and computer science, in order to understand how a highly interconnected network of simple processing elements can perform useful computation. Such networks can be used as associative memories, or as analog computers to solve optimization problems. This article reviews the wo...

We designed an Electronic Neural Network (ENN) memory with 256 neurons on a single chip using a combination of analog and digital VLSI technology plus a custom microfabrication process. Amplifiers with inverting and noninverting outputs are used for the neurons to make inhibitory and excitatory connections. The connections between the individual ne...

This paper contains some ideas for tidying up and extending the standard neural network model. It includes some fundamental generalities as well as some arcane technicalities. One central theme is how to make a network that is as close to ideal as possible, given various restrictions on the available fabrication resources. Another theme is how to u...

Recent proposals for neural network models indicate that an array of amplifiers coupled to a lattice of wires with resistive components at the crosspoints can perform calculations using collective properties similar to those observed in biological systems. Such a network can perform both memory and processing functions. The promise of the connectio...

We have been studying a gas of hydrogen atoms at very low temperatures, within a few tenths of a degree of absolute zero. At these temperatures any ordinary substance would have turned into a dense liquid or solid, but the hydrogen atoms remain an ideal gas--the atoms are about 10 times as far apart as the molecules of ordinary air. Normally, these...

This paper reviews basic lithographic considerations for current integrated circuit fabrication at the 1 micron level and illustrates methods for making structures with dimensions smaller than 0.1 micron. Many of these methods are well suited to fabricating high-density, resistive neural-networks.