Gianluca Kosmella’s research while affiliated with Eindhoven University of Technology and other places

What is this page?


This page lists works of an author who doesn't have a ResearchGate profile or hasn't added the works to their profile yet. It is automatically generated from public (personal) data to further our legitimate goal of comprehensive and accurate scientific recordkeeping. If you are this author and want this page removed, please let us know.

Publications (4)


Schematic depiction of the noise model of ONNs that we study. First, data x is modulated onto light. This step adds an AWGN term Nm. This light enters the Photonic Layer, in which a weighted addition takes place, adding AWGN Nw. The activation function is then applied, adding AWGN Na. The activation function may be applied by photo-detecting the signal of the weighted addition, turning it to a digital signal and applying the activation function on a computer. The result of that action would then be modulated again, to produce the optical output of the photonic neuron. The modulator is thus only required in the first layer, as each photonic neuron takes in light and outputs light.
(a) Base 4−3−2 network, light circles indicate matrix–vector products, dark circles indicate merging/averaging (see (b) and (c)), boxes indicate activation functions. The light box indicates a single instance of a photonic layer as in figure 1. (b) Example for the tree-like design with 2 layers as input copies to each subsequent layer. The light circles indicate the linear operations/matrix-vector products. The results of the linear operation is averaged (single solid-blue circle) and fed through the activation function, producing the multiple version of the layers output (boxes). (c) Example of accordion-like design.
MSE ( ⋅102) for the tree-like design (top) and the accordion-like design (bottom) as function of copies on LeNet5 trained for MNIST classification. The pale area contains the 95%-confidence intervals. The red squares highlight the ‘low copies regime’ of which the values are highlighted in the abstract.
Relative accuracy for the tree-like design (top) and the accordion-like design (bottom) as function of copies on LeNet5 trained for MNIST classification. The pale area contains the 56.5%-confidence intervals. The red squares highlight the ‘low copies regime’ of which the values are highlighted in the abstract.
Accuracy of LeNet ONNs, depending on the amount of inserted identity layers and the variance level of the ONN, for (a) a network with tanh activation function and one copy, (b) a network with ReLU activation function and one copy, (c) a network with linear activation function and one copy, (d)a network with tanh activation function and two copies, (e) a network with ReLU activation function and two copies, (f) a network with linear activation function and two copies.

+1

Noise-resilient designs and analysis for optical neural networks
  • Article
  • Full-text available

October 2024

·

29 Reads

Gianluca Kosmella

·

·

All analog signal processing is fundamentally subject to noise, and this is also the case in next generation implementations of optical neural networks (ONNs). Therefore, we propose the first hardware-based approach to mitigate noise in ONNs. A tree-like and an accordion-like design are constructed from a given NN that one wishes to implement. Both designs have the capability that the resulting ONNs gives outputs close to the desired solution. To establish the latter, we analyze the designs mathematically. Specifically, we investigate a probabilistic framework for the tree-like design that establishes the correctness of the design, i.e. for any feed-forward NN with Lipschitz continuous activation functions, an ONN can be constructed that produces output arbitrarily close to the original. ONNs constructed with the tree-like design thus also inherit the universal approximation property of NNs. For the accordion-like design, we restrict the analysis to NNs with linear activation functions and characterize the ONNs’ output distribution using exact formulas. Finally, we report on numerical experiments with LeNet ONNs that give insight into the number of components required in these designs for certain accuracy gains. The results indicate that adding just a few components and/or adding them only in the first (few) layers in the manner of either design can already be expected to increase the accuracy of ONNs considerably. To illustrate the effect we point to a specific simulation of a LeNet implementation, in which adding one copy of the layers components in each layer reduces the mean-squared error (MSE) by 59.1% for the tree-like design and by 51.5% for the accordion-like design. In this scenario, the gap in accuracy of prediction between the noiseless NN and the ONNs reduces even more: 93.3% for the tree-like design and 80% for the accordion-like design.

Download


Noise-Resilient Designs for Optical Neural Networks

August 2023

·

53 Reads

All analog signal processing is fundamentally subject to noise, and this is also the case in modern implementations of Optical Neural Networks (ONNs). Therefore, to mitigate noise in ONNs, we propose two designs that are constructed from a given, possibly trained, Neural Network (NN) that one wishes to implement. Both designs have the capability that the resulting ONNs gives outputs close to the desired NN. To establish the latter, we analyze the designs mathematically. Specifically, we investigate a probabilistic framework for the first design that establishes that the design is correct, i.e., for any feed-forward NN with Lipschitz continuous activation functions, an ONN can be constructed that produces output arbitrarily close to the original. ONNs constructed with the first design thus also inherit the universal approximation property of NNs. For the second design, we restrict the analysis to NNs with linear activation functions and characterize the ONNs' output distribution using exact formulas. Finally, we report on numerical experiments with LeNet ONNs that give insight into the number of components required in these designs for certain accuracy gains. We specifically study the effect of noise as a function of the depth of an ONN. The results indicate that in practice, adding just a few components in the manner of the first or the second design can already be expected to increase the accuracy of ONNs considerably.


Fig. 3. A screenshot from movebank displaying the raw GPS data.
Fig. 5. Density-based histogram of singular values for √ n ˆ L for the words sequential data in blue bars and the theoretical predictions associated with the improvement clustering with K = 200 as the red line. Not visible in this figure is that both empirical distributions have long tails. Still 9% of the singular values ofˆNofˆ ofˆN / √ n exceed 10 and 1% of the singular values of √ n ˆ L exceed 30.
Fig. 7. Density-based histogram of singular values for √ n ˆ L andˆNandˆ andˆN / √ n for the animal movement data in blue bars and the theoretical predictions associated with the improvement clustering with K = 10 as the red line and with K = 100 as the purple dashed line.
Fig. 11. (left) The singular value density ofˆNofˆ ofˆN for a simulated DC-BMC (blue bars) as compared to the theory (blue line) and a perturbed model (red bars). (right) The singular value density ofˆLofˆ ofˆL.
Fig. 12. A plot of the matrix {1[ ˆ Fij > 0]}i,j , where the rows and columns are sorted according to the improved clustering. We plotted the matrix like such becausê F is quite sparse due to the trajectory's length = 2451 being quite short: the minimum, median, mean, and maximum of the entries of the matrix { ˆ Fi,j }i,j are 0, 0, /n 2 ≈ 0.027, and 14, respectively.
Detection and Evaluation of Clusters within Sequential Data

October 2022

·

832 Reads

Motivated by theoretical advancements in dimensionality reduction techniques we use a recent model, called Block Markov Chains, to conduct a practical study of clustering in real-world sequential data. Clustering algorithms for Block Markov Chains possess theoretical optimality guarantees and can be deployed in sparse data regimes. Despite these favorable theoretical properties, a thorough evaluation of these algorithms in realistic settings has been lacking. We address this issue and investigate the suitability of these clustering algorithms in exploratory data analysis of real-world sequential data. In particular, our sequential data is derived from human DNA, written text, animal movement data and financial markets. In order to evaluate the determined clusters, and the associated Block Markov Chain model, we further develop a set of evaluation tools. These tools include benchmarking, spectral noise analysis and statistical model selection tools. An efficient implementation of the clustering algorithm and the new evaluation tools is made available together with this paper. Practical challenges associated to real-world data are encountered and discussed. It is ultimately found that the Block Markov Chain model assumption, together with the tools developed here, can indeed produce meaningful insights in exploratory data analyses despite the complexity and sparsity of real-world data.