Machine learning techniques are proving to be very useful for design of optical amplifiers, noise characterization of frequency combs, optimization of fiber-optic communications systems, inverse design of photonics components and quantum-noise limited signal detection. In this talk, we will review some of the successful applications of machine learning in photonics, and look into what is next in this emerging field. More specifically, we will look into how reinforcement learning can be used for the generation of programmable pulse shapes, which has a broad range of applications in classical and quantum engineering.

In quantum optics a small number of building blocks, like resonators, waveguides, driving-induced coupling, and parametric interactions allow the design of a broad variety of devices and functionalities, distinguished by their scattering properties. These include transducers, amplifiers, and nonreciprocal devices, like isolators or circulators. Usually, the design of such systems is handcrafted by an experienced scientist. In our work, we develop a discovery algorithm that automates this process. By optimizing the continuous and discrete system properties our automated search identifies the minimal resources required to realize the requested scattering behavior. The discovered architectures represent classes of solutions and are not bound to special numerical values or platforms. Our approach is applicable to optical, microwave, mechanical, electrical, and hybrid circuits.

We will review our work in the field of smart ultrafast photonics where machine-learning algorithms are combined with nonlinear optical systems allowing for optimized performance and control. In particular, we will show how the techniques of machine learning can be efficiently exploited for the analysis of nonlinear instabilities; the prediction of complex supercontinuum generation dynamics with orders of magnitude increased computation speed when compared to conventional direct numerical integration; the optimized and precise control of the spectrum of broadband supercontinuum sources.

We experimentally demonstrate an optoacoustic recurrent operator (OREO) based on stimulated Brillouin scattering, which enables recurrent functionalities for photonic machine learning and neural network applications. OREO employs sound waves to catch and process the context defined by a sequence of optical pulses. It controls the coherent recurrent operation completely optically on pulse-by-pulse level without the need of an artificial reservoir. We demonstrate OREO's capability to compute correlations in pulse trains. Then, we use the pulse-by-pulse control of OREO to implement recurrent dropout. Furthermore, we use OREO to recognize patterns of optical pulse trains, in which we can distinguish up-to 27 different patterns. Eventually, OREO can be used as key component of a bi-directional perceptron, bring a new class of photonic neural networks within reach.

Machine learning techniques using artificial neural networks (ANN) have proven to be extremely ef-fective in designing nanophotonic systems. This presentation focuses on two applications where ANNs are utilized for designing nanophotonic scatterers. In the first scenario, ANNs act as surrogate solvers for Maxwell's equations, allowing the design of scatterers tailored to specific fabrication technologies like laser nanoprinting. Designing low-index material scatterers is complex, so solving the inverse problem multiple times from different starting points is crucial. A Fourier neural operator ANN serves as a surrogate Maxwell solver, simplifying this process. The second scenario integrates ANNs into a holistic metasurface design framework. Individual meta-atoms are efficiently described by their scattering responses, typically expressed as polarizability or T-matrix that provide metasurfaces with functionality on demand. Then, suitably trained ANNs are used to identify feasible physical objects that offer the desired T-matrices.

We present a framework for optical metrology in which, through the hardware implementation of artificial intelligence via metasurfaces, a conventional camera becomes a metrology system capable of retrieving observables from a light beam. We show the experimental realization of a prototype of this system and the results of its use for measuring the properties of thin films.

Analysing the optical field of multimode fibers by intensity images machine learning for design in photonics-based sensing and imaging applications. However, existing mathematical algorithms, iterative methods, or AI algorithms encounter scalability issues. In this study, we incorporate the physics principle of mode superposition into neural networks for mode decomposition. This integration eliminates the need for an extensive amount of training data and the time-consuming training process. The proposed method, without pre-training, can effectively perform mode decomposition for up to 220 modes. With the extracted amplitude and phase information, the correlation coefficient between the reconstructed optical field and the original image surpasses 98%. Investigations with noisy data demonstrate the network's efficiency in extracting both phase and magnitude information, even when the signal-to-noise ratio of the image is as low as 1dB which is crucial for secure data communication with multimode fibers.

In this work, we study the problem of coupling between nanostructures by means of artificial neural networks. We consider the perturbation of the optical response of a nanoparticle induced by nanostructures in its close proximity. We train an ANN to predict the near-field characteristics of the system based on the far-field scattering spectra readily achievable from the experiment.

By integrating chalcogenide phase change materials to the fabrication of coupled silicon waveguide arrays we can manipulate the propagation of light through photonic devices, by the introducing nanoscale refractive index perturbations. Prediction of the required pixel pattern needed to produce a given unitary transmission matrix is a complex problem. In order to optimize for multiple input and outputs simultaneously we employ an artificial neural network, for both forward prediction and inverse design. This work hopes to pave the way towards all optical computation and the production of reconfigurable analogue quantum simulators.

I will present the trajectory of our work on the application of machine learning techniques to problems in photonic crystals and materials analysis. I will highlight our work on contrastive pre-training approaches for photonic crystal analysis, opportunities and techniques in multimodal pre-training for settings with multiple sources of complementary data, and, finally, interpretable machine learning systems with applications to topological materials analysis.

Deep neural networks (DNNs) are data-driven systems that have transformed traditional research methods and are driving scientific discovery in artificial electromagnetic materials (AEMs). AEMs, including electromagnetic metamaterials, photonic crystals, and plasmonics, are research fields where DNNs have had significant results, validating the data-driven approach, especially for problems where conventional methods have failed. Although the universal approximation theorem indicates that deep learning is a universal solver and therefore can be applied to solve any problem, there are several drawbacks, including the requirement for large training datasets, the unknown size of required datasets, and the black box nature of models (i.e., no access to any physics learned by the model). Through incorporation of prior knowledge, informed deep neural networks can solve many of the outstanding problems in deep learning and may also learn new physics of systems under study. In view of the great potential of deep learning for the future of AEM research, we review the status of the field, focusing on recent advances, open challenges, and future directions.

Photonic neural networks perform ultrafast inference operations but are trained on slow computers. We highlight on-chip network training enabled by silicon photonics. We introduce quantum photonic neural networks and discuss the role of weak nonlinearities.

Photonic computing, especially neuromorphic processors, offers high-throughput linear processing with a crucial advantage in achieving low-latency applications over electronics. While conventional electronics prioritize throughput at the expense of latency, photonic systems excel in low-latency applications. Historically, neuromorphic photonic systems focused on machine learning, overlooking their potential in real-time control scenarios. However, recent trends highlight AI's role in complex control tasks like autonomous navigation and scientific experiments demanding low-latency inference. This talk presents a framework for implementing photonic neural networks in control applications, emphasizing their relevance in Model Predictive Control (MPC) and reinforcement learning (RL). Simulations demonstrate the capability of a modest number of neurons to handle nonlinear control tasks, surpassing linear controllers. Furthermore, a spiking implementation of photonic neural networks can bring additional benefits to challenging control tasks requiring ultra-low latency.

Neuromorphic photonics combines the advantages of photonics with the computational power of neural networks to create novel reconfigurable processing devices. Doing so enables new applications which are difficult or impossible for conventional digital electronic or RF processors to handle. This talk will highlight recent progress in neuromorphic photonic integrated circuits (PICs), beginning with photonic neurons which integrate both the linear and nonlinear functionality. It will then cover recent demonstrations utilizing PICs, including model predictive control (MPC), RF blind source separation (BSS), nonlinearity compensation in long-haul communications, and RF fingerprinting.

We will describe methods for implementing optical neural networks including our recent work on the otpical implementation of nonlinear operations using multiple scattering in a linear optical system.

This presentation will report our recent progress in achieving precise and reliable photonic neural networks. We'll explore developments in both integrated silicon photonics and metasurface-based implementations through hardware-software codesign approaches.

The advent of photonic neural networks posed a challenge of designing an activation function that would unlock the full potential of photonics (low-latency, energy-efficiency, WDM) for machine learning applications. In this work we demonstrate a coherent, multi-frequency photonic nonlinear activation function based on stimulated Brillouin scattering . These properties not only make it compatible with existing MZI mesh-based phase-reliant optical matrix multiplication schemes, but also facilitate resource-efficient frequency-basis information encoding. Our design features all-optical activation function shape tuning and is capable of providing net gain, compensating for insertion losses.

We present a single-layer optical inference system based on nonlinear optical diffraction. In this system, input optical data is superposed with control parameters and then focused into a nonlinear optical crystal leading to second harmonic generation which serves as the output of the system. We demonstrate image classification while utilizing a very low number of degrees of freedom. Importantly, the performance of the system can be tuned by controlling the level of spatial mode mixing in the nonlinear crystal.

Light propagation in disordered media can be seen as a linear operation on fields : a multiplication by a random matrix, between a set of input modes (for instance pixels of an SLM) and output modes (for instance pixels of a camera). This operation, akin to a single-layer of a neural network, can be leveraged for a wealth of signal processing and machine learning tasks. I will present some of our works, ranging from classification to time-series prediction, and importantly present our recent approaches to go beyond linear random projections, in order to provide deeper equivalent neural networks and better machine-learning performances across a variety of tasks.

We develop a novel Fourier-domain optical convolutional neural networks (FOCNNs) with multi-stage framework to hierarchical learn the image features at the speed of light. The FOCNN consists of two optical convolutional layers integrated with multiple parallel kernels and one optical fully-connected layer to form an all-optical CNN-like physical network structure. The FOCNN convolute the whole Fourier spectrum of the objects rather than the local receptive field of the objects, so it could extract the global and non-local features of the objects. In addition, the vortex phase is introduced to the optical convolutional kernels to extract the edge features. We incorporate this Fourier optics-based, parallel, one-step FOCNN in the tasks of semantic segmentation for pixel-level classification, and the capability of video-rate segmentation for objects is also demonstrated based on the programmable spatial light modulators, which demonstrated the computational power of FOCNN located in the range of Peta operations per second (POPS). Therefore, the FOCNN is useful for the real-time dynamic inference tasks, such as robotic vision, autonomous driving, and so on.

Neuromorphic computing is emerging as a promising solution to address the ever-growing demand for computational power driven by artificial neural networks. We present a photonic hardware accelerator, implemented on the Silicon on Insulator platform, based on an incoherent crossbar array. We outline the system architecture and showcase live convolution processing using the photonic hardware accelerator. Furthermore, we integrate phase-change material which serves as a non-linear building block for a reconfigurable photonic neural network. We train the neural network for language classification.

We experimentally demonstrate an autonomous, fully tunable and scalable optical neural network of 400+ parallel nodes based on a large area, multimode semiconductor laser. We implement hardware compatible, online learning strategies based on reinforcement learning and evolutionary strategies and evaluate them in terms of performance and energy cost. Our system achieves high performance and a high classification bandwidth of 15KHz for the MNIST dataset. Our approach is highly scalable both in terms of classification bandwidth and neural network size due to our device's short response time (nanosecond).

Our research in neuromorphic computing leverages nonlinear optical dynamics to emulate neural network functionalities. In our experiments, we explore supercontinuum generation and other complex wave dynamics for information processing in the optical domain. Utilizing spectral-domain phase modulation and nonlinear femtosecond pulse broadening in multiple nonlinear fibers, we demonstrate effective data encoding and processing followed by a read-out layer training, akin to Extreme Learning Machines. Our benchmarks on diverse datasets showcase the scalability and inference capabilities of our system, and the distinct performance differences of two nonlinear domains, i.e. self-phase modulation and soliton fission. This work opens new avenues in quantifying physics-based analog computing platforms, suggesting implications for green computing, Big Data communications, and intelligent diagnostics.

Photonic systems, exhibiting multi-gigahertz bandwidth, facilitate data transmission at gigabit-per-second rates. While traditionally used in optical communication for data transfer, semiconductor lasers are now being explored for their potential in optical computation and signal processing. Injecting information into these lasers leads to nonlinear transformations and high-speed processing. Experimentally, a single semiconductor laser shows essential features for versatile computation, such as high-dimensional and nonlinear responses within sub-nanoseconds. To boost computational power, we study numerically the training of delay-coupled laser networks. The objective is, akin to training artificial neural networks, optimizing laser network's to improve performance and computational efficiency in challenging machine learning tasks. However, relying on offline optimization methods and physical models raises challenges due to device variability and limited system observability. Here, we propose evolutionary strategies to optimize physical systems without needing precise model knowledge, offering a promising approach for online system optimization.

Free-space optical neural networks (ONNs) are promising for their potential in intricate, high-resolution image processing. However, the introduction of nonlinear units, which are crucial for advancing the capabilities of free-space ONNs, poses a significant technical obstacle. Our research explores the feasibility of integrating nonlinear metasurfaces with free-space ONNs to preserve their high-throughput, low-loss advantages while enabling enhanced capabilities. We specifically examine the role of nonlinear amplitude modulation via metasurfaces in enhancing ONN performance. Through simulations, we evaluate its impact on the network's capability in pattern recognition task.

The efficient design of metasurfaces presents a challenging optimization problem due to the relatively large number of meta-atoms and their mutual coupling. In this work, we present two novel multi-output surrogate models used in the context of the Bayesian optimization of a beam splitter. We show how learning the vectorial quantities forming the final objective can lead to more accurate results and significant speed-ups when compared to classical optimization of scalar objectives. Furthermore, we discuss how to incorporate gradient information with respect to design parameters to further accelerate the optimization.

Brief laser pulses can induce autonomous organization of nanostructures pattern without external guidance. This interaction between a laser light and a material is governed by Maxwell's equations. These equations provide the theoretical framework for understanding how electromagnetic waves propagate and interact with matter. The Finite-Difference Time-Domain (FDTD) method models the laser-material interactions, providing insights into absorption, reflection, and scattering over time, ultimately contributing to self-organization within the material. Despite a theoretical understanding, there is no reliable model to predict the self-organization process responsible for the nanostructures. Our work addresses this issue by aiming to predict the surface changes after multiple laser irradiations using neural networks. Deep learning models have undergone advancements and prove suitable for extracting meaningful insights and simulating physical processes. This combination of laser physics and deep learning offer a promising approach to improve our ability to control nanostructures formation on materials.