1.

Introduction

Optical techniques have been widely investigated and implemented in the field of information security, encryption, and authentication.^1,2 Information security is a crucial challenge to modern society,³ for example, data encryption for the critical databases of private and commercial information. Much impressive progress with optical techniques has been investigated and demonstrated in information encryption and authentication. The main purpose of utilizing optical techniques for information security is that data waveforms possess multiple dimensions, including phase, amplitude, spectrum, polarization, orbital angular momentum, etc. These dimensions can be combined in various approaches to enhance encryption security and make it more resistant to attacks. After the first proposal of double random phase encryption,⁴ many optical encryption schemes with optoelectronic setups have been developed. The classical encryption system based on holography,^5,6 interference,^7,8 ptychography,^9,10 single-pixel imaging^11,12 and scattering imaging^13,14 are proposed, which are a further expansion of the application with the traditional optical system. Recently, to upgrade the security level and fully exploit the abundant degrees of freedom of encryption systems, more advanced optical systems have been designed and demonstrated, for example, nonlinear encryption engine,¹⁵ meta-optics cryptography system,^16,17 and angular-momentum holography nested encryption.¹⁸

Utilizing a random scattering media as a coding mask is a simple yet effective way for image encryption.³ Speckle correlography is a cutting-edge technique that enables the construction of high-resolution images of laser-illuminated objects.^19,20 Briefly, the backscattered laser speckle intensity patterns are collected and then used to create a detailed visual representation of the diffuse object with computational approaches. The imaging approach is grounded on the principle that the average energy spectrum of laser-illuminated objects can be ascertained through the autocorrelation of speckle patterns in the Fourier domain. In the context of speckle correlography, an iterative phase retrieval (PR) algorithm is employed to reconstruct images from the estimated Fourier modulus of the diffuse object. Therefore, speckle correlography can be employed as a random phase encoding approach for optical encryption applications.²¹ Despite coherent scattering-based encryption schemes, the incoherent scattering method based on speckle correlation is also proposed for image encryption.^13,22 The physics-informed method for a ciphertext-only attack hinges on utilizing speckle correlography, which establishes that the autocorrelation of the ciphertext is fundamentally indistinguishable from that of the speckle.²³

Recent advances in optical encryption methods mainly focus on the complexity of experimental setups, which are based on advanced optoelectronic devices. The photorefractive crystal or metasurface device encodes information with light signals and the plaintext-ciphertext form is consistent with a frame-based camera. The same data form would lead to a high risk of the system being cracked. Meanwhile, traditional intensity images are redundant and temporally sparse. Therefore, a different data dimension between encryption and decryption might be a promising approach to facing information security.²⁴

Event cameras are representative neuromorphic devices that are inspired by the architecture of the human brain and have the potential to revolutionize computing by providing a more efficient and effective way to process information.²⁵ Compared to conventional frame-based cameras, the event camera with bionic circuit units has microsecond-level responsiveness and higher dynamic range, hence showing greater feasibility for optical encryption under high-speed recordings and poor illumination conditions. Event cameras offer trade-offs in terms of power consumption and frame rate, in exchange for sparsely sampling visual scenes and still achieving remarkable performance on computational imaging tasks. Equipped with neuromorphic devices and computational algorithms, computational neuromorphic imaging (CNI) is anticipated to emerge as a promising and rapidly advancing field in optical imaging.²⁶ CNI-informed computing modality achieves superior efficiency with low power consumption and low latency for certain algorithms,^27,28 and information security would greatly benefit from focusing on well-defined optical encryption.

An event camera is utilized in the neuromorphic encryption system to capture per-pixel intensity changes in an asynchronous manner, generating event streams that encode information on the time, pixel position, and polarity. Consequently, when the object undergoes motion, the speckle patterns are moved correspondingly, and speckle events are generated and recorded by an event camera. These speckle events arise due to light scattering, which produces a distinctive, unpredictable pattern of intensity fluctuations that can be employed to form a process-prior-empowered key. Therefore, a neuromorphic encryption system with different data forms processing can be developed using an event camera.

In this work, a neuromorphic encryption technique is proposed using event-stream speckles, which has a different data form of the ciphertext and high encrypted efficiency. The proposed encryption scheme has been demonstrated with simulated analysis and experimental verification. The inherent randomness and unpredictability of speckle events make it extremely difficult for attackers to replicate the same pattern of events and produce the correct key. Furthermore, the neuromorphic encryption method has the potential for high-speed encryption and decryption processing, which makes it suitable for practical applications. For image decryption, we propose a physical key based on a physics-informed neural network, which is trained with a simulated forward model. In addition, the proposed encryption method has remarkable noise immunity, showing impressive robustness in practical noisy scenarios.

2.

Principle and Method

2.1.

Optical Cryptosystem with Speckle Correlography

Speckle correlography is mainly the coherent scattering process in that we can use the statistical correlation between speckle patterns to encrypt information. As shown in Fig. 1, the encryption process is realized by the speckle correlography modulation and events recorded with the neuromorphic sensors. The detailed forward processes are depicted in the blue dashed box. That is, the origin information modulated with a coherent scattering process and the generation of corresponding event-stream ciphertext are synchronously realized via the neuromorphic encryption system.

Fig. 1

Schematic illustration of the neuromorphic encryption and decryption processes.

The generation of the encrypted speckles can be illustrated based on the imaging correlography.¹⁹ The plaintext object is illuminated with a coherent light source, and the non-imaged speckle patterns then can be collected by a frame-based camera. The encrypted object exhibits optical roughness, and its microscale surface height variations are random and comparable in size to the light’s wavelength. As a result, the reflected beam undergoes random and coherent dephasing, resulting in the detector in the image plane recording the intensity pattern of speckle grains.^19,29 The fully developed speckles $I(u)$ can be described as the squared modulus of the Fourier transform of the object field:

Eq. (1)

$$I(u)={|F[f_n(v)]|}^2,$$

where $F$ denotes a Fourier transform, $u$ denotes a 2D coordinate in the observation plane, and $v$ is a 2D spatial coordinate vector in object space. $f_n(v)=|f_0(v)e^{i{\varphi }_n(v)}|$ stands for the optical field coded by the diffuse object, $f_0(v)$ is the field amplitude of the encrypted object, and ${\varphi }_n(v)$ denotes the stochastic phase of the $n$th instance of the encrypted object field that is related with the height profile of the diffuse object. Therefore, the autocovariance of the collected speckles can be estimated as

Eq. (2)

$${\widehat{C}}_I(\mathrm{\Delta }u,N)={\int }_{-\infty }^{\infty }P(u+\mathrm{\Delta }u)\{\frac{1}{N}\sum _{n=1}^N[I_n(u+\mathrm{\Delta }u)-\overline{I}][I_n(u)-\overline{I}]\}\mathrm{d}u,$$

where $\mathrm{\Delta }u$ is a vector separation in the image plane, $\overline{I}$ is the average intensity of the speckle grains, and $P(u)$ is the circular pupil function can be defined as

Eq. (3)

$$P(u)=\{\begin{array}{cc}1,& u\in P\\ 0,& u\notin P\end{array}.$$

While an infinite number of speckle grains is used for estimating statistical autocovariance¹⁹

Eq. (4)

$$C_I(\mathrm{\Delta }u,N)=\underset{N\to +\infty }{\lim }{\widehat{C}}_I(\mathrm{\Delta }u,N)=\mathcal{H}(\mathrm{\Delta }u)·|\mathrm{\Gamma }(\mathrm{\Delta }u){|}^2=[P(u)\star P(u)]·|\mathrm{\Gamma }(\mathrm{\Delta }u){|}^2,$$

where $\mathcal{H}$ is the transfer function of the optical system, $\mathrm{\Gamma }(\mathrm{\Delta }u)=|F[f_0(v)]{|}^2$ is the 2D Fourier transform amplitude of the object’s field.³⁰ For plaintext image encryption, the power spectrum can be employed in iterative PR algorithms,³¹ which is the same process for image decryption.

2.4.

Experimental Setup and Datasets Preparation

The neuromorphic encryption system is schematically illustrated in Fig. 2, which includes an illumination laser (Thorlabs, HNL100LB, central wavelength: 632.8 nm) and a corresponding beam expander (Thorlabs, BE10M-A). A diffuse reflective object is selected as the plaintext and is mounted on a motorized translation stage (Winner Optics, WN262TA20). An event camera (iniVation, DAVIS 346) and a 40× objective lens (Nikon) are employed to collect the speckle events. Despite the encryption process involving a complex transformation, the hardware implementation is relatively simpler than conventional methods.³ The object is a $10\mathrm{mm}\times 10\mathrm{mm}$ letter with a reflective diffuse metal surface and the distance between the object and the event camera is about 300 mm. The movement speed of the translation stage is 0.2 mm/s in the horizontal direction.

Fig. 2

Neuromorphic encryption system. (a) The optical configuration. (b) An encrypted letter “N” is made with a diffuse metal surface, captured with frame-mode and a focusing lens. (c) An event camera is employed as the neuromorphic sensor to collect the encrypted data.

In order to reduce the cost of data collection, a synthetic framework is proposed to generate the simulated data for DNN training. The training data are synthesized with the forward physical model of the speckle correlography¹⁹ and the v2e process for event generation.⁴⁶ Here, the light source is assumed as a collimated coherent laser beam with a wavelength of 632.8 nm. The object is modulated by a random mask as the diffuse object and is numerically propagated over a far-field distance. The far-field distance $d$ is also set to 300 mm, resulting in a speckle pattern $I(x,y)$. v2e toolbox that can generate realistic synthetic events from conventional frames, which is employed to simulate and make the training data of speckle events. The speckle field was shifted with two pixels with three consecutive grounds, which is used to generate the degraded speckle events. Then, the same set of speckle patterns of $32\times 32$ in size is used for accumulated autocorrelation preprocess in both the simulation and experiment, and $256\text{pixel}\times 256\text{pixel}$ is selected for autocorrelation calculation.

3.

Results

For the neuromorphic encryption system, image encryption with physical modulation and data form transform are simultaneously conducted. In the simulated experiments, the detailed encryption process is analyzed in the method part and the intrinsic data are also presented for a better understanding of the forward encrypted process.

First, the characteristics of event-stream data with different scattering conditions are analyzed. As shown in Fig. 3, the accumulated speckle and corresponding retrieved autocorrelation are presented. The statistical characteristics can be obviously distinguished from the normalized intensity of the white dashed line. We can draw a similar conclusion with scalable imaging through unknown scattering media.³⁹ Therefore, the preprocessing step with the retrieved autocorrelation is essential for the physics-informed learning model, which is suitable for practical-synthetic environments and can reduce the differences in the data characteristics. By training on the autocorrelations of simulated event-stream data and their associated ground truth images, the proposed PR model reduces the potential for model mismatch and improves image reconstruction efficiency.

Fig. 3

Analysis of encryption-related data. (a) The accumulated speckle and corresponding retrieved autocorrelation and the final decrypted results in different scattering conditions are presented. (b) The normalized intensity of the dashed line in the accumulated speckle. (c) The normalized intensity of the dashed line in the retrieved autocorrelation.

To demonstrate the effectiveness of the neuromorphic encryption method, detailed experiments are presented and analyzed. If we use conventional cameras, the frame-based cyphertext can be recorded as the intensity speckles, which is presented as the virtual cyphertext. The encrypted event-stream data can be further synthesized by the moving virtual speckles. Numerical simulations have been carried out to illustrate the effectiveness of the neuromorphic encryption method. In the simulated experiments, the size of all the images is set as $256\text{pixel}\times 256\text{pixel}$. A total of nine groups of speckle events are generated, which have distinct degraded processes with different scattering transport modes.^47,48 One group of data was used for test verification of the encryption, and the remaining eight sets of data were used for training the decryption algorithm model. According to the simulation results in Fig. 4, the final encrypted results are event-stream data. Therefore, the optical encryption scheme based on the CNI paradigm can encrypt images with different data forms. The original image recovery of the encrypted event is performed by the proposed physical key. Using the physics-informed PR algorithm for the encrypted event-stream data, the autocorrelation is first retrieved and employed for decrypted image reconstruction. From the final retrieved results, the decrypted image has been initially validated based on numerical simulations. Therefore, the effectiveness of the information security and high dimensional resistance to ciphertext attacks of the proposed method can be seen from the encrypted time data and decryption results.

Fig. 4

Simulation results. The virtual cyphertext of speckle patterns from intermediate processes is used for event-stream data generation. The accumulated speckles with encrypted events and decrypted images with the learning PR algorithm.

In addition, we investigate the robustness of the proposed neuromorphic encryption scheme against cropping and noise. For practical information transport, the original information might be lost which results in part of the damaged event-stream data. Therefore, the capability to reconstruct useful information with damaged data is crucial for image decryption. The results are respectively presented in Figs. 5 and 6. The four images on the left side of Fig. 5 show the cropped ciphertexts with cropping ratios of $1/16$, $1/8$, $1/4$, and $1/2$, respectively. The autocorrelation of plaintext can be retrieved from the cropped ciphertext. According to the decryption results, we can see that the proposed method has resistance to the different cropping ratios. The two images on the left of Fig. 6 present the ciphertexts added zero-mean, Gaussian white noise with variance of 0.01 and 0.02; the two images on the right side of Fig. 6 present the ciphertexts added salt and pepper noise with 0.01 and 0.02 noise density. The retrieved autocorrelations of the corrupted ciphertexts and the reconstructed plaintext images are also presented together. From the final decryption results, the proposed method has the robustness capability against noise with different levels and types. Therefore, these results indicate that the neuromorphic encryption approach has superior robustness against data cropping and noise disturbance.

Fig. 5

Robustness test against cropping. Ciphertext with different cropping ratios are 1/16, 1/8, 1/4, and 1/2, respectively.

Fig. 6

Robustness test against noise. Ciphertext added zero-mean Gaussian white noise with variances of 0.01 and 0.02 and added salt and pepper noise with 0.01 and 0.02 noise densities.

Further experimental validation is carried out for the effectiveness and practicality of the neuromorphic encryption scheme. We implement the practical CNI system as illustrated in Fig. 2 and verify it by the proposed algorithm. As shown in Fig. 7, the encrypted events, intermediate images, and corresponding decrypted results of the letters targets (i.e., “N” and “L”) and quick draw targets (i.e., “Tshirt” and “Apple”) are presented.⁴⁹ The event-stream data are selected and preprocessed with accumulation and autocorrelation. From the experimental results, the accumulated speckle images with different time periods differ significantly. However, the corresponding autocorrelations have similar main structural information, and the final decrypted images are nearly identical to each other, which demonstrates the proposed method is robust to variable scattering processes. The known degraded physical process is the key element for the information decryption. Therefore, neuromorphic encryption is an innovative bio-inspired method for information security with a data-form transformation process.

Fig. 7

Experimental results. (a) Letter target “N.” (b) Letter target “L.” (c) Quick draw target “Tshirt.” (d) Quick draw target “Apple.” For each subfigure: the first column from the left is the ground truth plaintext; the first row is the collected encrypted events that have the same time period and different time periods; the second and third rows are the accumulated speckle and corresponding autocorrelation; the last row is the decrypted result via the learning PR algorithm.

4.

Analysis

Since event cameras only capture motion, they predominantly record high-frequency information about moving targets, while low-frequency information with smooth grey-scale changes can be easily lost. Conventional phase recovery algorithms struggle to accurately reconstruct power spectra and inverse series with such incomplete information. By understanding the target’s encryption process, we can generate corresponding degradation simulation data and employ learning methods to recover some lost information during the encryption process. As illustrated in Fig. 8, we compare the traditional hybrid input-output (HIO) PR algorithm to our learning method for event data decryption.^19,31 For both simulated and experimental data, the HIO algorithm fails to produce reliable results, whereas our proposed learning PR algorithm effectively compensates for information loss in a simulated dataset containing a final forward encryption model.

Fig. 8

Comparison results of the traditional HIO PR algorithm with our learning method for event data decryption.

In Eq. (6), the length of the time duration $\mathrm{\Delta }t$ is a predefined parameter for accumulated images with speckle events. By selecting different periods within a single continuous event stream, the accumulated images and the corresponding autocorrelation are calculated with the same starting point in time. As shown in Fig. 9, the short time duration (i.e., $0.1\mathrm{\Delta }t$ and $0.2\mathrm{\Delta }t$) can lead to sparse accumulated speckles, whereas the long one (i.e., $5\mathrm{\Delta }t$ and $10\mathrm{\Delta }t$) can result in the speckles being overlapped and blurred. As a result, the wrong time duration selection will produce the incorrectly retrieved autocorrelation and can not obtain the proper decrypted images. Therefore, an appropriate time duration with prior knowledge of the events process is crucial for accurate decryption. In specific extreme scenarios, accumulating adequate event information for signal parsing might be essential. This process can be further improved by employing warp or refocusing techniques for event stream data washout, which is one of our upcoming research topics.

Fig. 9

Cracking resistance test with wrong time duration selection (in the red dash box). The correct process with a proper time duration selection is also presented in the green dash box.

5.

Discussion

According to the experimental results and analysis, several highlights are pointed out as follows:

6.

Conclusion

In summary, we present a bio-inspired encryption system for optical information security, which is the first cryptography scheme using the CNI technique, to the best of our knowledge. The proposed methodology enhances speckle correlation through event-stream data, which provides a new paradigm for optical image encryption. CNI technique and speckle correlography are combined to improve the security and efficiency of optical encryption, which shows the proposed methodology surpasses traditional methods in terms of security and encryption level. The potential benefits of neuromorphic encryption using event-stream data make it a promising research area that could transform the field of information security, which paves the way to CNI applications and provides a reference for complex scenarios computing with bio-inspired sensors.