CN107958475B - Variable-angle illumination tomography method and device based on deep learning generative network - Google Patents

Abstract

The invention discloses a variable-angle illumination tomography method and device based on a deep learning generation network, wherein the method includes: deriving the light in a non-uniform transparent state according to the Helmholtz equation of the wave and the Fourier propagation model of the light Distribution model of diffractive field and refraction field when propagating layer by layer in the medium; imitating the physical process to build a deep learning generative neural network that propagates in the time domain and frequency domain in the form of complex numbers; The angular spectrum propagation formula is propagated backward as the input optical complex number field data, and the collected complex number field passing through the sample to be reconstructed is used as the output optical complex number field data; the deep learning network parameters are adjusted according to the reconstruction resolution condition to train the network ; The three-dimensional refractive index distribution of the sample is obtained by solving the weight obtained by training, and the tomographic reconstruction of the sample is realized. The method realizes the tomographic reconstruction capability of low acquisition volume and high resolution, and effectively improves the resolution accuracy of sample tomographic reconstruction.

Description

Variable-angle illumination tomography method and device based on deep learning generative network

technical field

The invention relates to the technical fields of computational optics, computer vision and computational photography, in particular to a variable-angle illumination tomography method and device based on a deep learning generation network.

Background technique

At present, high-resolution tomographic reconstruction of microscopic samples, especially living biological samples, is a hot research issue in the fields of computational optical imaging, computer vision, and computational photography. In related tomography techniques, since most living biological cells have the characteristics of weak difference in intensity and high difference in phase, phase imaging technology is widely used for research. However, most of the existing phase tomography technologies need to collect a large amount of data, including images illuminated at different angles or images captured at different depths, and the acquisition speed limits the development and application of phase tomography.

Another common problem in the tomographic reconstruction of living biological samples is that the existing reconstruction techniques often have a relatively serious elongation phenomenon in the optical axis direction, causing large errors, thus limiting the tomographic resolution in the optical axis direction. , affecting the effect of tomographic reconstruction, and it is difficult to achieve nano-scale tomography. Meanwhile, in the tomographic methods proposed before, most of the light field propagation models used are linear propagation models that ignore multiple scattering. Doing so can make the algorithm simpler and more convenient, but it will affect the tomographic effect.

SUMMARY OF THE INVENTION

The present invention aims to solve one of the technical problems in the related art at least to a certain extent.

Therefore, an object of the present invention is to propose a variable-angle illumination tomography method based on a deep learning generation network, which realizes the tomographic reconstruction capability of low acquisition volume and high resolution, and effectively improves the resolution of sample tomographic reconstruction. rate accuracy.

Another object of the present invention is to provide a variable-angle illumination tomography device based on a deep learning generation network.

In order to achieve the above object, an embodiment of the present invention proposes a variable-angle illumination tomography method based on a deep learning generation network, which includes the following steps: deriving from the Helmholtz equation of waves and the Fourier propagation model of light. Obtain the distribution model of the diffraction field and the refraction field when light propagates layer by layer in a non-uniform transparent medium; according to the deduced distribution model, a deep learning generating neural network that propagates in the time domain and frequency domain in the form of complex numbers is built according to the physical process, wherein , the weight to be trained is the bulk refractive index distribution of the sample to be tomography, the training sample is the complex field distribution of the input light and the complex field distribution of the output light at the corresponding angle; the samples are irradiated in multiple groups within a preset angle range, and the camera is fixed Data acquisition is performed at the back end of the optical axis, so that the output complex number field of the collected non-transmitted sample is propagated backward through the angular spectrum propagation formula as the input optical complex number field data, and the collected complex number transmitted through the sample to be reconstructed is transmitted. The field is used as the output optical complex field data; the parameters of the deep learning network are adjusted according to the reconstruction resolution condition to train the network; the three-dimensional refractive index distribution of the sample is obtained by solving the weight obtained by the training, and the tomographic reconstruction of the sample is realized.

The variable-angle illumination tomography method based on the deep learning generation network according to the embodiment of the present invention can comprehensively consider the scattering and refraction process of light propagating in the multi-layer transparent sample by using the layered propagation model based on the beam propagation method. Combined with deep learning, the most effective optimization method at present, with the digital holographic acquisition method, the tomographic reconstruction capability of low acquisition volume and high resolution is realized, and the resolution accuracy of sample tomographic reconstruction is effectively improved.

In addition, the variable-angle illumination tomography method based on the deep learning generation network according to the above-mentioned embodiments of the present invention may also have the following additional technical features:

Further, in an embodiment of the present invention, the distribution models of the diffraction field and the refraction field are expressed as:

分别为傅里叶变换运算符与傅里叶逆变换运算符，ω_x、ω_y为傅里叶域坐标，

为波数，n₀为背景介质折射率，j为虚数单位，δn(r)为待训练权重。Among them, x, y, z are the three-dimensional coordinates of the refractive index distribution of the sample body, δz is the distance between adjacent layers in the optical axis direction in the layered model,

are the Fourier transform operator and the inverse Fourier transform operator, respectively, ω _x , ω _y are the Fourier domain coordinates,

is the wave number, n ₀ is the refractive index of the background medium, j is the imaginary unit, and δn(r) is the weight to be trained.

Further, in an embodiment of the present invention, according to the deduced distribution model, the deep learning generating neural network that propagates in the time domain and the frequency domain in the form of complex numbers is constructed by imitating a physical process, further comprising:

Taking the envelope complex amplitude a(r) of the light distribution field as the node of the neural network, for the nodes a(x, y, z) and a(x, y, z+δz) of the adjacent two layers , the operation relationship is divided into two parts corresponding to diffraction and refraction in the process of light propagation:

分别为傅里叶变换运算符与傅里叶逆变换运算符，

为傅里叶域坐标，

为波数，n₀为背景介质折射率，j为虚数单位，δn(r)为待训练权重。Among them, x, y, z are the three-dimensional coordinates of the refractive index distribution of the sample body, δz is the distance between adjacent layers in the optical axis direction in the layered model,

are the Fourier transform operator and the inverse Fourier transform operator, respectively,

is the Fourier domain coordinate,

is the wave number, n ₀ is the refractive index of the background medium, j is the imaginary unit, and δn(r) is the weight to be trained.

Further, in an embodiment of the present invention, the framework used in building the neural network is TensorFlow, and the layers used in the network include an input layer, a diffraction layer, a refraction layer, and a low-pass filter layer, wherein the The diffractive layer and the refraction layer correspond to the above-mentioned diffraction process operation and refraction process operation, respectively, and the low-pass filter layer corresponds to the collected frequency domain characteristics.

Further, in an embodiment of the present invention, it is characterized in that, in the network, the expression of the loss function is as follows:

loss=∑|y _predict -y _true |+S,

Among them, y _predict represents the data generated by the network, y _true represents the real collected data, and S represents the sparse item constraint.

In order to achieve the above object, another embodiment of the present invention proposes a variable-angle illumination tomography device based on a deep learning generation network, including: a derivation module, which is used for the Helmholtz equation of waves and the Fourier transform of light. The propagation model is derived to obtain the distribution model of the diffraction field and the refraction field when the light propagates layer by layer in a non-uniform transparent medium; the building module is used to build a complex number in the time domain and frequency domain according to the deduced distribution model according to the physical process. The deep learning of propagation generates a neural network, wherein the weight to be trained is the bulk refractive index distribution of the sample to be tomography, and the training sample is the complex number field distribution of the input light and the complex number field distribution of the output light at the corresponding angle; the acquisition module is used for preset Multiple groups of illumination are performed on the sample within the angular range, and the camera is fixed at the back end of the optical axis for data acquisition, so that the output complex number field of the collected sample that does not pass through is propagated backward through the angular spectrum propagation formula as the input optical complex number field data, and use the collected complex field passing through the sample to be reconstructed as the output optical complex field data; the training module is used to adjust the parameters of the deep learning network according to the reconstruction resolution condition to train the network; the reconstruction module is used to pass The three-dimensional refractive index distribution of the sample is obtained by solving the weight obtained from the training, and the tomographic reconstruction of the sample is realized.

The variable-angle illumination tomography device based on the deep learning generation network of the embodiment of the present invention can comprehensively consider the scattering and refraction process of light propagating in a multi-layer transparent sample by using a layered propagation model based on the beam propagation method. Combined with deep learning, the most effective optimization method at present, with the digital holographic acquisition method, the tomographic reconstruction capability of low acquisition volume and high resolution is realized, and the resolution accuracy of sample tomographic reconstruction is effectively improved.

In addition, the variable-angle illumination tomography device based on the deep learning generation network according to the above-mentioned embodiments of the present invention may also have the following additional technical features:

Further, in an embodiment of the present invention, the distribution models of the diffraction field and the refraction field are expressed as:

分别为傅里叶变换运算符与傅里叶逆变换运算符，ω_x、ω_y为傅里叶域坐标，

为波数，n₀为背景介质折射率，j为虚数单位，δn(r)为待训练权重。Among them, x, y, z are the three-dimensional coordinates of the refractive index distribution of the sample body, δz is the distance between adjacent layers in the optical axis direction in the layered model,

are the Fourier transform operator and the inverse Fourier transform operator, respectively, ω _x , ω _y are the Fourier domain coordinates,

is the wave number, n ₀ is the refractive index of the background medium, j is the imaginary unit, and δn(r) is the weight to be trained.

Further, in an embodiment of the present invention, the building module is further configured to use the envelope complex amplitude a(r) of the light distribution field as the node of the neural network, for the nodes of two adjacent layers a(x, y, z) and a(x, y, z+δz), the operational relationship is divided into two parts corresponding to diffraction and refraction in the process of light propagation:

分别为傅里叶变换运算符与傅里叶逆变换运算符，ω_x、ω_y为傅里叶域坐标，

为波数，n₀为背景介质折射率，j为虚数单位，δn(r)为待训练权重。Among them, x, y, z are the three-dimensional coordinates of the refractive index distribution of the sample body, δz is the distance between adjacent layers in the optical axis direction in the layered model,

are the Fourier transform operator and the inverse Fourier transform operator, respectively, ω _x , ω _y are the Fourier domain coordinates,

is the wave number, n ₀ is the refractive index of the background medium, j is the imaginary unit, and δn(r) is the weight to be trained.

Further, in an embodiment of the present invention, the framework used in building the neural network is TensorFlow, and the layers used in the network include an input layer, a diffraction layer, a refraction layer, and a low-pass filter layer, wherein the The diffractive layer and the refraction layer correspond to the above-mentioned diffraction process operation and refraction process operation, respectively, and the low-pass filter layer corresponds to the collected frequency domain characteristics.

Further, in an embodiment of the present invention, in the network, the expression of the loss function is as follows:

loss=∑|y _predict -y _true |+S,

Among them, y _predict represents the data generated by the network, y _true represents the real collected data, and S represents the sparse item constraint.

Additional aspects and advantages of the present invention will be set forth, in part, from the following description, and in part will be apparent from the following description, or may be learned by practice of the invention.

Description of drawings

The above and/or additional aspects and advantages of the present invention will become apparent and readily understood from the following description of embodiments taken in conjunction with the accompanying drawings, wherein:

1 is a flowchart of a method for variable-angle illumination tomography based on a deep learning generation network according to an embodiment of the present invention;

2 is a schematic diagram of a simulation image of a layered propagation model based on a beam propagation method according to an embodiment of the present invention;

3 is a schematic structural diagram of a neural network framework based on a deep learning generation network according to an embodiment of the present invention;

4 is a schematic structural diagram of a holographic acquisition system based on variable-angle illumination according to an embodiment of the present invention;

5 is a schematic diagram of a tomographic reconstruction result for a simulated pellet according to an embodiment of the present invention;

6 is a flowchart of a method for variable-angle illumination tomography based on a deep learning generation network according to a specific embodiment of the present invention;

FIG. 7 is a schematic structural diagram of a variable-angle illumination tomography device based on a deep learning generation network according to an embodiment of the present invention.

Detailed ways

The following describes in detail the embodiments of the present invention, examples of which are illustrated in the accompanying drawings, wherein the same or similar reference numerals refer to the same or similar elements or elements having the same or similar functions throughout. The embodiments described below with reference to the accompanying drawings are exemplary, and are intended to explain the present invention and should not be construed as limiting the present invention.

The method and device for variable-angle illumination tomography based on a deep learning generation network proposed according to the embodiments of the present invention will be described below with reference to the accompanying drawings. First, the variable-angle illumination layer based on a deep learning generation network proposed according to the embodiments of the present invention will be described with reference to the accompanying drawings. analysis method.

FIG. 1 is a flowchart of a method for variable-angle illumination tomography based on a deep learning generation network according to an embodiment of the present invention.

As shown in Figure 1, the variable-angle illumination tomography method based on deep learning generation network includes the following steps:

In step S101, the distribution model of the diffraction field and the refraction field when the light propagates layer by layer in the non-uniform transparent medium is derived according to the Helmholtz equation of the wave and the Fourier propagation model of the light.

It can be understood that the embodiments of the present invention can derive the diffraction field and the refraction field when light propagates layer by layer in a non-uniform transparent medium according to the Helmholtz equation of waves, the Fourier propagation model of light, and a series of reasonable approximate assumptions. distribution model.

For example, as shown in FIG. 2 , the BPM (Beam Propagation Method, beam propagation method) layered propagation model derived in the embodiment of the present invention can be simulated by using a computer to simulate the theoretical model. Set the input light complex field as the amplitude and phase distribution of the Gaussian beam, focus the imaging position on the central layer of the sample, and output the image.

Optionally, in an embodiment of the present invention, the distribution models of the diffraction field and the refraction field are expressed as:

分别为傅里叶变换运算符与傅里叶逆变换运算符，ω_x、ω_y为傅里叶域坐标，

为波数，n₀为背景介质折射率，j为虚数单位，δn(r)为待训练权重。Among them, x, y, z are the three-dimensional coordinates of the refractive index distribution of the sample body, δz is the distance between adjacent layers in the optical axis direction in the layered model,

are the Fourier transform operator and the inverse Fourier transform operator, respectively, ω _x , ω _y are the Fourier domain coordinates,

is the wave number, n ₀ is the refractive index of the background medium, j is the imaginary unit, and δn(r) is the weight to be trained.

Specifically, the basic principle of the embodiment of the present invention is the Helmholtz equation in an inhomogeneous medium and its derived formula for the propagation field distribution of paraxial waves, which are Equation 1 and Equation 2, respectively,

为拉普拉斯算子，I是特征算子，

是r位置的光的波数。Among them, r=(x, y, z) represents the spatial position distribution, u is the distribution field of the light at the r position,

is the Laplace operator, I is the characteristic operator,

is the wavenumber of the light at the r position.

n₀是背景介质的折射率，a(r)表示u(r)的复振幅包络in,

n ₀ is the refractive index of the background medium, a(r) represents the complex amplitude envelope of u(r)

第二种近似是在折射率分布扰动δn(r)较小的前提下，忽略(δn(r))²及其他高阶项。由此可以推得：Two approximate simplifications are introduced to the above two equations. The first approximation considers that the change of the complex amplitude of the envelope of the plane wave is slow, that is

The second approximation is to ignore (δn(r)) ² and other higher-order terms under the premise that the perturbation of the refractive index profile δn(r) is small. From this it can be deduced that:

Equation 3 is the Helmholtz equation for paraxial waves.

Taking the Fourier transform of Equation 3 can finally deduce,

Equation 4 can be expressed in two parts, diffraction and refraction:

In step S102, a deep learning generating neural network that propagates in the time domain and the frequency domain in the form of a complex number is constructed according to the derived distribution model by imitating the physical process, wherein the weight to be trained is the bulk refractive index distribution of the sample to be tomography, and the training sample is The complex number field distribution of the input light and the complex number field distribution of the output light corresponding to the angle.

That is to say, the embodiment of the present invention can build a deep learning generating neural network that propagates in the time domain and frequency domain in the form of complex numbers according to the deduced optical propagation model and imitating the physical model, and the weight to be trained is the bulk refractive index distribution of the sample, The training samples are the input light complex field distribution and the output light complex field distribution corresponding to the angle. Among them, the framework structure of the deep learning generative neural network is shown in Figure 3.

For example, the embodiment of the present invention applies the concept of micro-elements to micro-element and mesh the sample, and it is considered that the refractive index in each micro-element is the same. The size of this micro-element also reflects the size of the tomographic resolution. Applying the formula (4) derived earlier, the network can be built.

Further, in an embodiment of the present invention, a deep learning generative neural network that propagates in the time domain and frequency domain in the form of complex numbers is built according to the physical process according to the derived distribution model, further comprising: using the envelope of the distribution field of light to complex The amplitude a(r) is the node of the neural network. For the nodes a(x, y, z) and a(x, y, z+δz) of the adjacent two layers, the operation relationship is divided into corresponding to the light propagation process. The two parts of diffraction and refraction:

分别为傅里叶变换运算符与傅里叶逆变换运算符，ω_x、ω_y为傅里叶域坐标，

为波数，n₀为背景介质折射率，j为虚数单位，δn(r)为待训练权重。Among them, x, y, z are the three-dimensional coordinates of the refractive index distribution of the sample body, δz is the distance between adjacent layers in the optical axis direction in the layered model,

are the Fourier transform operator and the inverse Fourier transform operator, respectively, ω _x , ω _y are the Fourier domain coordinates,

is the wave number, n ₀ is the refractive index of the background medium, j is the imaginary unit, and δn(r) is the weight to be trained.

Specifically, in this embodiment of the present invention, the complex amplitude a(r) of the envelope of the light distribution field can be used as the node of the neural network, and for the nodes a(x, y, z) and a(x) of two adjacent layers , y, z+δz), its operational relationship can be divided into two parts:

correspond to diffraction and refraction, respectively, during light propagation.

Further, in an embodiment of the present invention, the framework used in the process of building a neural network is TensorFlow, and the layers used in the network include an input layer, a diffraction layer, a refraction layer, and a low-pass filter layer, wherein the diffraction layer and the refraction layer The layers correspond to the above-mentioned diffraction process operations and refraction process operations, respectively, and the low-pass filter layer corresponds to the collected frequency domain characteristics.

It can be understood that the framework that can be used in the process of building a deep learning neural network in this embodiment of the present invention is TensorFlow, and the layers used in the network include an input layer (Input), a diffraction layer (DiffractionLayer), a refraction layer (RefractionLayer) and a low-level layer. A low-pass filter layer (LowPassLayer), wherein the diffraction layer and the refraction layer correspond to the above-mentioned diffraction process operation and refraction process operation respectively, and the low-pass filter layer corresponds to the frequency domain characteristics of the acquisition device in the acquisition system. For a given sample, the diffractive layer is the same, so it can be shared; the refraction layer contains the parameter δn(r) to be obtained, which cannot be shared, and is the layer to be trained, and δn(r) is the weight to be trained.

Optionally, in an embodiment of the present invention, in the network, the expression of the loss function is as follows:

loss=∑|y _predic -y _true |+S,

Among them, y _predict represents the data generated by the network, y _true represents the real collected data, and S represents the sparse item constraint.

Specifically, in the network, the defined loss function expression is as follows:

loss=∑|y _predict -y _true |+S,

Among them, y _predict represents the data generated by the network, y _true represents the real collected data, S represents the sparse item constraint, and its expression is as follows:

Among them, γ ₁ represents the data item sparsity constraint, γ ₂ represents the differential term sparsity constraint, and w represents the weight, that is, δn(r).

In step S103, multiple groups of irradiation are performed on the sample within a preset angle range, and the camera is fixed at the rear end of the optical axis to perform data collection, so that the collected output light complex field that does not pass through the sample is passed through the angular spectrum propagation formula Backward propagation is used as input optical complex field data, and the collected complex field transmitted through the sample to be reconstructed is used as output optical complex field data.

It can be understood that, in the embodiment of the present invention, multiple groups of illumination can be performed on the sample within a certain angle range, the camera is fixed at the rear end of the optical axis to perform data collection, and the collected output light complex field of the sample that is not transmitted through the angular spectrum. The propagation formula is propagated backward as the input optical complex field data, and the collected complex field transmitted through the sample to be reconstructed is used as the output optical complex field data.

For example, as shown in FIG. 3 , in the embodiment of the present invention, the angle of the incident light can be changed by rotating the galvanometer, and the position of the camera at the acquisition end is fixed; the hologram is captured through the interference effect with the reference light, thereby obtaining the output light The complex field of .

In step S104, the parameters of the deep learning network are adjusted according to the reconstruction resolution condition to train the network.

It can be understood that, in this embodiment of the present invention, the deep learning generation network parameters can be adjusted according to the requirements of the reconstruction resolution, including the reconstruction sample parameters—reconstruction three-dimensional network size, resolution unit size, etc., and deep learning hyperparameters—initial learning rate, The network is trained with batch size and sparsity constraint parameters.

For example, in this embodiment of the present invention, the deep learning neural network built in the previous content can be used to set the parameters of the neural network according to the scale of the actual sample, the resolution requirement and the magnification of the acquisition end, such as the number of layers of the network (corresponding to axis directional resolution), the size of each layer (corresponding to the horizontal plane resolution), the batch size (depending on the number of groups of collected data), the initial learning rate, and the number of iterations, etc., after which the network is trained.

In step S105, the three-dimensional refractive index distribution of the sample is obtained by solving the weights obtained by the training, so as to realize the tomographic reconstruction of the sample.

It can be understood that, in this embodiment of the present invention, the bulk refractive index distribution of the sample can be obtained by using the adaptive moment estimation (Adam) optimization method for training, and the three-dimensional reconstruction of the sample can be realized by solving the obtained weight.

Specifically, after the network training is completed, the weight parameter of the training, namely δn(r), is taken out and drawn in layers, that is, the layering of the samples is realized. Among them, FIG. 5 is a schematic diagram of a tomographic reconstruction result for a simulated pellet.

In a specific embodiment of the present invention, as shown in FIG. 6 , the method of the embodiment of the present invention includes the following steps:

Step S1, the distribution model of the refraction field in the case where the light is diffracted in the medium;

Step S2, imitating the physical process to build a deep learning generation network in complex number form;

Step S3, performing multiple groups of irradiation with coherent light at different angles;

Step S4, the output terminal collects the complex field of light;

Step S5, adjust network parameters, and train;

In step S6, the volume refractive index distribution is calculated according to the training weight to realize tomography.

To sum up, the embodiment of the present invention utilizes the deep learning neural network to cooperate with the variable-angle plane wave illumination acquisition system to realize the functions of high-speed acquisition and accurate tomography.

According to the variable-angle illumination tomography method based on the deep learning generation network proposed in the embodiment of the present invention, by using the layered propagation model based on the beam propagation method, the scattering and refraction processes of light propagating in the multi-layer transparent sample can be comprehensively considered, Combining it with deep learning, the most effective optimization method at present, combined with the digital holographic acquisition method, the tomographic reconstruction capability of low acquisition volume and high resolution is realized, and the resolution accuracy of sample tomographic reconstruction is effectively improved.

Next, a variable-angle illumination tomography device based on a deep learning generation network proposed according to an embodiment of the present invention will be described with reference to the accompanying drawings.

FIG. 7 is a schematic structural diagram of a variable-angle illumination tomography device based on a deep learning generation network according to an embodiment of the present invention.

As shown in FIG. 7 , the variable-angle illumination tomography device 10 based on the deep learning generation network includes: a deriving module 100 , a building module 200 , an acquisition module 300 , a training module 400 and a reconstruction module 500 .

Wherein, the deriving module 100 is used for deriving, according to the Helmholtz equation of the wave and the Fourier propagation model of the light, the distribution model of the diffraction field and the refraction field when the light propagates layer by layer in the non-uniform transparent medium. The building module 200 is used to build a deep learning generating neural network that propagates in the time domain and the frequency domain in the form of complex numbers according to the derived distribution model according to the physical process, wherein the weight to be trained is the bulk refractive index distribution of the sample to be tomography, and the training sample is The complex number field distribution of the input light and the complex number field distribution of the output light corresponding to the angle. The acquisition module 300 is used to irradiate the sample in multiple groups within a preset angle range, and the camera is fixed at the rear end of the optical axis for data acquisition, so as to pass the collected output light complex field of the sample that does not pass through the angular spectrum propagation formula Backward propagation is used as input optical complex field data, and the collected complex field transmitted through the sample to be reconstructed is used as output optical complex field data. The training module 400 is used to adjust the parameters of the deep learning network according to the reconstruction resolution condition, so as to train the network. The reconstruction module 500 is used for obtaining the three-dimensional refractive index distribution of the sample by solving the weights obtained by training, so as to realize the tomographic reconstruction of the sample. The apparatus 10 according to the embodiment of the present invention realizes the tomographic reconstruction capability of low acquisition volume and high resolution, and effectively improves the resolution accuracy of sample tomographic reconstruction.

Further, in an embodiment of the present invention, the distribution models of the diffraction field and the refraction field are expressed as:

分别为傅里叶变换运算符与傅里叶逆变换运算符，ω_x、ω_y为傅里叶域坐标，

为波数，n₀为背景介质折射率，j为虚数单位，δn(r)为待训练权重。Among them, x, y, z are the three-dimensional coordinates of the refractive index distribution of the sample body, δz is the distance between adjacent layers in the optical axis direction in the layered model,

are the Fourier transform operator and the inverse Fourier transform operator, respectively, ω _x , ω _y are the Fourier domain coordinates,

is the wave number, n ₀ is the refractive index of the background medium, j is the imaginary unit, and δn(r) is the weight to be trained.

Further, in an embodiment of the present invention, the building module 200 is further configured to use the envelope complex amplitude a(r) of the light distribution field as a node of the neural network, and for the nodes a(x) of two adjacent layers , y, z) and a(x, y, z+δz), the operational relationship is divided into two parts corresponding to diffraction and refraction in the process of light propagation:

分别为傅里叶变换运算符与傅里叶逆变换运算符，ω_x、ω_y为傅里叶域坐标，

为波数，n₀为背景介质折射率，j为虚数单位，δn(r)为待训练权重。Among them, x, y, z are the three-dimensional coordinates of the refractive index distribution of the sample body, δz is the distance between adjacent layers in the optical axis direction in the layered model,

are the Fourier transform operator and the inverse Fourier transform operator, respectively, ω _x , ω _y are the Fourier domain coordinates,

is the wave number, n ₀ is the refractive index of the background medium, j is the imaginary unit, and δn(r) is the weight to be trained.

Further, in an embodiment of the present invention, the framework used in the process of building a neural network is TensorFlow, and the layers used in the network include an input layer, a diffraction layer, a refraction layer, and a low-pass filter layer, wherein the diffraction layer and the refraction layer The layers correspond to the above-mentioned diffraction process operations and refraction process operations, respectively, and the low-pass filter layer corresponds to the collected frequency domain characteristics.

Further, in an embodiment of the present invention, in the network, the expression of the loss function is as follows:

loss=∑|y _predict -y _true |+S,

Among them, y _predict represents the data generated by the network, y _true represents the real collected data, and S represents the sparse item constraint.

It should be noted that the foregoing explanation of the embodiment of the variable-angle illumination tomography method based on the deep learning generation network is also applicable to the variable-angle illumination tomography device based on the deep learning generation network of this embodiment, and will not be repeated here.

The variable-angle illumination tomography device based on the deep learning generation network proposed according to the embodiment of the present invention can comprehensively consider the scattering and refraction processes of light propagating in the multi-layer transparent sample by using the layered propagation model based on the beam propagation method, Combining it with deep learning, the most effective optimization method at present, combined with the digital holographic acquisition method, the tomographic reconstruction capability of low acquisition volume and high resolution is realized, and the resolution accuracy of sample tomographic reconstruction is effectively improved.

In the description of the present invention, it should be understood that the terms "center", "longitudinal", "lateral", "length", "width", "thickness", "upper", "lower", "front", " Rear, Left, Right, Vertical, Horizontal, Top, Bottom, Inner, Outer, Clockwise, Counterclockwise, Axial, The orientations or positional relationships indicated by "radial direction", "circumferential direction", etc. are based on the orientations or positional relationships shown in the accompanying drawings, which are only for the convenience of describing the present invention and simplifying the description, rather than indicating or implying the indicated devices or elements. It must have a specific orientation, be constructed and operate in a specific orientation, and therefore should not be construed as a limitation of the present invention.

In addition, the terms "first" and "second" are only used for descriptive purposes, and should not be construed as indicating or implying relative importance or implying the number of indicated technical features. Thus, a feature delimited with "first", "second" may expressly or implicitly include at least one of that feature. In the description of the present invention, "plurality" means at least two, such as two, three, etc., unless otherwise expressly and specifically defined.

In the present invention, unless otherwise expressly specified and limited, the terms "installed", "connected", "connected", "fixed" and other terms should be understood in a broad sense, for example, it may be a fixed connection or a detachable connection , or integrated; it can be a mechanical connection or an electrical connection; it can be directly connected or indirectly connected through an intermediate medium, it can be the internal connection of two elements or the interaction relationship between the two elements, unless otherwise specified limit. For those of ordinary skill in the art, the specific meanings of the above terms in the present invention can be understood according to specific situations.

In the present invention, unless otherwise expressly specified and limited, a first feature "on" or "under" a second feature may be in direct contact between the first and second features, or the first and second features indirectly through an intermediary touch. Also, the first feature being "above", "over" and "above" the second feature may mean that the first feature is directly above or obliquely above the second feature, or simply means that the first feature is level higher than the second feature. The first feature being "below", "below" and "below" the second feature may mean that the first feature is directly below or obliquely below the second feature, or simply means that the first feature has a lower level than the second feature.

In the description of this specification, description with reference to the terms "one embodiment," "some embodiments," "example," "specific example," or "some examples", etc., mean specific features described in connection with the embodiment or example , structure, material or feature is included in at least one embodiment or example of the present invention. In this specification, schematic representations of the above terms are not necessarily directed to the same embodiment or example. Furthermore, the particular features, structures, materials or characteristics described may be combined in any suitable manner in any one or more embodiments or examples. Furthermore, those skilled in the art may combine and combine the different embodiments or examples described in this specification, as well as the features of the different embodiments or examples, without conflicting each other.

Although the embodiments of the present invention have been shown and described above, it should be understood that the above-mentioned embodiments are exemplary and should not be construed as limiting the present invention. Embodiments are subject to variations, modifications, substitutions and variations.

Claims (8)

1. A variable-angle illumination chromatography method based on deep learning generation network is characterized by comprising the following steps:

deriving a distribution model of a diffraction field and a refraction field when light propagates in the non-uniform transparent medium layer by layer according to a Helmholtz equation of the wave and a Fourier propagation model of the light, wherein the distribution model of the diffraction field and the refraction field is expressed as:

wherein x, y and z are three-dimensional coordinates of the refractive index distribution of the sample body, and deltaz is the distance between adjacent layers in the optical axis direction in the layered model,

the operator of Fourier transform and the operator of inverse Fourier transform, omega_x、ω_yIn order to be the fourier domain coordinates,

is the wave number, n₀Taking the refractive index of a background medium, j is an imaginary number unit, and δ n (r) is the weight to be trained;

establishing a deep learning generation neural network which is propagated in a time domain and a frequency domain in a complex form according to the deduced distribution model by imitating a physical process, wherein weights to be trained are the volume refractive index distribution of a sample to be chromatographed, and training samples are the input light complex field distribution and the output light complex field distribution of corresponding angles;

carrying out multi-group irradiation on the sample within a preset angle range, fixing a camera at the rear end of an optical axis for data acquisition, backward propagating an acquired output light complex field which does not penetrate through the sample through an angular spectrum propagation formula to be used as input light complex field data, and taking the acquired complex field which penetrates through the sample to be reconstructed as output light complex field data;

adjusting the parameters of the deep learning network according to the reconstruction resolution condition so as to train the network; and

and solving the three-dimensional refractive index distribution of the sample through the trained weight to realize the chromatographic reconstruction of the sample.

2. The method for generating variable-angle illumination tomography based on deep learning of the network according to claim 1, wherein the deep learning neural network propagated in the time domain and the frequency domain in a complex form is built according to the derived distribution model and the physical process, and further comprising:

taking the envelope complex amplitude a (r) of the distribution field of the light as a node of the neural network, and for the nodes a (x, y, z) and a (x, y, z + deltaz) of two adjacent layers, the operational relationship is divided into two parts corresponding to diffraction and refraction in the light propagation process:

wherein x, y and z are three-dimensional coordinates of the refractive index distribution of the sample body, and deltaz is the distance between adjacent layers in the optical axis direction in the layered model,

the operator of Fourier transform and the operator of inverse Fourier transform, omega_x、ω_yIn order to be the fourier domain coordinates,

is the wave number, n₀As the background medium refractive index, j is an imaginary unit, and δ n (r) is the weight to be trained.

3. The method for generating the variable-angle illumination tomography of the network based on the deep learning of claim 2 is characterized in that a framework used in the process of building the neural network is TensorFlow, and layers used in the network comprise an input layer, a diffraction layer, a refraction layer and a low-pass filter layer, wherein the diffraction layer and the refraction layer respectively correspond to the diffraction process operation and the refraction process operation, and the low-pass filter layer corresponds to the collected frequency domain characteristics.

4. The method for generating variable-angle illumination tomography of network based on deep learning as claimed in any one of claims 1-3, wherein the loss function in the network is expressed as follows:

loss＝∑|y_predct-y_true|+S，

wherein, y_predictRepresenting network-generated data, y_trueRepresenting the data of the real acquisition and S representing the sparse term constraint.

5. A variable-angle illumination tomography device based on deep learning generation network comprises:

the derivation module is used for deriving and obtaining a distribution model of a diffraction field and a refraction field when light propagates in the non-uniform transparent medium layer by layer according to a Helmholtz equation of the wave and a Fourier propagation model of the light, wherein the distribution model of the diffraction field and the refraction field is expressed as:

wherein x, y and z are three-dimensional coordinates of the refractive index distribution of the sample body, and deltaz is the distance between adjacent layers in the optical axis direction in the layered model,

the operator of Fourier transform and the operator of inverse Fourier transform, omega_x、ω_yIn order to be the fourier domain coordinates,

is the wave number, n₀Taking the refractive index of a background medium, j is an imaginary number unit, and δ n (r) is the weight to be trained;

the building module is used for building a deep learning generation neural network which propagates in a time domain and a frequency domain in a complex number form according to the deduced distribution model by imitating a physical process, wherein the weight to be trained is the bulk refractive index distribution of a sample to be chromatographed, and the training sample is the input light complex field distribution and the output light complex field distribution of corresponding angles;

the acquisition module is used for irradiating a plurality of groups of samples within a preset angle range, fixing a camera at the rear end of an optical axis for data acquisition, backward transmitting an acquired output light complex field which does not penetrate through the samples through an angular spectrum transmission formula to serve as input light complex field data, and using the acquired complex field which penetrates through the samples to be reconstructed as output light complex field data;

the training module is used for adjusting the deep learning network parameters according to the reconstruction resolution condition so as to train the network; and

and the reconstruction module is used for solving the three-dimensional refractive index distribution of the sample through the trained weight to realize the chromatographic reconstruction of the sample.

6. The variable-angle illumination tomography device based on deep learning generation network of claim 5, wherein the building module is further configured to use the envelope complex amplitude a (r) of the distribution field of light as a node of the neural network, and for the nodes a (x, y, z) and a (x, y, z + δ z) of two adjacent layers, the operational relationship is divided into two parts corresponding to diffraction and refraction in the light propagation process:

wherein x, y and z are three-dimensional coordinates of the refractive index distribution of the sample body, and deltaz is the distance between adjacent layers in the optical axis direction in the layered model,

the operator of Fourier transform and the operator of inverse Fourier transform, omega_x、ω_yIn order to be the fourier domain coordinates,

is the wave number, n₀As the background medium refractive index, j is an imaginary unit, and δ n (r) is the weight to be trained.

7. The deep learning network-based variable-angle illumination tomography device according to claim 6, wherein a framework used in the process of building the neural network is TensorFlow, and layers used in the network comprise an input layer, a diffraction layer, a refraction layer and a low-pass filter layer, wherein the diffraction layer and the refraction layer respectively correspond to the diffraction process operation and the refraction process operation, and the low-pass filter layer corresponds to the collected frequency domain characteristics.

8. The deep learning based network-generating variable-angle illumination tomography apparatus according to any one of claims 5 to 7, wherein in the network, the loss function is expressed as follows:

loss＝∑|y_predct-y_true|+S，

wherein, y_predictRepresenting network-generated data, y_trueRepresenting the data of the real acquisition and S representing the sparse term constraint.

Images