JP7579196B2 - Digital hologram signal processing device and digital hologram imaging/reproducing device - Google Patents

Description

The present invention relates to a digital hologram signal processing device and a digital hologram imaging and reproducing device, and to a digital hologram signal processing device and a digital hologram imaging and reproducing device that can create a desired reconstructed image whether coherent light or incoherent light is used.

Digital holography technology uses the interference of light to capture a hologram reflecting the three-dimensional information of a subject using optoelectronic devices such as an image sensor or photodetector, and stores this as a digital hologram in the memory of the signal processing unit. By displaying the digital hologram stored in this memory using a spatial light modulator, a three-dimensional image of the subject that can be viewed with the naked eye can be reproduced. In addition, by performing diffraction propagation calculations on the digital hologram in the signal processing unit, a two-dimensional image at any depth position can be reconstructed, and refocusing is possible to arbitrarily change the focal position after capturing the subject (see Non-Patent Document 1 below).

By using digital holography technology, it is possible to realize useful functions such as the above-mentioned stereoscopic imaging/display and refocusing, but most conventional digital holography technologies require the use of highly coherent laser light as a light source. However, advances in incoherent digital holography technology have alleviated the problem of spatial coherence required of light sources, making it possible to capture digital holograms using incoherent light such as sunlight, fluorescent light, and LEDs, and to realize the above-mentioned stereoscopic imaging/display and refocusing, and the range of applications of digital holography is steadily expanding (see Patent Documents 1 and 2 below).

Digital holography, which uses laser light (coherent light) as a light source (see Non-Patent Document 1 below), and incoherent digital holography, which uses incoherent light as a light source (see Patent Documents 1 and 2 below), use different coherence lengths of the light sources used and different configurations of the interference optical systems used to create digital holograms, but they share the same diffraction propagation calculations for reconstructing two-dimensional images, and in this case, Fresnel diffraction integrals and angular spectrum methods (or plane wave expansions) are commonly used. These diffraction calculations implicitly assume that the light has large temporal and spatial coherence, and the blurring of the reconstructed image indicates the response of the coherent light.
In incoherent digital holography, incoherent light is used during imaging, but the image reconstructed exhibits properties related to coherent light.
This response of coherent light is essential for reconstructing three-dimensional information using wavefront information of light, and does not cause any problems when reproducing three-dimensional images.

Therefore, the blurriness of an image reconstructed from a two-dimensional image in digital holography is completely different from the blurriness of an ordinary camera, such as a single-lens reflex camera or a smartphone, which is based on incoherent light. In other words, if a two-dimensional image is output using the ordinary method described above using digital holography, a technology for capturing three-dimensional images, users who are used to using ordinary cameras will find the blurriness of the two-dimensional image in digital holography to be unnatural.

Furthermore, it is known that humans normally judge the depth of an object and the order of distance in the depth direction based on the degree of blur (the degree of defocus) (see Non-Patent Documents 2 and 3 below), and conventional digital holographic imaging and playback devices are unable to provide a correct sense of depth from a two-dimensional image. Also, when video material is created by combining images captured by a digital holographic imaging and playback device with images captured by a normal camera other than a digital holographic imaging and playback device, inconsistencies arise.

Special Publication No. 2016-533542 JP 2020-060709 A

Georges Nehmetallah and Partha P. Banerjee, “Applications of digital and analog holography in three-dimensional imaging”, Advances in Optics and Photonics, (2012), vol. 4, pp. 472-553. Alex Paul Pentland, “A new sense for depth of field,” IEEE Transactions on pattern analysis and machine intelligence, (1987), vol. 4, pp. 523-531. George Mather and David R R Smith, “Blur discrimination and its relation to blur-mediated depth perception”, Perception, (2002), vol. 31, pp. 1211-1219.

When reconstructing a two-dimensional image using diffraction propagation calculations in conventional digital holography, the blurred state of the light distribution on the defocused plane away from the focus plane exhibits a response to coherent light, resulting in ringing and optical singularities, and it is not possible to obtain the smooth blur that can be obtained when using a normal camera.

As a result, not only is it impossible to obtain a correct sense of depth from the subject's depth position or depth order, but it is also impossible to synthesize the images with those captured by a normal camera. Furthermore, the amount of blur in an image in digital holography is determined by the distance between the imaging/reconstruction device and the subject.

The present invention has been made to solve these problems, and aims to provide a digital hologram signal processing device and a digital hologram imaging and reproducing device that can accurately obtain a sense of depth, such as the depth position and depth order of a subject, can be synthesized with an image captured by a normal camera, and can easily adjust the amount of blur of an image in digital holography.

The digital hologram signal processing device of the present invention comprises:
A digital hologram signal processing device performs signal processing for a digital hologram imaging and reproducing device that captures three-dimensional information of a subject by utilizing interference of light,
a focused image/depth position information acquisition means for acquiring a desired number of focused images of an object and information on the depth positions of the focused images from a reconstructed image of a digital hologram obtained by an interference optical system of the digital hologram imaging and reconstructing device;
a reference image acquisition means for setting each element of the depth of field, the focal length of the lens, the aperture diameter of the lens, and the distance from the lens to an image plane in accordance with a desired depth position within the depth of field so as to specify an in-area in-focus image set at the desired depth position within the depth of field, and acquiring a reference image by adding together a predetermined number of the in-area in-focus images thus obtained;
a defocused image group acquiring means for acquiring a defocused image group by calculating a convolution integral between an intensity distribution of each of the obtained out-of-area focused images and an intensity distribution of a point spread function of an incoherent imaging system, the defocused image group acquiring means setting each element of the focal length of the lens, the aperture diameter of the lens, and the distance from the lens to an image plane according to a desired depth position so as to specify an out-of-area focused image located at a desired depth position outside the depth of field, and the intensity distribution of each of the obtained out-of-area focused images and an intensity distribution of a point spread function of an incoherent imaging system;
a reference image/defocused image group adding means for adding the reference image obtained by the reference image obtaining means and the defocused image group obtained by the defocused image group obtaining means to each other to obtain an incoherent response image;
The present invention is characterized by comprising:

In addition, it is preferable that the calculation process for acquiring the reference image in the reference image acquisition means is a calculation process related to the convolution integral of a combined intensity distribution acquired by adding together the intensity distributions of each of the in-region focused images obtained by changing the depth position, and the intensity distribution of the point spread function of the incoherent imaging system.

但し、前記レンズから被写体の合焦像までの距離をｚ_１、該レンズから像面までの距離をｚ_２、該レンズの焦点距離をｆとする。 Furthermore, when performing the arithmetic processing relating to the convolution integral of the combined intensity distribution and the intensity distribution of a point spread function of the incoherent imaging system, it is preferable that parameters of the point spread function of the incoherent imaging system are set so as to satisfy the relationship of the following formula (A):

Here, the distance from the lens to the focused image of the subject is z ₁ , the distance from the lens to the image plane is z ₂ , and the focal length of the lens is f.

It is also preferable that the process of acquiring a desired number of focused images of the subject and information on the depth positions of the focused images in the focused image/depth position information acquisition means is performed based on the evaluation results of the sharpness of the reconstructed image of the digital hologram.

The digital hologram imaging and reproducing device of the present invention is characterized by comprising an imaging element, an imaging optical system that causes two systems of light, at least one of which carries subject information, to interfere with each other to form a hologram image on the imaging element, and any one of the digital hologram signal processing devices described above.

In addition, the digital hologram imaging and reproducing device preferably includes a coherent light source and an interferometer optical system that obtains a digital hologram using coherent light from the light source, and the digital hologram signal processing device preferably includes a memory that stores the depth resolution of the interferometer optical system. It is also preferable that the digital hologram signal processing device includes a self-interferometer optical system that obtains a digital hologram when incoherent light from a subject is incident on the self-interferometer optical system, and the digital hologram signal processing device preferably includes a memory that stores the depth resolution of the self-interferometer optical system.

The digital hologram signal processing device and digital hologram imaging and reproducing device of the present invention add together a reference image formed by adding together focused images obtained by adjusting reconstructed images located within the depth of field, and a group of defocused images formed by adjusting reconstructed images located outside the depth of field. This converts the blur characteristics of the image of the subject located at the defocused position from a coherent response to an incoherent response, and further allows the amount of blur to be controlled as desired by changing the parameters of the point spread function of the incoherent imaging system.

This makes it possible to reconstruct a two-dimensional image that shows the same blur as a normal camera, even from data captured using digital holography technology. It will also be possible to synthesize it with images captured using a normal camera.

In addition, it is possible to impart different amounts of blur depending on the depth position of the subject, and to freely control the amount of blur, something that was not possible with normal cameras or conventional digital holography. By adding blur, which is a feature of imaging, to an arbitrarily focused image, it is possible to create a dramatic effect.

1 is a conceptual diagram for mainly explaining the flow of signal processing related to blur conversion by a digital hologram signal processing device according to an embodiment of the present invention. 1 is a schematic diagram showing a configuration of an incoherent imaging system of a digital hologram imaging and reproducing apparatus according to an embodiment of the present invention. 1A is a schematic diagram showing an example of the configuration of a digital hologram imaging and reproducing device according to an embodiment of the present invention (FIG. 1B is a schematic diagram showing a case where a coherent light source is used as illumination light, and FIG. 1B is a schematic diagram showing a case where ambient incoherent light is used as illumination light). FIG. 2 is a block diagram showing an example of a digital hologram signal processing unit according to an embodiment of the present invention. 4A shows a subject image when the optical system of the device shown in FIG. 3A is used, and FIG. 4B shows a digital hologram obtained by capturing the subject image. These show two-dimensional images reconstructed based on the digital hologram shown in Figure 5(b) (reconstructed images when the deviation from the focal plane position is -20 mm (a), -10 mm (b), 0 mm (c), 10 mm (d), and 20 mm (e) respectively). The results of blur conversion using the technology of this embodiment are shown (the F-number (in-focus plane position) is 8.33, and the deviations from the in-focus plane position are -20 mm (a), -10 mm (b), 0 mm (c), 10 mm (d), and 20 mm (e)). The results of blur conversion using the technology of this embodiment are shown (the F-number (in-focus plane position) is 25, and the deviations from the in-focus plane position are -20 mm (a), -10 mm (b), 0 mm (c), 10 mm (d), and 20 mm (e)). The results of blur conversion using the technology of this embodiment are shown (the F-number (in-focus plane position) is 83.33, and the deviations from the in-focus plane position are -20 mm (a), -10 mm (b), 0 mm (c), 10 mm (d), and 20 mm (e)). In an embodiment of the present invention, when imaging multiple objects arranged in the depth direction, (a) shows the relative positions of three objects and (b) shows the digital hologram obtained by imaging the three objects. These represent the cases where the reconstructed image based on the digital hologram shown in Figure 10(b) is focused on (the case of the reconstructed image of object "1" (a), the case of the reconstructed image of object "2" (b), and the case of the reconstructed image of object "3" (c)). Using an embodiment technique of the present invention, the results of converting the blur of a reconstructed image based on a digital hologram shown in Figure 11(a) with the depth position of object "1" as the focal plane are shown ((a) is for D = 1.0 (F-number: 20), (b) is for D = 0.6 (F-number: 33.33), and (c) is for D = 0.2 (F-number: 100)). Using an embodiment technique of the present invention, the results of converting the blur of a reconstructed image based on a digital hologram shown in Figure 11(b) with the depth position of object "2" as the focal plane are shown ((a) is for D = 1.0 (F-number: 20), (b) is for D = 0.6 (F-number: 33.33), and (c) is for D = 0.2 (F-number: 100)). Using an embodiment technique of the present invention, the results of converting the blur of a reconstructed image based on a digital hologram shown in Figure 11(c) with the depth position of object "3" as the focal plane are shown ((a) is for D = 1.0 (F-number: 20), (b) is for D = 0.6 (F-number: 33.33), and (c) is for D = 0.2 (F-number: 100)).

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS A digital hologram signal processing device and a digital hologram imaging and reproducing device according to embodiments of the present invention will be described below with reference to the drawings.
The flow of signal processing of the present invention is shown in Fig. 1. Note that each step after step 12 (S12) is performed in a digital hologram signal processing device (hereinafter also referred to as signal processing units 18, 18': see Figs. 3(a) and 3(b)), specifically, by using a computer program, and is displayed on a monitor.
First, a digital hologram is captured (S11) using an interference optical system (described later) of the digital hologram capture and reconstructor, and then stored. Based on the captured digital hologram data, depth position information and focused images of each subject (subjects 1 to 4) are acquired in the following procedure (S12): <1> indicates the focused images and depth positions of each of subjects 1 to 4.

That is, in this step 12 (S12), a calculation of diffraction propagation is applied to the intensity distribution of the digital hologram or the complex amplitude distribution extracted from the digital hologram by the phase shift method or the Fourier fringe analysis method. For the calculation of diffraction propagation, a method such as Fresnel diffraction integral or angular spectrum method (plane wave expansion) can be used. When calculating diffraction propagation, the propagation distance is changed stepwise and sequentially to obtain a two-dimensional reconstructed image at each propagation distance. In these two-dimensional reconstructed images, the image becomes focused or blurred depending on the change in the depth position of the subject, that is, depending on the propagation distance. When a focused image is formed, the propagation distance at that time corresponds to the depth position of the subject. Therefore, by searching for the most focused image from the two-dimensional reconstructed images at each propagation distance, the depth position of the subject can be obtained. In addition, the sharpness of the two-dimensional reconstructed image can be evaluated to search for the most focused image. The sharpness can be evaluated by comparing the differential of each image or the magnitude of the spatial frequency components of each image. These techniques are known as autofocus techniques in the field of digital holography.

The sharpness of each of the obtained two-dimensional reconstructed images is evaluated, and the propagation distance at which the sharpness is highest is taken as the depth position of the subject. This depth position information and the focused image of the subject are acquired and stored in the signal processing units 18, 18' (see Figure 3). Note that the depth position of the subject may also be acquired by using a device such as a lidar in combination.

Next, the signal processors 18, 18' (see FIG. 3) are assigned with information on the depth position to be focused on, information on the desired depth of field centered on the depth position, and parameters of the incoherent imaging system corresponding to the amount of blur to be realized (S13). The depth of field is set to a value greater than the depth resolution of the optical system in the digital hologram imaging and reproducing device 19, 19'.
This allows the response of the imaging system to be calculated physically correctly, and unnecessary calculation processing can be omitted. Furthermore, information on the depth resolution of the optical system is stored in the memory of the signal processing unit 18, 18', reflecting information on the configuration of the optical system.

ここで，P(x,y)はレンズの開口関数であり、ｘ、ｙは瞳面の空間領域の座標系である。
本実施形態においては、通常のカメラレンズを想定して、レンズの開口関数を直径Ｄの円形開口とするが、アポダイゼーション、矩形開口やランダムな開口をはじめ、付与したいぼやけ状態に応じて自由に設定してもよい。λは光源の波長であり、ｆ_ｘ、ｆ_ｙは空間周波数領域の座標系の変数である。 2, there are four parameters of the incoherent imaging system: focal length f of virtual lens 6, aperture diameter D, distance _z1 from virtual lens (hereinafter simply referred to as lens) 6 to the focused images of subjects 1 to 4, and distance _z2 from virtual lens 6 to image plane 7. Based on these four parameters, the point spread function of the incoherent imaging system is formed as shown in the following equation (1).

Here, P(x, y) is the aperture function of the lens, and x, y are the coordinate system of the spatial domain of the pupil plane.
In this embodiment, assuming a normal camera lens, the aperture function of the lens is a circular aperture with a diameter D, but it may be freely set according to the desired blur state, including apodization, a rectangular aperture, and a random aperture. λ is the wavelength of the light source, and f _x and f _y are variables of the coordinate system in the spatial frequency domain.

Using the parameters set as described above, a two-dimensional image with blur conversion is generated in the following procedure.
As shown in FIG. 2, the in-focus images of subjects 2 and 3 existing within the region of the set depth position information and depth of field are added together to obtain an added image I (S14: <2> shows the added image I of the in-focus images within the depth of field).
Then, using the following equation (2), a reference image B is obtained by convolution integral of the added image I and the point spread function (PSF) of the incoherent imaging system (S15: reference image B is shown in <3>).

When performing the calculation process of the above equation (2), by setting the parameters of the point spread function of the above equation (1) so as to satisfy the relationship of the below equation (3), it is possible to obtain an image as if the subjects 2 and 3 were focused on by a normal camera.

On the other hand, since the effect of P(x, y) in the above equation (1), that is, the limitation of the finite aperture diameter of the lens, also includes an effect of reducing the resolution, if it is desired to suppress the reduction in the resolution, the PSF in the above equation (2) can be calculated as a delta function, or this calculation itself can be omitted and B = I can be used.
In addition, when calculating the above formulas (1) and (2), a method using a Fourier transform algorithm is generally used. In that case, in order to prevent an increase or decrease in energy depending on the algorithm, energy is normalized and care is taken to ensure that the intensity is constant.

ここで、ｄは被写界深度の領域外の被写体の奥行距離である。
なお、上式（１）に対して上式（４）が相違するのは、ｚ_１がｄに置き換わっている点である。この演算を、被写界深度の領域外の被写体１、４毎にｄの値を変化させて適用する。 A defocused image Dd of the subjects 1 and 4 outside the depth of field is created separately from the creation of the reference image B described above. When creating the defocused image Dd, the following equations (4) and (5) are applied.
That is, a defocused image Dd is obtained by convolution integral of each focused image of the subjects 1 and 4 outside the depth of field with the point spread function of the imaging system (S16: the defocused image Dd outside the depth of field is shown in <4>).

Here, d is the depth distance of the subject outside the depth of field.
The difference between the above formula (4) and the above formula (1) is that _z1 is replaced with d. This calculation is applied by changing the value of d for each of the subjects 1 and 4 outside the depth of field.

Finally, by adding the reference image B and all the defocused images (the group of defocused images Dd), a two-dimensional image with a converted blur state can be obtained (S17): <5> shows the two-dimensional image obtained as a result of signal processing (by adding all the images together).

In the above calculations, the value of f and the value of _z2 are fixed in order to impart a blur state to all of the subjects 1 to 4 when a common lens (virtual lens) 6 is used. However, both of these may be changed depending on the subjects 1 to 4.
The depth position information of each of the subjects 1 to 4 may also be changed to an arbitrary value. By performing the above operations, it is possible to obtain a two-dimensional image that is blurred in a way that cannot be captured by a normal camera, and since the depth position information can also be arbitrarily replaced, the range of video expression can be expanded.

As a digital hologram imaging and reproducing device that realizes the above-mentioned signal processing for blur conversion, for example, the devices shown in FIGS.
The optical system of the imaging and reproducing device shown in Figures 3(a) and (b) is a typical configuration in the field of digital holography. However, a feature of this imaging and reproducing device is that, in addition to being equipped with signal processing units 18, 18' that perform signal processing for blur conversion, information on the depth resolution of the optical system, which is necessary to increase the efficiency of calculations for the signal processing of the blur conversion, is stored and set in the signal processing units 18, 18'.

Figure 3(a) shows a digital hologram imaging and reproducing device 19 when coherent light is used as the light source. Laser light from a laser light source 16 in the digital hologram imaging and reproducing device 19 is converted into a plane wave by a spatial filter 15 and a lens 11, and then split by a beam splitter 13. One of the split lights is irradiated onto an object 12, and the other split light is reflected by a plane mirror 14. The former light is the object light and the latter light is the reference light. After being combined by the beam splitter 13, the two are made to interfere on the surface of the imaging element 17. This is captured as a digital hologram and stored in the memory section (not shown) of the signal processing section 18.

On the other hand, Figure 3(b) shows a digital hologram imaging and reproducing device 19' when ambient incoherent light is used as the light source. The reflected light from the subject 12' illuminated with ambient incoherent light is collected by lens 11' and split by beam splitter 13'. The split incoherent light is reflected by plane mirror 14' and concave mirror 20, returned to beam splitter 13' and combined, and made to interfere on the surface of image sensor 17'. This is imaged as a digital hologram by image sensor 17' and stored in the memory section (not shown) of signal processing section 18'.

Figure 4 is a block diagram showing the internal configuration of the signal processing unit 18 (similar to the signal processing unit 18'). That is, the signal processing unit 18 includes a focused image/depth position information acquisition means 18A, a reference image acquisition means 18B, a defocused image group acquisition means 18C, and a reference image/defocused image group addition means 18D.

First, the focused image/depth position information acquisition means 18A is configured to acquire the focused image of the subject 12 (for each of subject images 1 to 4) and the depth position information of the focused image from the reconstructed image of the digital hologram obtained by the interference optical system of the digital hologram imaging and reproduction device 19, 19'.

Furthermore, the reference image acquisition means 18B sets each element of the depth of field, the focal length f of the lens 6, the aperture diameter D of the lens 6, and the distance _z2 from the lens 6 to the image plane 7 according to the depth position so as to specify an in-area in-focus image that is set at a desired depth position within the area of the depth of field, and acquires a reference image B by adding together the intensity distributions of a predetermined number of in-area in-focus images thus obtained.

The calculation process for acquiring the reference image B in the reference image acquisition means 18B is preferably a calculation based on the convolution integral of the intensity distribution of the point spread function of the incoherent imaging system and the combined intensity distribution obtained by adding together the intensity distributions of the in-region focused images (focused images of subjects 2 and 3) obtained by sequentially changing the depth position setting value.

On the other hand, the defocused image group acquisition means 18C sets each element, i.e., the focal length f of the lens 6, the aperture diameter D of the lens 6, and the distance z2 from the lens 6 to the image plane, in accordance with the depth position so as to be able to specify an out-of-area focused image (focused image of subjects 1, ₄₎ that is located at a desired depth position outside the depth of field, and calculates the convolution integral of the intensity distribution of each of the obtained out-of-area focused images (focused images of subjects 1, 4) and the intensity distribution of the point spread function of the incoherent imaging system to obtain a defocused image group Dd.
In other words, this defocused image group acquisition means 18C converts the blur state of the image of the subject 12 present at the defocused position from a coherent response to an incoherent response, and further adjusts the amount of blur by changing the parameters of the point spread function of the incoherent imaging system.

Furthermore, the reference image/defocused image group addition means 18D adds the reference image B obtained by the reference image acquisition means 18B and the defocused image group Dd obtained by the defocused image group acquisition means 18C to each other to obtain an incoherent response image.
Although not shown in FIG. 4, the signal processing unit 18 is provided with a memory unit for storing the depth resolution of the optical system of the interferometer, etc.

Note that the optical system of the digital hologram imaging and reproducing device 19, 19' shown in Figures 3(a) and (b) represents one example of the optical system of the digital hologram imaging and reproducing device according to the present invention, and various other configurations can be adopted as long as they are capable of imaging a digital hologram using the interference phenomenon and deriving depth position information of the optical system.

Below, the reconstruction results obtained by the embodiments of the present invention will be explained, but the contents and results of a simulation relating to the prior art will also be explained as a comparative example.
In the simulations according to this embodiment and the comparative example, the diffraction of light waves is calculated by the angular spectrum method, and the propagation of light can be calculated accurately.
The digital hologram imaging and reproducing device used was one using the coherent light shown in Fig. 3(a). The wavelength of the coherent light was 633 nm, the subject 12 was a two-dimensional image shown in Fig. 5(a), and the depth position of the subject 12 was 100 mm. The reference light was a plane wave having a planar phase. The imaging element 17 used had a pixel count of 256 x 256, a pixel pitch of 10 μm, and gradation of 8 bits. Here, the hologram image acquired by the imaging element in the comparative example was similar to that shown in Fig. 5(b).

From this hologram obtained by the imaging element, a complex amplitude distribution was extracted by the phase shift method, and a reconstructed image is shown in Figure 6. A two-dimensional image is obtained at each position while changing the propagation distance at 10 mm intervals in an area within a range of ±20 mm centered on the focus position of the subject. That is, the reconstruction result when the set propagation distance is 0 mm (the focus position of the subject) is shown in (c), and the reconstruction results when the set propagation distance is -20 mm, -10 mm, 10 mm, and 20 mm are shown in (a), (b), (d), and (e), respectively.
From the reconstruction result in FIG. 6, it is clear that in the reconstruction calculation of ordinary digital holography, ringing occurs in the defocused image, and the blur state is different from that of an ordinary camera.

Next, the result of reconstruction by applying the technique of this embodiment is shown in Fig. 7. The parameters of the incoherent imaging system and the virtual lens 6 are _z1 = 100 mm, _z2 = 100 mm, f = 50 mm, D = 6 mm (F value 8.33), and the length of the depth of field region is 8 mm. It is clear that the defocused image is smoother by applying the technique of this embodiment compared to the case of Fig. 6.
Furthermore, the processing results when D = 2 mm (F-number 25) and D = 0.6 mm (F-number 83.33) are shown in Figures 8 and 9 (the propagation distances for (a) to (e) are the same as in Figure 6). It is clear that the blur amount of the defocused image is reduced by increasing the F-number, and the blur amount is well controlled.

Next, the results of application when three subjects 12 are placed 60 mm apart in the depth direction as shown in Figure 10(a) will be described below. When these three subjects 12 are imaged, the digital hologram shown in Figure 10(b) is obtained. Hereinafter, for convenience, the three subjects 12 will be referred to as "Object 1", "Object 2" and "Object 3" in the depth direction in Figure 10(a).

The results of applying the phase shift method and the diffraction propagation calculation to the digital hologram of Fig. 10(b) are shown in Fig. 11(a) to (c). When each subject 12 is focused and reconstructed, it is clear that a defocused image with the blur (having a striped pattern) characteristic of coherent light is obtained.
The reconstruction results obtained by applying the technique of this embodiment to the reconstruction results shown in FIGS. 11(a) to 11(c) are shown in FIGS. 12, 13, and 14. FIG.

12, 13, and 14 respectively show the reconstruction results when the positions of objects "1", "2", and "3" are set as the depth positions of the focus and the depth of field is set to 10 mm.
In addition, for each of the amounts of blur, (a) in Figures 12 to 14 shows the results when the aperture diameter D is 1 (F-number 20), (b) in Figures 12 to 14 shows the results when D is 0.6 (F-number 33.33), and (c) in Figures 12 to 14 shows the results when D is 0.2 (F-number 100).
As is clear from the reconstruction results described above, by using the technology of this embodiment, it is possible to convert the blur state from a state using coherent light to a state using incoherent light, and further, it is clear that the amount of blur can be adjusted as desired.

Note that the simulation results described above are for the case where a digital holographic imaging and reproducing device 19 using coherent light is used, but the same effects can be achieved even in the case of a digital holographic imaging and reproducing device 19' using incoherent light as shown in Figure 3(b).

The digital hologram signal processing device and digital hologram imaging and reproducing device of the present invention are not limited to the above-mentioned embodiments, and various other modifications are possible.

1-4, 12, 12' Subject 6 Virtual lens (lens)
Reference image acquisition means 18C; Blur amount adjustment means 18D; Reference image/defocused image group adding means 19, 19'; Digital hologram imaging and reproducing device 20; Concave mirror B; Reference image Dd; Defocused image group I; Addition image

Claims (7)

A digital hologram signal processing device performs signal processing for a digital hologram imaging and reproducing device that captures three-dimensional information of a subject by utilizing interference of light,
a focused image/depth position information acquisition means for acquiring a desired number of focused images of an object and information on the depth positions of the focused images from a reconstructed image of a digital hologram obtained by an interference optical system of the digital hologram imaging and reconstructing device;
a reference image acquisition means for setting each element of the depth of field, the focal length of the lens, the aperture diameter of the lens, and the distance from the lens to an image plane in accordance with a desired depth position within the depth of field so as to specify an in-area in-focus image set at the desired depth position within the depth of field, and acquiring a reference image by adding together a predetermined number of the in-area in-focus images thus obtained;
a defocused image group acquiring means for acquiring a defocused image group by calculating a convolution integral between an intensity distribution of each of the obtained out-of-area focused images and an intensity distribution of a point spread function of an incoherent imaging system, the defocused image group acquiring means setting each element of the focal length of the lens, the aperture diameter of the lens, and the distance from the lens to an image plane according to a desired depth position so as to specify an out-of-area focused image located at a desired depth position outside the depth of field, and the intensity distribution of each of the obtained out-of-area focused images and an intensity distribution of a point spread function of an incoherent imaging system;
a reference image/defocused image group adding means for adding the reference image obtained by the reference image obtaining means and the defocused image group obtained by the defocused image group obtaining means to each other to obtain an incoherent response image;
A digital hologram signal processing device comprising: The digital hologram signal processing device according to claim 1, characterized in that the calculation process for acquiring the reference image in the reference image acquisition means is a calculation process related to the convolution integral of a combined intensity distribution acquired by adding together the intensity distributions of each of the in-region focused images obtained by changing the depth position, and the intensity distribution of the point spread function of the incoherent imaging system. The digital hologram signal processing device according to claim 2, characterized in that, when performing the arithmetic processing relating to the convolution integral of the combined intensity distribution and the intensity distribution of the point spread function of the incoherent imaging system, parameters of the point spread function of the incoherent imaging system are set so as to satisfy the relationship of the following equation (A).

Here, the distance from the lens to the focused image of the subject is z ₁ , the distance from the lens to the image plane is z ₂ , and the focal length of the lens is f. A digital hologram signal processing device according to any one of claims 1 to 3, characterized in that the process of acquiring a desired number of focused images of the subject and information on the depth positions of the focused images in the focused image/depth position information acquisition means is performed based on the evaluation results of the sharpness of the reconstructed image of the digital hologram. A digital hologram imaging and reproducing device comprising an imaging element, an imaging optical system that causes two systems of light, at least one of which carries subject information, to interfere with each other to form a hologram image on the imaging element, and a digital hologram signal processing device according to any one of claims 1 to 4. The digital hologram imaging and reproducing device according to claim 5, further comprising a coherent light source and an interferometer optical system that obtains a digital hologram using the coherent light from the light source, and further comprising a memory in the digital hologram signal processing device that stores the depth resolution of the interferometer optical system. The digital hologram imaging and reproducing device according to claim 5, further comprising a self-interferometer optical system that obtains a digital hologram when incoherent light from a subject is incident thereon, and further comprising a memory within the digital hologram signal processing device that stores the depth resolution of the self-interferometer optical system.

Images