JP6987831B2 - Optical instruments and image formation methods - Google Patents

Description

The present invention relates to optical instruments and image forming methods.

For example, in an optical device such as an optical microscope, various functions such as improvement of resolution and correction of aberration are realized by using phase modulation of light. Patent Document 1 describes a phase modulator that modulates the phases of light fluxes having a plurality of wavelengths different from each other within a predetermined wide spectrum width. This phase modulator diffracts the luminous flux emitted from the light source by a diffraction grating at a different angle for each wavelength. The luminous flux diffracted for each wavelength enters the spatial phase modulator via the condenser lens. The phase modulator has a plurality of phase modulation regions corresponding to each wavelength, and the light beam of each wavelength is incident on the phase modulation region corresponding to each wavelength by diffraction by the diffraction grating.

Japanese Unexamined Patent Publication No. 2010-25922

In the technique of Patent Document 1, in order to make a light flux incident on a phase modulation region corresponding to each wavelength, it is required to diffract the light flux by a diffraction grating, a prism or the like. Therefore, there are problems that the optical system becomes complicated and the configuration for phase modulation becomes complicated.
Further, since the phase modulator of Patent Document 1 has a plurality of phase modulation regions for each wavelength, there is a problem that the phase modulator becomes large in size.

The first aspect of the present invention relates to an optical instrument. The optical device according to this embodiment has a common phase modulation mask that imparts phase modulation to light having a first wavelength and light having a second wavelength, and light having a first wavelength and light having a second wavelength. The irradiation optical system incident on the same incident region on the phase modulation mask, and the light of the first wavelength and the light of the second wavelength phase-modulated by the phase modulation mask are condensed and subjected to the point image distribution function. It includes a condensing optical system that forms an image on the imaging surface. Here, the phase modulation mask is a phase plate, and the phase plate has an optimum thickness T1 for forming an image corresponding to light of the first wavelength on the imaging surface at each incident position in the same incident region. , The thickness is between T2, which is the optimum thickness for forming an image corresponding to the light of the second wavelength on the imaging surface, and both the light of the first wavelength and the light of the second wavelength are points. have a thickness T3 which forms an image corresponding to the image distribution function, the thickness T3 is in the following formulas, b / a is set to be in the range of 2/8 or more 8/2 or less.
T3 = (T1 × a + T2 × b) / (a + b)
In the above equation, both a and b are positive real numbers.

A second aspect of the present invention relates to a method of forming an image corresponding to a point spread function from light having a first wavelength and light having a second wavelength. The method for forming an image according to this embodiment is to apply the light of the first wavelength and the light of the first wavelength to the same incident region on a common phase plate that applies phase modulation to the light of the first wavelength and the light of the second wavelength. Light of two wavelengths is incident, and the light of the first wavelength and the light of the second wavelength phase-modulated by the phase plate are condensed to form an image corresponding to the point image distribution function on the imaging surface. Phase plate, at each incident position of the same incident region, the optimum thickness T1 in forming an image according to the light of the first wavelength on an imaging plane, an image corresponding to the light of the second wavelength the optimum thickness T2 to form an image on the imaging surface, a thickness of between, and the thickness both of the first wavelength light and the second wavelength light forms an image in accordance with the point spread function T3 have a thickness T3, in the following formulas, b / a is set to be in the range of 2/8 or more 8/2 or less.
T3 = (T1 × a + T2 × b) / (a + b)
In the above equation, both a and b are positive real numbers.

According to the present invention, a plurality of fluorescence phases can be modulated with a simple configuration.

FIG. 1 is a schematic diagram showing a configuration of an optical device provided with a phase modulation mask according to an embodiment. FIG. 2A is a schematic diagram showing that the two bright spot images according to the embodiment rotate on the imaging surface according to the positions of the emission points of fluorescence in the Z-axis direction. FIG. 2B is a diagram showing a DH-PSF according to an embodiment. FIG. 3A is a schematic diagram showing that all the first fluorescent dyes according to the embodiment are in an active state. FIG. 3B is a schematic diagram showing that all the first fluorescent dyes according to the embodiment are in the inactive state. 3 (c) and 3 (d) are schematic views showing that a part of the first fluorescent dye according to the embodiment is in an active state. FIG. 3 (e) is a schematic diagram showing that all the second fluorescent dyes according to the embodiment are in an active state. FIG. 3 (f) is a schematic diagram showing that all the second fluorescent dyes according to the embodiment are in the inactive state. 3 (g) and 3 (h) are schematic views showing that a part of the second fluorescent dye according to the embodiment is in an active state. 4 (a) to 4 (e) are schematic views showing a procedure for acquiring a first three-dimensional super-resolution image according to an embodiment. FIG. 4 (f) is a schematic diagram showing a procedure for acquiring the number of the first substance according to the embodiment. FIG. 5A shows a first three-dimensional super-resolution image acquired from a plurality of first two-dimensional images according to an embodiment, and a second three acquired from a plurality of second two-dimensional images. It is a schematic diagram which shows the dimensional super-resolution image. FIG. 5B is a schematic diagram showing first and second three-dimensional super-resolution images acquired from a plurality of common two-dimensional images according to a modified example of the embodiment. FIG. 6A is a schematic diagram showing the configuration of an optical device including the phase modulator according to the embodiment. FIG. 6B is a schematic diagram showing the configuration of the phase modulator according to the embodiment. 7 (a) and 7 (b) are diagrams showing a first phase modulation pattern and a second phase modulation pattern set in the phase modulator according to the embodiment, respectively. 8 (a) and 8 (b) are diagrams showing an image formation state and a graph when the first fluorescence is observed using the first phase modulation pattern according to the embodiment. 8 (c) and 8 (d) are diagrams showing an image formation state and a graph when the second fluorescence is observed using the first phase modulation pattern according to the embodiment. 8 (e) and 8 (f) are diagrams showing an image formation state and a graph when the first fluorescence is observed using the second phase modulation pattern according to the embodiment. 8 (g) and 8 (h) are diagrams showing an image formation state and a graph when the second fluorescence is observed using the second phase modulation pattern according to the embodiment. FIG. 9 is a diagram showing the configuration of the phase modulation pattern of the first embodiment. FIG. 10 is a diagram showing an image formation state of the first fluorescence when the phase modulation pattern of Example 1 is used. FIG. 11 is a diagram showing an image formation state of the second fluorescence when the phase modulation pattern of Example 1 is used. 12 (a) and 12 (b) are diagrams showing an image formation state and a graph when the first fluorescence is observed using the phase modulation pattern of Example 1. 12 (c) and 12 (d) are diagrams showing an image formation state and a graph when the second fluorescence is observed using the phase modulation pattern of Example 1. 13 (a) to 13 (c) are diagrams for explaining in detail the integration of the first phase modulation pattern and the second phase modulation pattern according to the phase modulation pattern of the first embodiment. 13 (a) to 13 (c) are diagrams schematically showing one pixel position of the liquid crystal panel in the first phase modulation pattern, the second phase modulation pattern, and the phase modulation pattern of Example 1, respectively. be. FIG. 14A is a schematic diagram showing the arrangement of regions in the phase modulation pattern of the second embodiment. 14 (b) to 14 (d) are schematic views showing the configuration of the phase modulation pattern of the second embodiment. 15 (a) and 15 (b) are diagrams showing an image formation state and a graph when the first fluorescence is observed using the phase modulation pattern of Example 2. 15 (c) and 15 (d) are diagrams showing an image formation state and a graph when the second fluorescence is observed using the phase modulation pattern of Example 2. 15 (e) and 15 (f) are diagrams showing an image formation state when the first fluorescence and the second fluorescence are observed using the phase modulation pattern of the second embodiment. 16 (a) and 16 (b) are diagrams showing the results of observing the first fluorescence and the second fluorescence in a wide field of view using the phase modulation pattern of Example 2, respectively. FIG. 17A is a schematic diagram showing the arrangement of regions in the phase modulation pattern of the third embodiment. 17 (b) to 17 (d) are schematic views showing the configuration of the phase modulation pattern of the third embodiment. 18 (a) and 18 (b) are diagrams showing an image formation state and a graph when the first fluorescence is observed using the phase modulation pattern of Example 3. 18 (c) and 18 (d) are diagrams showing an image formation state and a graph when the second fluorescence is observed using the phase modulation pattern of Example 3. 18 (e) and 18 (f) are diagrams showing an image formation state when the first fluorescence and the second fluorescence are observed using the phase modulation pattern of Example 3. 19 (a) and 19 (b) are diagrams showing the results of observing the first fluorescence and the second fluorescence in a wide field of view using the phase modulation pattern of Example 3, respectively. FIG. 20A is a schematic diagram showing the arrangement of regions in the phase modulation pattern of the fourth embodiment. 20 (b) to 20 (d) are schematic views showing the configuration of the phase modulation pattern of the fourth embodiment. 21 (a) and 21 (b) are diagrams showing an image formation state and a graph when the first fluorescence is observed using the phase modulation pattern of Example 4. 21 (c) and 21 (d) are diagrams showing an image formation state and a graph when the second fluorescence is observed using the phase modulation pattern of Example 4. 21 (e) and 21 (f) are diagrams showing an image formation state when the first fluorescence and the second fluorescence are observed using the phase modulation pattern of Example 4. 22 (a) and 22 (b) are diagrams showing the results of observing the first fluorescence and the second fluorescence in a wide field of view using the phase modulation pattern of Example 4, respectively. FIG. 23A is a schematic diagram showing the arrangement of regions in the phase modulation pattern of the fifth embodiment. 23 (b) to 23 (e) are schematic views showing the configuration of the phase modulation pattern of the fifth embodiment. 24 (a) and 24 (b) are diagrams showing an image formation state and a graph when the first fluorescence is observed using the phase modulation pattern of Example 5. 24 (c) and 24 (d) are diagrams showing an image formation state and a graph when the second fluorescence is observed using the phase modulation pattern of Example 5. 24 (e) and 24 (g) are diagrams showing an image formation state when the first fluorescence is observed using the phase modulation pattern of Example 5. 24 (f) and 24 (h) are diagrams showing an image formation state when the second fluorescence is observed using the phase modulation pattern of Example 5. 25 (a) and 25 (b) are diagrams showing the results of observing the first fluorescence and the second fluorescence in a wide field of view using the phase modulation pattern of Example 5, respectively. FIG. 26 is a diagram showing the configuration of the phase modulation pattern of the sixth embodiment. 27 (a), (c), (e), and (g) are diagrams showing an image formation state when the first fluorescence is observed using the phase modulation pattern of Example 6. 27 (b), (d), (f), and (h) are diagrams showing an image formation state when the second fluorescence is observed using the phase modulation pattern of Example 6. 28 (a), (c), (e), and (g) are diagrams showing an image formation state when the first fluorescence is observed using the phase modulation pattern of Example 6. 28 (b), (d), (f), and (h) are diagrams showing an image formation state when the second fluorescence is observed using the phase modulation pattern of Example 6. FIG. 29A is a schematic diagram showing a phase plate manufactured so as to correspond to the first phase modulation pattern according to the embodiment. FIG. 29B is a schematic diagram showing a phase plate manufactured so as to correspond to the second phase modulation pattern according to the embodiment. FIG. 29 (c) is a schematic diagram showing an integrated phase plate similar to the phase modulation pattern of the first embodiment. 29 (d) to 29 (f) are schematic views showing a cut surface when the phase plate according to the embodiment is cut in a plane parallel to the thickness direction. FIG. 30A is a schematic diagram showing a phase plate manufactured so as to correspond to the first phase modulation pattern according to the embodiment. FIG. 30B is a schematic diagram showing a phase plate manufactured so as to correspond to the second phase modulation pattern according to the embodiment. FIG. 30C is a schematic diagram showing an integrated phase plate similar to the phase modulation pattern of the second embodiment. 30 (d) to 30 (f) are diagrams schematically showing a region when the phase plate according to the embodiment is viewed in the thickness direction. 30 (g) to 30 (i) are schematic views showing a cut surface when the phase plate according to the embodiment is cut in a plane parallel to the thickness direction.

The following embodiment is an application of the present invention to an optical instrument for observing two types of fluorescence having different center wavelengths. The optical device of the embodiment is a fluorescence microscope that irradiates a sample with light and captures the fluorescence generated from the sample by an imaging unit. The optical device to which the present invention can be applied is not limited to the following embodiments, and may be a microscope other than a fluorescence microscope, an imaging device such as a camera, a telescope, an endoscope, a planetarium, or the like. Further, the optical device to which the present invention can be applied is not limited to a device for capturing and observing fluorescence, but may be a device for capturing and observing light other than fluorescence.

As shown in FIG. 1, the optical device 10 includes an irradiation optical system 30, a beam expander 37, a stage 40, a phase modulation mask 50, a condensing optical system 60, controllers 71 and 72, and an information processing device. It is equipped with 100 and. FIG. 1 shows XYZ axes that are orthogonal to each other.

A slide glass 41 on which a sample is placed is installed on the stage 40. The sample of the embodiment is a biological sample containing a test cell. The test cells are collected, for example, from lesioned tissue. The nucleus of the test cell contains the first substance and the second substance. The first substance and the second substance to be imaged are, for example, biological substances such as genes, proteins or peptides that serve as disease markers. The first substance of the embodiment is the HER2 gene, and the second substance of the embodiment is CEP17, which is the centromere region of chromosome 17.

By preparing the sample, a fluorescent substance is bound to each of the first substance and the second substance. In the embodiment, the fluorescent substance bound to the first substance is the first fluorescent dye, and the fluorescent substance bound to the second substance is the second fluorescent dye. Further, by preparing the sample, the nucleus of the test cell is specifically stained with the third fluorescent dye.

The first fluorescent dye has an active state in which the first fluorescence having the first wavelength as the central wavelength is generated when the light from the light source 21 described later is irradiated, and the first fluorescence even when the light from the light source 21 is irradiated. It is possible to switch to an inactive state that does not occur. The first fluorescent dye is inactivated when irradiated with light from the light source 21, and is activated when irradiated with light from the light source 23 described later. The second fluorescent dye has an active state that produces second fluorescence with the second wavelength as the central wavelength when irradiated with light from the light source 22, which will be described later, and second fluorescence even when light from the light source 22 is irradiated. It is possible to switch to an inactive state that does not occur. The second fluorescent dye is inactivated when irradiated with light from the light source 22, and is activated when irradiated with light from the light source 23. When the third fluorescent dye is irradiated with light from the light source 23, the third fluorescent dye produces a third fluorescence having a third wavelength as a central wavelength.

The substance to be imaged is not limited to the fluorescent dye that binds to the substance as described above, and may be a substance that causes autofluorescence. The sample placed on the slide glass 41 is not limited to the biological sample. The substance to be imaged is not limited to the substance contained in the biological sample, and may be a substance not derived from the biological sample. For example, the substance to be imaged may be a fluorescent substance such as particles that generate fluorescence, such as fluorescent beads.

The irradiation optical system 30 includes a light source unit 20, a shutter 31, a 1/4 wave plate 32, a beam expander 33, a condenser lens 34, a dichroic mirror 35, and an objective lens 36. The irradiation optical system 30 irradiates the sample with light to generate the first to third fluorescence from the fluorescently labeled first to third substances contained in the sample, respectively, and causes the first fluorescence and the second fluorescence, respectively. , Is incident on the same incident region in the phase modulation mask 50.

The light source unit 20 includes light sources 21 to 23, mirrors 24, and dichroic mirrors 25 and 26.

Each of the light sources 21 to 23 emits light having a different wavelength. Specifically, the wavelengths of the light emitted from the light sources 21, 22 and 23 are 640 nm, 488 nm and 405 nm, respectively. As the light sources 21 to 23, it is preferable to use a laser light source, but a mercury lamp, a xenon lamp, an LED or the like may be used. As described above, the light emitted from the light source 21 excites the first fluorescent dye contained in the test cells to generate the first fluorescence and inactivates the first fluorescent dye. The light emitted from the light source 22 excites the second fluorescent dye contained in the test cells to generate the second fluorescence, and at the same time, inactivates the second fluorescent dye. The light emitted from the light source 23 excites the third fluorescent dye contained in the test cells to generate the third fluorescence, and activates the first fluorescent dye and the second fluorescent dye. In the embodiment, the first wavelength, which is the center wavelength of the first fluorescence, is 690 nm, and the second wavelength, which is the center wavelength of the second fluorescence, is 530 nm.

The mirror 24 reflects the light from the light source 21. The dichroic mirror 25 transmits the light from the light source 21 and reflects the light from the light source 22. The dichroic mirror 26 transmits the light from the light sources 21 and 22 and reflects the light from the light source 23. The optical axes of the light from the light sources 21 to 23 are aligned with each other by the mirror 24 and the dichroic mirrors 25 and 26. Instead of the light sources 21 to 23, one light source may emit light having wavelengths of 640 nm, 488 nm, and 405 nm.

The shutter 31 is driven by the controller 71 and switches between a state in which the light emitted from the light source unit 20 passes through and a state in which the light emitted from the light source unit 20 is blocked. As a result, the irradiation time of light on the test cells is adjusted. The 1/4 wave plate 32 converts the linearly polarized light emitted from the light source unit 20 into circularly polarized light. The fluorescent dye reacts to light in a predetermined polarization direction. Therefore, by converting the excitation light emitted from the light source unit 20 into circularly polarized light, the polarization direction of the excitation light can easily match the polarization direction in which the fluorescent dye reacts. This makes it possible to efficiently excite the fluorescence of the fluorescent dye contained in the test cells. The beam expander 33 expands the irradiation region of light on the slide glass 41. The condenser lens 34 collects light so that the slide glass 41 is irradiated with parallel light from the objective lens 36.

The dichroic mirror 35 reflects the light emitted from the light source unit 20 and transmits the fluorescence generated from the test cells. The objective lens 36 guides the light reflected by the dichroic mirror 35 to the slide glass 41. The stage 40 is driven by the controller 72 and moves in the horizontal plane XY. The fluorescence generated from the test cells passes through the objective lens 36, passes through the dichroic mirror 35, and is made into parallel light by the beam expander 37.

The phase modulation mask 50 applies phase modulation to the first fluorescence and the second fluorescence. The phase modulation mask 50 is arranged on the Fourier plane of the optical system composed of the objective lens 36, the dichroic mirror 35, the beam expander 37, and the condenser lens 61, and is incident on the same incident region of the phase modulation mask 50. It has the effect of modulating the phase of the light.

The case where the phase difference is "θ" and the case where the phase difference is "θ + n × 2π" (n = ± 0, 1, 2, 3 ...) Are substantially the phase difference given to the fluorescence. Is the same as. Therefore, the phase difference given to the first fluorescence and the phase difference given to the second fluorescence by the phase modulation mask 50 are not limited to one value, but may be a value obtained by adding n × 2π to one value.

The phase modulation mask 50 is arranged as shown in FIG. 1 when the phase of the light passing through the phase modulation mask 50 is modulated. The phase modulation mask 50 that modulates the phase of the transmitted light is composed of, for example, a phase plate made of a transparent member such as acrylic resin, a phase modulator provided with a liquid crystal panel, and the like.

The phase modulation mask 50 is arranged as shown in FIG. 6A when the phase of the light reflected by the phase modulation mask 50 is modulated. The phase modulation mask 50 that modulates the phase of the reflected light is composed of, for example, a phase modulator provided with a liquid crystal panel, a deformable mirror, a reflective member configured in the same manner as the deformable mirror, and the like. When the phase modulation mask 50 is composed of a phase modulator equipped with a liquid crystal panel, as shown in FIG. 6A, a polarizing plate 38 and a mirror 39 are arranged in place of the beam expander 37, and phase modulation is performed. The mask 50 is arranged at the position of the phase modulator 51. When the phase modulation mask 50 is composed of a deformable mirror, a reflective member or the like configured in the same manner as the deformable mirror, the polarizing plate 38 is omitted from the configuration shown in FIG. 6A, and the phase modulation mask 50 is a phase modulation mask 50. It is arranged at the position of the phase modulator 51.

A configuration example of the phase modulation mask 50 that modulates the phase of the transmitted light will be described later with reference to FIGS. 29 (a) to 30 (i). A configuration example of the phase modulation mask 50 that modulates the phase of the reflected light will be described later with reference to FIGS. 6 (a) to 28 (h).

The phase modulation mask 50 forms an image corresponding to the point image distribution function of the first fluorescence generated from the first fluorescent dye, and forms an image corresponding to the point image distribution function of the second fluorescence generated from the second fluorescent dye. do. In the phase modulation mask 50 of the embodiment, the first fluorescence generated from one first fluorescent dye is imaged at two focal points on the image pickup surface 62a of the image pickup unit 62, and the second fluorescence generated from one second fluorescent dye is formed. Is imaged at two focal points on the imaging surface 62a of the imaging unit 62. Such a point image distribution function is called a DH-PSF (Double-Helix Point Spread Function). The phase modulation mask 50 modulates the phases of the first fluorescence and the second fluorescence incident on the Fourier plane, and forms an image corresponding to the DH-PSF on the imaging surface 62a for both the first fluorescence and the second fluorescence.

Note that the phase modulation mask 50 is not configured to form an image corresponding to DH-PSF on the imaging surface 62a for the third fluorescence, unlike the cases of the first fluorescence and the second fluorescence. The reason for this is that, as will be described later, the third fluorescence is used only for identifying the nuclear region. Although the phase of the third fluorescence transmitted through the phase modulation mask 50 is slightly modulated by the phase modulation mask 50, if the third fluorescence is imaged by the imaging unit 62, the nuclear region can be sufficiently specified.

The condensing optical system 60 condenses the phase-modulated first fluorescence and the second fluorescence to form an image corresponding to the DH-PSF. The condensing optical system 60 includes a condensing lens 61 and an imaging unit 62. The condenser lens 61 collects the fluorescence that has passed through the phase modulation mask 50 and guides it to the image pickup surface 62a of the image pickup unit 62. The image pickup unit 62 takes an image of the fluorescence irradiated on the image pickup surface 62a and generates a two-dimensional image. The image pickup unit 62 is composed of, for example, a CCD or the like.

Here, as described above, the first fluorescence generated from one first fluorescent dye and the second fluorescence generated from one second fluorescent dye are each formed by the action of the phase modulation mask 50 on the imaging surface 62a. It is imaged at two focal points above. That is, images corresponding to the DH-PSF of the first fluorescence and the second fluorescence are formed on the imaging surface 62a. At this time, due to the action of the phase modulation mask 50, the bright spot image corresponding to each of the two focal points is the emission point of fluorescence in the Z-axis direction, that is, the depth direction of the slide glass 41, as shown in FIG. 2A. Depending on the position, it rotates on the imaging surface 62a. That is, the angle formed by the straight line connecting the two bright spot images and the reference straight line changes on the image pickup surface 62a according to the position of the emission point of fluorescence in the Z-axis direction.

That is, the phase modulation mask 50 forms a DH-PSF in which the two bright spot images of the first fluorescence rotate on the imaging surface 62a according to the distance between the objective lens 36 and the first fluorescent dye in the sample. It is configured to modulate the phase of the first fluorescence. Similarly, the phase modulation mask 50 forms a DH-PSF in which the two bright spot images of the second fluorescence rotate on the imaging surface 62a according to the distance between the objective lens 36 and the second fluorescent dye in the sample. As described above, it is configured to modulate the phase of the second fluorescence.

For example, the fluorescence generated from the fluorescent dyes at two different positions in the Z-axis direction on the slide glass 41 is divided into two by the phase modulation mask 50 and irradiated on the image pickup surface 62a. At this time, the straight line connecting the two bright spot images on the imaging surface 62a forms an angle of + θ1 with the reference line for one fluorescent dye, as shown in FIG. 2A, for example, and the other fluorescent dye. Is an angle of + θ2 with the reference plane. Therefore, if the angle formed by the straight line connecting the two bright spot images with respect to the reference line is acquired, the position of the fluorescent dye in the Z-axis direction can be acquired. In the embodiment, the positions of the first fluorescence and the second fluorescence in the Z-axis direction are acquired as described above based on the two-dimensional image captured by the imaging unit 62.

The DH-PSF can be expressed by the formula shown in FIG. 2 (b). In the equation of FIG. 2B, "bright spot 1" and "bright spot 2" indicate two bright spot images formed on the imaging surface 62a as shown in FIG. 2 (a). The "coordinates on the image pickup surface" indicate the coordinates on the image pickup surface 62a of the fluorescent dye that is the source of the two bright spot images.

As described above, both the first fluorescence and the second fluorescence are incident on the same incident region of the phase modulation mask 50, and the phase modulation mask 50 has the first fluorescence and the second fluorescence having different wavelengths. , Each phase is modulated. Thereby, it is not necessary to guide the first fluorescence to the region for modulating the phase of the first fluorescence and the second fluorescence to the region for modulating the phase of the second fluorescence. Therefore, since it is not necessary to separately provide the phase modulation region of the first fluorescence and the phase modulation region of the second fluorescence, in order to diffract the light beam in order to guide the first fluorescence and the second fluorescence to the separate phase modulation regions. There is no need for a diffraction grating or a prism for branching the light beam. Therefore, according to the phase modulation mask 50, the phase can be modulated for both the first fluorescence and the second fluorescence with a simple configuration, and an image corresponding to the DH-PSF of the first fluorescence and the second fluorescence can be formed.

Further, when the first fluorescence and the second fluorescence are branched into different optical paths, there is a possibility that an image corresponding to a desired DH-PSF cannot be formed due to a deviation in the assembly of optical elements in each optical path. There is. However, according to the phase modulation mask 50, since the optical element for branching the optical path is not required, the influence caused by the deviation of the assembly of the optical element or the like can be suppressed. Therefore, a highly accurate two-dimensional image can be generated. Such improvement in the accuracy of the two-dimensional image is particularly preferable in the optical device 10 of the embodiment that generates a three-dimensional super-resolution image described later.

Returning to FIG. 1, the information processing apparatus 100 is a personal computer, and includes a main body 110, a display unit 120, and an input unit 130. The main body 110 includes a processing unit 111, a storage unit 112, and an interface 113.

The processing unit 111 is, for example, a CPU. The storage unit 112 is a ROM, RAM, hard disk, or the like. The processing unit 111 controls the light sources 21 to 23 of the light source unit 20, the image pickup unit 62, and the controllers 71 and 72 via the interface 113 based on the program stored in the storage unit 112.

Based on the two-dimensional image of the first fluorescence, the processing unit 111 acquires the position of the emission point of the first fluorescence in the Z-axis direction as described above, and generates a three-dimensional super-resolution image of the first fluorescence. Similarly, the processing unit 111 acquires the position of the emission point of the second fluorescence in the Z-axis direction as described above based on the two-dimensional image of the second fluorescence, and obtains the three-dimensional super-resolution image of the second fluorescence. Generate. In the following, the two-dimensional image of the first fluorescence is referred to as a "first two-dimensional image", the two-dimensional image of the second fluorescence is referred to as a "second two-dimensional image", and the two-dimensional image of the third fluorescence is referred to as "the second two-dimensional image". It is called "third two-dimensional image". The three-dimensional super-resolution image of the first fluorescence is referred to as a "first three-dimensional super-resolution image", and the three-dimensional super-resolution image of the second fluorescence is referred to as a "second three-dimensional super-resolution image".

The display unit 120 is a display for displaying the processing result or the like by the processing unit 111. The input unit 130 is a keyboard and a mouse for receiving input of instructions by the user.

Next, the procedure for generating the first and second three-dimensional super-resolution images will be described.

First, the procedure for acquiring the first two-dimensional image will be described with reference to FIGS. 3A to 3D.

As shown in FIG. 3A, in the preparation of the sample, the first fluorescent dye is bound to the first substance via an intermediate substance that specifically binds to the first substance. Since the first substance is a gene, a nucleic acid probe can be used as an intermediate substance. A large number of first fluorescent dyes are bound to one first substance. FIG. 3A schematically shows two first substances to which the first fluorescent dye is bound. In the initial state, all the first fluorescent dyes are in the active state. When the test cells are irradiated with the light from the light source 21 for a predetermined time in the state shown in FIG. 3A, all the first fluorescent dyes become inactive as shown in FIG. 3B. ..

When the test cells are irradiated with the light from the light source 23 for a predetermined time in the state shown in FIG. 3 (b), for example, as shown in FIG. 3 (c), a part of the first fluorescent dye is activated. .. By adjusting the irradiation time of the light from the light source 23, the ratio of the activated first fluorescent dye changes. In the state shown in FIG. 3 (c), when the light from the light source 21 is irradiated again to the test cells for a predetermined time, the activated first fluorescent dye produces the first fluorescence, and then FIG. 3 (b). As shown in, all the first fluorescent dyes are inactive.

Then, when the light from the light source 23 is irradiated to the test cells again for a predetermined time, for example, as shown in FIG. 3D, a part of the first fluorescent dye is activated. In the state shown in FIG. 3 (d), when the light from the light source 21 is irradiated again to the test cells for a predetermined time, the activated first fluorescent dye produces the first fluorescence, and then FIG. 3 (b). As shown in, all the first fluorescent dyes are inactive. As shown in FIGS. 3 (c) and 3 (d), the distribution of the first fluorescent dye activated by each activation treatment is different each time.

The processing unit 111 drives the light sources 21 and 23 to repeatedly activate and inactivate the first fluorescent dye as described above. The image pickup unit 62 takes an image of a different distribution of the first fluorescent dye each time. In this way, the processing unit 111 acquires a plurality of first two-dimensional images, for example, 3000 first two-dimensional images.

Subsequently, the procedure for acquiring the second two-dimensional image will be described with reference to FIGS. 3 (e) to 3 (h). The acquisition of the second two-dimensional image is performed in substantially the same manner as the acquisition of the first two-dimensional image.

That is, when the test cells are irradiated with the light from the light source 22 for a predetermined time in the initial state shown in FIG. 3 (e), all the second fluorescent dyes are inactive as shown in FIG. 3 (f). It becomes a state. When the test cells are irradiated with the light from the light source 23 for a predetermined time in the state shown in FIG. 3 (f), for example, as shown in FIG. 3 (g), a part of the second fluorescent dye is activated. .. In the state shown in FIG. 3 (g), when the light from the light source 22 is irradiated again to the test cells for a predetermined time, the activated second fluorescent dye produces second fluorescence, and then FIG. 3 (f). As shown in, all the second fluorescent dyes are inactive. Then, when the test cells are irradiated with the light from the light source 23 again for a predetermined time, a part of the second fluorescent dye is activated, for example, as shown in FIG. 3 (h).

The processing unit 111 drives the light sources 22 and 23 to repeatedly activate and inactivate the second fluorescent dye as described above. The imaging unit 62 images the distribution of the second fluorescent dye, which is different each time. In this way, the processing unit 111 acquires a plurality of second two-dimensional images, for example, 3000 second two-dimensional images.

Subsequently, the procedure for generating the first three-dimensional super-resolution image will be described with reference to FIGS. 4 (a) to 4 (e). Since the procedure for generating the second three-dimensional super-resolution image is the same as that for the first three-dimensional super-resolution image, only the procedure for generating the first three-dimensional super-resolution image will be described below. do.

As shown in FIG. 4A, a plurality of first two-dimensional images are acquired as described above. On the first two-dimensional image shown in FIG. 4A, the captured first fluorescence is indicated by a black circle. As shown in FIG. 4B, the processing unit 111 extracts the bright spot 81 of the first fluorescence by Gauss fitting in each first two-dimensional image. Then, the processing unit 111 acquires the coordinates and the luminance in the XY plane for the extracted bright spot 81.

Subsequently, the processing unit 111 refers to two bright spots 81 having the same brightness and a distance within a predetermined range. The processing unit 111 fits the referenced two bright spots 81 with the template of the two bright spots stored in the storage unit 112 in advance. The processing unit 111 pairs the two bright spots 81 that can be fitted with a certain accuracy or higher, assuming that the first fluorescence generated from one first fluorescent dye is divided by the phase modulation mask 50.

Subsequently, as shown in FIG. 4C, the processing unit 111 is a point in the XY plane of the first fluorescent dye that is the source of the two bright spots 81 based on the two bright spots 81 that are paired with each other. Get 82. Subsequently, as shown in FIG. 4D, the processing unit 111 acquires an angle θ formed by the straight line connecting the two paired bright spots 81 and the reference line. The processing unit 111 calculates the coordinates of the first fluorescent dye in the Z-axis direction based on the acquired angle θ. In this way, as shown in FIG. 4 (e), the processing unit 111 acquires the three-dimensional coordinates of the plurality of first fluorescent dyes based on the coordinates in the XY plane and the coordinates in the Z-axis direction. Then, as shown in FIG. 4 (e), the processing unit 111 generates a first three-dimensional super-resolution image by superimposing a plurality of three-dimensional coordinates acquired in each first two-dimensional image. do.

In this way, when the first and second three-dimensional super-resolution images are acquired, the emission point of the first fluorescence and the second fluorescence are higher than those in the case where the first and second two-dimensional images are used. The emission point can be grasped accurately. As a result, doctors and the like can grasp the distribution of the first substance in the Z-axis direction by referring to the first and second three-dimensional super-resolution images, and can more accurately determine the medical condition and the treatment policy. ..

Next, a procedure for obtaining the number of the first substance will be described with reference to FIG. 4 (f).

As shown in FIG. 4 (f), the processing unit 111 classifies the coordinate points of the first three-dimensional super-resolution image into a group corresponding to the first substance. For example, the processing unit 111 scans a predetermined reference space in a three-dimensional coordinate space, and the number of coordinate points included in the reference space is larger than the threshold value and the number of coordinate points is larger than the surroundings. Extract the position. Then, the processing unit 111 classifies the group of the coordinate points included in the reference space at the extracted position into the group corresponding to one first substance as shown by the broken line in FIG. 4 (f).

Subsequently, the processing unit 111 acquires the range of the nucleus in the three-dimensional space of the test cell. Specifically, the processing unit 111 displaces the objective lens 36 in the Z-axis direction to acquire a third two-dimensional image based on the third fluorescence at a plurality of focus positions different in the Z-axis direction. In the third two-dimensional image, the region where the third fluorescence is detected corresponds to the nucleus, and the region where the third fluorescence is not detected corresponds to other than the nucleus, that is, the cytoplasm and the like. The processing unit 111 acquires the contour of the nucleus from the region where the third fluorescence is detected for each of the plurality of third two-dimensional images. Then, the processing unit 111 acquires the range of the nucleus in the three-dimensional coordinate space based on each focus position and the contour of the nucleus at that position.

Subsequently, the processing unit 111 acquires the number of groups included in the range of the nucleus of the test cell as the number of the first substance in the three-dimensional coordinate space. When a plurality of test cells are included in the first three-dimensional super-resolution image, the number of the first substance is, for example, by averaging the number of the first substance acquired for each test cell. To be acquired. The processing unit 111 also acquires the number of the second substance based on the second three-dimensional super-resolution image.

The processing unit 111 calculates the ratio of the number of the first substance and the number of the second substance acquired as described above, that is, "the number of the first substance / the number of the second substance". For example, if the "number of first substance / number of second substance" is larger than 2.2, it can be determined to be positive for breast cancer, if it is smaller than 1.8, it can be determined to be negative for breast cancer, and if it is 1.8 or more and 2.2 or less. If there is, it can be judged as a boundary.

In this way, when the numbers of the first substance and the second substance are acquired based on the first and second three-dimensional images, "the number of the first substance / the number of the second substance" can be calculated accurately. This makes it possible to present a more accurate determination result to a doctor or the like.

<Example of changing the imaging procedure>
In the above imaging procedure, the first fluorescence and the second fluorescence were separately imaged by the imaging unit 62. In this case, as shown in FIG. 5A, the processing unit 111 acquires a plurality of second two-dimensional images after acquiring a plurality of first two-dimensional images. Then, the processing unit 111 acquires a first three-dimensional super-resolution image based on the plurality of first two-dimensional images, and based on the plurality of second two-dimensional images, the second three-dimensional super-resolution image is obtained. Get a resolution image. In this way, if the first two-dimensional image and the second two-dimensional image are separately imaged, the time required for imaging becomes long.

Therefore, instead of the above-mentioned imaging procedure, the imaging unit 62 may simultaneously capture the first fluorescence and the second fluorescence. In this case, the processing unit 111 turns on the light sources 21 and 22 at the same time when the first and second fluorescent dyes are in the active state, and simultaneously irradiates the test cells with the light from the light sources 21 and 22. As a result, the first fluorescence and the second fluorescence are simultaneously generated from the test cells, and the imaging surface 62a of the imaging unit 62 is simultaneously irradiated with the first fluorescence and the second fluorescence. As shown in FIG. 5B, the two-dimensional image acquired at this time is a common two-dimensional image in which the first two-dimensional image and the second two-dimensional image are superimposed. When the first fluorescence and the second fluorescence are generated at the same time in this way, the image pickup unit 62 is composed of a color CCD or the like.

Also in this case, as shown in FIG. 5B, the processing unit 111 generates a first three-dimensional super-resolution image based on the first fluorescence in the common two-dimensional image, and the common two-dimensional image. A second 3D super-resolution image is generated based on the second fluorescence in. When the first fluorescence and the second fluorescence are simultaneously imaged by the imaging unit 62 in this way, the time required for imaging can be significantly shortened.

<Preliminary verification of phase modulation mask>
As described above, the phase modulation mask 50 is configured to correspond to the first fluorescence and the second fluorescence having different wavelengths from each other. By the way, a phase modulation mask that causes an optimum phase difference for one type of fluorescence and appropriately forms an image corresponding to the point image distribution function of one type of fluorescence is generally known. Therefore, in order to make the phase modulation mask correspond to two types of fluorescence, the present inventor first verifies that the phase of the first fluorescence and the phase of the second fluorescence is modulated by the phase modulation mask optimal for the first fluorescence, and the first. We verified that the phases of the first fluorescence and the second fluorescence were modulated by the phase modulation mask optimal for the two fluorescences.

As shown in FIG. 6A, the optical instrument 10 used in this verification includes a polarizing plate 38 and a mirror 39 in place of the beam expander 37, and a phase modulation mask 50, as compared with the configuration shown in FIG. A phase modulator 51 is provided instead of the above. The polarizing plate 38 is composed of, for example, a polarizing prism. The polarizing plate 38 is installed so that the polarization direction is an appropriate polarization direction with respect to the phase modulator 51. The mirror 39 reflects the fluorescence that has passed through the polarizing plate 38 and guides it to the phase modulator 51. The phase modulation mask used in this verification is a phase modulator 51 that modulates the phase when reflecting incident light. The phase modulator 51 is arranged on the Fourier plane of the optical system composed of the objective lens 36, the dichroic mirror 35, the polarizing plate 38, the mirror 39, and the condenser lens 61. The phase modulator 51 has an effect of modulating the point image distribution function on the imaging surface 62a, similarly to the phase modulation mask 50 shown in FIG.

As shown in FIG. 6B, the phase modulator 51 includes a liquid crystal panel 51a. When the phase modulator 51 is driven, the liquid crystal molecules 51b in the liquid crystal panel 51a rotate according to the setting, and the width of the liquid crystal molecules 51b in the incident direction of light changes. When the width of each liquid crystal molecule 51b changes in the incident direction of light in this way, a difference in refractive index occurs depending on the position of the phase modulator 51 in the incident region. As a result, the phase of the light incident on the liquid crystal panel 51a and reflected by the mirror 51c is modulated according to the incident position.

When an image is input, the phase modulator 51 sets the gradation of each pixel of the liquid crystal panel 51a based on the input image. The image input to the phase modulator 51 holds information representing the gradation of each pixel of the phase modulator 51. The phase modulator 51 acquires the gradation set for each pixel from the input image, and sets the rotation angle of each liquid crystal molecule 51b so that the gradation of each pixel becomes a desired gradation based on the input image. do. In this way, the phase modulator 51 sets the rotation angle of the liquid crystal molecule 51b based on the input image, and sets the gradation pattern of each pixel. When the phase modulator 51 is configured to accept other than an image, each pixel of the liquid crystal panel 51a is based on data other than the image that holds information representing the gradation of each pixel of the phase modulator 51. The gradation of may be set.

In this verification, "LCOS-SLM01" manufactured by Hamamatsu Photonics Co., Ltd. was used as the phase modulator 51. The first fluorescent beads and the second fluorescent beads were arranged on the slide glass 41 of the stage 40. When the first fluorescent beads are irradiated with light from the light source 21, the first fluorescence beads have a central wavelength of 690 nm, that is, the first fluorescence. When the second fluorescent beads are irradiated with the light from the light source 22, the second fluorescence beads generate fluorescence having a center wavelength of 530 nm, that is, the second fluorescence.

In this verification, when observing the first fluorescence, the objective lens 36 was scanned in the Z-axis direction with respect to the first fluorescence bead, and the position of the first fluorescence bead in the Z-axis direction was relatively changed. Similarly, when observing the second fluorescence, the objective lens 36 was scanned in the Z-axis direction with respect to the second fluorescence bead, and the position of the second fluorescence bead in the Z-axis direction was relatively changed. By scanning the objective lens 36 in the Z-axis direction in this way, it is possible to obtain a state similar to a state in which a plurality of fluorescent beads are arranged at different positions in the Z-axis direction. Then, the fluorescence was imaged by the image pickup unit 62, and the fluorescence image was acquired at each scan position of the objective lens 36.

The liquid crystal panel 51a of the phase modulator 51 is configured to be capable of phase modulation with 256 gradations per pixel by changing the inclination of the liquid crystal molecules 51b as shown in FIG. 6B. The gradation of each pixel can be set to 0 to 255, and by setting the gradation to any of 0 to 255, phase modulation of 256 gradations is realized. The gradation pattern distribution set for all the pixels of the liquid crystal panel 51a is hereinafter referred to as a "phase modulation pattern" as a distribution that modulates the phase. That is, the gradation distribution set for each pixel of the liquid crystal panel 51a corresponds to the phase modulation pattern. By inputting an image to the phase modulator 51 and setting the phase modulation pattern, the gradation of each pixel of the liquid crystal panel 51a is set, and as shown in FIG. 6B, the fluorescence incident on the phase modulator 51 is set. The phase of is changed for each pixel.

The first phase modulation pattern shown in FIG. 7A is an optimum phase modulation pattern for light having a first wavelength, that is, for first fluorescence. The first phase modulation pattern imparts first phase modulation to light of the first wavelength, i.e., first fluorescence. The second phase modulation pattern shown in FIG. 7B is an optimum phase modulation pattern for light having a second wavelength, that is, for second fluorescence. The second phase modulation pattern imparts a second phase modulation to the light of the second wavelength, that is, the second fluorescence. When the phase modulation pattern of the phase modulator 51 is set to the first phase modulation pattern, an image corresponding to DH-PSF is formed on the image pickup surface 62a for the first fluorescence incident on the phase modulator 51. When the phase modulation pattern of the phase modulator 51 is set to the second phase modulation pattern, an image corresponding to DH-PSF is formed on the image pickup surface 62a for the second fluorescence incident on the phase modulator 51.

In FIGS. 7A and 7B, the pixel having a gradation of 0 is shown in black, and the pixel having a gradation of 255 is shown in white. Pixels with zero gradation do not modulate the phase of the incident light. The phase of the first fluorescence incident on the pixel having the gradation of 255 is shifted by 2π with respect to the phase of the first fluorescence incident on the pixel having the gradation of 0. The phase of the second fluorescence incident on the pixel having the gradation of 183 is shifted by 2π with respect to the phase of the second fluorescence incident on the pixel having the gradation of 0.

In this verification and the verification described later, when the first fluorescence is incident on the phase modulator 51 in order to correct the aberration caused by the incident surface of the phase modulator 51, the phase modulation pattern set in the phase modulator 51 is used. Synthesized a predetermined first correction mask, and when the second fluorescence was incident on the phase modulator 51, the predetermined second correction mask was synthesized for the phase modulation pattern set in the phase modulator 51. For a pixel whose gradation exceeds 255 due to the synthesis of the first correction mask, the value obtained by dividing the gradation by 256 is used as the gradation of the pixel. For a pixel whose gradation exceeds 183 due to the synthesis of the second correction mask, the value obtained by dividing the gradation by 184 is taken as the gradation of the pixel.

The result of the preliminary verification of the phase modulator will be described with reference to FIGS. 8A to 8H.

8 (a) and 8 (b) are verification results when the first phase modulation pattern is set in the phase modulator 51 and the first fluorescence is observed. In this case, as shown in FIG. 8A, the first fluorescence is imaged at two points on the image pickup surface 62a, and a straight line formed by the two image positions is formed by scanning the objective lens 36 in the Z-axis direction. It was rotated 180 degrees. Further, as shown in FIG. 8B, the relationship between the scan position of the objective lens 36 in the Z-axis direction and the angle of the straight line formed by the two imaging positions is a curve corresponding to one-to-one. rice field. From the results of FIGS. 8A and 8B, it was found that in this case, an image corresponding to the DH-PSF of the first fluorescence can be appropriately formed.

8 (c) and 8 (d) are verification results when the first phase modulation pattern is set in the phase modulator 51 and the second fluorescence is observed. In this case, as shown in FIG. 8C, the shape of the DH-PSF collapsed depending on the position in the Z-axis direction. Further, as shown in FIG. 8D, a large step appeared in a part of the curve showing the relationship between the angle and the scan position. From the results of FIGS. 8 (c) and 8 (d), it was found that in this case, an image corresponding to the DH-PSF of the second fluorescence could not be properly formed.

8 (e) and 8 (f) are verification results when the second phase modulation pattern is set in the phase modulator 51 and the first fluorescence is observed. In this case, as shown in FIG. 8 (e), the shape of the DH-PSF collapsed depending on the position in the Z-axis direction. Further, as shown in FIG. 8 (f), the curve showing the relationship between the scan position and the angle was greatly collapsed, and the scan position and the angle did not have a one-to-one correspondence. From the results of FIGS. 8 (e) and 8 (f), it was found that in this case, an image corresponding to the DH-PSF of the first fluorescence could not be properly formed.

8 (g) and 8 (h) are verification results when the second phase modulation pattern is set in the phase modulator 51 and the second fluorescence is observed. In this case, as shown in FIG. 8 (g), the two bright spot images became appropriate. Further, as shown in FIG. 8 (h), the relationship between the angle and the scan position is a curve corresponding to one-to-one. From the results of FIGS. 8 (g) and 8 (h), it was found that in this case, an image corresponding to the DH-PSF of the second fluorescence can be appropriately formed.

From the results of FIGS. 8 (a), (b), (g), and (h), the combination of the setting of the phase modulation pattern of the phase modulator 51 and the center wavelength of the fluorescence incident on the phase modulator 51 is optimal. In some cases, an image corresponding to the fluorescence DH-PSF could be properly formed. It can be said that this was valid as a result. On the other hand, if the combination of the setting of the phase modulation pattern of the phase modulator 51 and the center wavelength of the fluorescence incident on the phase modulator 51 is not optimal, an image corresponding to the fluorescence DH-PSF could not be formed properly. From this, it was found that it is necessary to appropriately select the setting of the phase modulation pattern of the phase modulator 51 and the fluorescence incident on the phase modulator 51.

Based on the above results, the present inventor has an optimum first phase modulation pattern for the first fluorescence and an optimum second phase modulation for the second fluorescence so that both the first fluorescence and the second fluorescence can be supported. I thought about integrating with the pattern. At that time, the present inventor paid attention to the fact that the wavelength bands of the first fluorescence and the second fluorescence overlap. The first fluorescence is light having an intensity peak at the first wavelength, and the second fluorescence is light having an intensity peak at the second wavelength. That is, the wavelength band of the first fluorescence has a certain extent with the first wavelength as the central wavelength, and the wavelength band of the second fluorescence has a certain extent with the second wavelength as the central wavelength. Then, a part of the wavelength band of the first fluorescence and a part of the wavelength band of the second fluorescence overlap each other.

The inventor has described that when a part of the wavelength band of the first fluorescence and a part of the wavelength band of the second fluorescence overlap, the first phase modulation pattern most suitable for the first fluorescence and the second most suitable for the second fluorescence. It was found that if the phase modulation pattern is integrated as follows, images corresponding to the DH-PSF of the first fluorescence and the second fluorescence can be appropriately formed by the integrated phase modulation pattern. The phase modulation patterns of Examples 1 to 6 shown below are examples in which the first phase modulation pattern and the second phase modulation pattern are integrated by various methods. In addition, the inventor has verified whether or not the phase modulation patterns of Examples 1 to 6 properly form images corresponding to the DH-PSF of the first fluorescence and the second fluorescence.

<Phase modulation pattern of Example 1>
The phase modulation pattern of Example 1 is produced by synthesizing a first phase modulation pattern and a second phase modulation pattern at a predetermined ratio for each position. When the first phase modulation pattern and the second phase modulation pattern are combined a to b, the gradation of the phase modulation pattern of Example 1 is the first phase modulation pattern as shown in the following equation. It is calculated based on the gradation and the second gradation of the second phase modulation pattern.

Gradation of phase modulation pattern of Example 1 = (first gradation × a + second gradation × b) / (a + b)
(A and b are both positive real numbers)

Specifically, when the phase modulation pattern of Example 1 is used, an image corresponding to the phase modulation pattern of Example 1 is produced, the produced image is input to the phase modulator 51, and the phase is based on the input image. The phase modulation pattern of the first embodiment is realized in the modulator 51. The gradation in each pixel of the image corresponding to the phase modulation pattern of the first embodiment is set to the gradation between the gradation for light of the first wavelength and the gradation for light of the second wavelength. .. That is, it is calculated by the above formula based on the gradation of the first phase modulation pattern optimal for the first fluorescence at the same pixel position and the gradation of the second phase modulation pattern optimal for the second fluorescence at the same pixel position. To. By inputting the image corresponding to the phase modulation pattern of Example 1 thus produced into the phase modulator 51, the phase modulation pattern of Example 1 is set in the phase modulator 51, and each liquid crystal of the liquid crystal panel 51a. The rotation angle of the molecule 51b is set.

9 shows (a, b) = (9, 1), (8, 2), (7, 3), (6, 4), (5, 5), (4, 6), (3, 7). ), (2, 8), (1, 9), it is a figure which shows the phase modulation pattern of Example 1. The results of verification of these nine phase modulation patterns are shown below.

FIG. 10 shows the first fluorescence on the imaging surface 62a when the first fluorescence is observed using the nine types of phase modulation patterns of Example 1, the first phase modulation pattern, and the second phase modulation pattern as described above. It is a figure which shows the image formation state of 1 fluorescence. The vertical direction indicates the scan position of the objective lens 36. The leftmost "690 nm" and the rightmost "530 nm" indicate the case where the first phase modulation pattern and the second phase modulation pattern are used, respectively.

As shown in FIG. 10, at any of the scan positions, as the compounding ratio of the second phase modulation pattern increases and the phase modulation pattern approaches the second phase modulation pattern, it corresponds to the DH-PSF. It was seen that the bright spot image was blurred. When the scan position was -3 μm or -4 μm, a new bright spot based on the 0th-order light appeared in the middle of the two bright spots as it proceeded to the right. In particular, when the phase modulation pattern of (a, b) = (1, 9) and the second phase modulation pattern are used, the emission of the bright spot image of the 0th-order light is shown in the portion surrounded by the solid line frame. The point intensity was higher than that of the bright spot image corresponding to DH-PSF. In such a case, by erroneously recognizing the 0th-order light as the bright spot of the DH-PSF, there is a possibility that a curve showing the relationship between the scan position and the angle and a three-dimensional super-resolution image cannot be properly generated.

FIG. 11 shows the first on the imaging surface 62a when the second fluorescence is observed using the nine types of phase modulation patterns of Example 1, the first phase modulation pattern, and the second phase modulation pattern as described above. It is a figure which shows the image formation state of 2 fluorescence.

As shown in FIG. 11, at any of the scan positions, as the compounding ratio of the first phase modulation pattern increases and the phase modulation pattern approaches the first phase modulation pattern, it corresponds to the DH-PSF. It was seen that the bright spot image was blurred. In particular, when the phase modulation pattern of (a, b) = (9, 1), (8, 2), (7, 3) and the first phase modulation pattern are used, they are shown in the portion surrounded by the solid line frame. As described above, the number of bright spots was more than two depending on the scan position.

From the results of FIGS. 10 and 11, (a, b) = (6, 4), (5, 5), (4, 6), (3, 7), (2, 8) is the phase modulation of Example 1. It was found that when the pattern was set, an image corresponding to both the first fluorescence and the second fluorescence DH-PSF could be properly formed.

Further, the verification results when the phase modulation pattern of Example 1 is set in (a, b) = (5, 5) will be described with reference to FIGS. 12 (a) to 12 (d).

12 (a) and 12 (b) show the result when the first fluorescence was observed, and FIGS. 12 (c) and 12 (d) show the result when the second fluorescence was observed. As shown in FIGS. 12 (a) and 12 (c), both the first fluorescence and the second fluorescence DH-PSF have an inclination of the straight line formed by the image formation positions of the two points in the range of −90 degrees to +90 degrees. I was able to properly form an image according to the above. Further, as shown in FIGS. 12 (b) and 12 (d), the relationship between the angle and the scan position is a curve corresponding to one-to-one.

From the above verification results, when the phase modulation pattern of Example 1 is set in the phase modulator 51, the first phase modulation pattern and the second phase modulation pattern are combined at a predetermined ratio to form the phase modulation pattern of Example 1. Then, it was found that the phase modulation pattern of Example 1 can appropriately form an image corresponding to both the first fluorescence and the second fluorescence DH-PSF. For three or more types of fluorescence having different center wavelengths, similarly, by synthesizing the optimum phase modulation pattern for the three types of fluorescence at a predetermined ratio, an image corresponding to the DH-PSF of the three types of fluorescence is obtained. Can be formed properly.

Next, with reference to FIGS. 13 (a) to 13 (c), the synthesis of the first phase modulation pattern and the second phase modulation pattern as described above will be described in detail. 13 (a) to 13 (c) are diagrams schematically showing one pixel position of the liquid crystal panel 51a in the first phase modulation pattern, the second phase modulation pattern, and the phase modulation pattern of the first embodiment, respectively. Is. In FIGS. 13A to 13C, it is assumed that the gradation of one pixel is set by one liquid crystal molecule 51b for convenience.

As shown in FIG. 13 (a), in order to properly form an image corresponding to DH-PSF based on the first fluorescence, the first fluorescence incident on a predetermined pixel position passes through the liquid crystal molecule 51b in a reciprocating manner. The distance is defined as the first distance L1. Similarly, as shown in FIG. 13 (b), in order to properly form an image corresponding to the DH-PSF of the second fluorescence, the second fluorescence incident on the predetermined pixel position passes through the liquid crystal molecule 51b in a reciprocating manner. The distance to be used is defined as the second distance L2. The magnitude at which the phase of the first fluorescence is modulated by the first distance L1 is equal to the magnitude at which the phase of the second fluorescence is modulated by the second distance L2. That is, considering the wavelength of the first fluorescence, the wavelength of the second fluorescence, and the refractive index of the liquid crystal molecule 51b, the optical path length of the first fluorescence based on the first distance L1 and the second fluorescence based on the second distance L2. The optical path lengths are equal to each other.

As shown in FIG. 13 (c), in order to properly form an image corresponding to both the first fluorescence and the second fluorescence DH-PSF, the first fluorescence and the second fluorescence incident on a predetermined pixel position are The distance passing through the liquid crystal molecule 51b in a round trip is defined as the third distance L3. At this time, the third distance L3 is set between the first distance and the second distance. That is, the magnitude of the phase modulation at each position of the phase modulation pattern of Example 1 is set to a magnitude between the magnitude of the phase modulation in the first phase modulation pattern and the magnitude of the phase modulation in the second phase modulation pattern. Has been done. Specifically, the third distance L3 is calculated by the following equation as in the equation of the phase modulation pattern of the first embodiment as described above.

Third distance L3 = (first distance L1 × a + second distance L2 × b) / (a + b)

When the phase modulation pattern of the first embodiment is set, the setting as shown in FIG. 13 (c) is performed at all the pixel positions. As a result, the phase modulation pattern of Example 1 can appropriately form an image corresponding to both the first fluorescence and the second fluorescence DH-PSF.

When (a, b) = (5, 5), the third distance L3 is set between the first distance L1 and the second distance L2. In this case, the first fluorescence has a phase modulation of an intermediate magnitude between the magnitude of phase-modulating the first fluorescence by the first phase modulation pattern and the magnitude of phase-modulating the first fluorescence by the second phase modulation pattern. .. In the second fluorescence, a phase modulation having an intermediate magnitude between the magnitude in which the second fluorescence is phase-modulated by the first phase modulation pattern and the magnitude in which the second fluorescence is phase-modulated by the second phase modulation pattern occurs.

<Phase modulation pattern of Example 2>
As shown in FIG. 14A, the phase modulation pattern of Example 2 is produced by arranging the first phase modulation pattern and the second phase modulation pattern in a mosaic pattern. In the phase modulation pattern of the second embodiment, the region group in which the incident region is divided into a large number is divided into a first region composed of a light gray region and a second region composed of a dark gray region. Each region of the first region and each region of the second region are adjacent to each other. The first phase modulation pattern is set in the light gray area, and the second phase modulation pattern is set in the dark gray area. In the phase modulation pattern of the second embodiment, one region of the first region and one region of the second region both have a square shape having M pixels on each side.

That is, the phase modulation pattern of the second embodiment is a region for applying the first phase modulation to the light of the first wavelength in the incident region, that is, the first region in which the first phase modulation pattern is set and the second region. It includes a region for applying a second phase modulation to light of a wavelength, that is, a second region in which a second phase modulation pattern is set. In other words, in the phase modulation pattern of the second embodiment in a certain region of the incident region, the distance that the light beam incident on the first region travels for phase modulation is set to the first distance L1 shown in FIG. 13 (a). , The distance that the light beam incident on the second region travels for phase modulation is set to the second distance L2 shown in FIG. 13 (b).

14 (b) to 14 (d) are diagrams showing the phase modulation pattern of Example 2 when M = 1, 3, and 5, respectively. In FIGS. 14 (b) to 14 (d), the arrangement of the first phase modulation pattern and the second phase modulation pattern near the center is enlarged and displayed. The results of verification of these three phase modulation patterns are shown below.

15 (a) to 15 (d) are diagrams showing the results of observing fluorescence using the phase modulation pattern of Example 2 with M = 1. As shown in FIGS. 15A and 15B, when the first fluorescence was observed, the two bright spot images became appropriate, and the relationship between the scan position and the angle also became a curve corresponding to one-to-one. As shown in FIGS. 15 (c) and 15 (d), even when the second fluorescence was observed, the two bright spot images became appropriate, and the relationship between the scan position and the angle became a curve corresponding to one-to-one. .. Therefore, it was found that by using the phase modulation pattern of Example 2 with M = 1, an image corresponding to both the first fluorescence and the second fluorescence can be appropriately formed.

15 (e) and 15 (f) are diagrams showing the results of observing the first fluorescence and the second fluorescence using the phase modulation pattern of Example 2 of M = 5. Also in this case, the two bright spot images were appropriate in both the case where the first fluorescence was observed and the case where the second fluorescence was observed. Therefore, it was found that by using the phase modulation pattern of Example 2 of M = 5, an image corresponding to both the first fluorescence and the second fluorescence DH-PSF can be appropriately formed.

FIG. 16A is a diagram showing the results of observing the first fluorescence in a wide field of view using the phase modulation pattern of Example 2 of M = 1 and 3. FIG. 16B is a diagram showing the results of observing the second fluorescence in a wide field of view using the phase modulation pattern of Example 2 of M = 1 and 3. In FIGS. 16A and 16B, the upper part is an image generated by the imaging unit 62 when the fluorescence is directly observed. The lower row is an image in which the contrast of the upper row image is adjusted. In FIGS. 16A and 16B, the image on the left side corresponds to the phase modulation pattern of M = 1, and the image on the right side corresponds to the phase modulation pattern of M = 3.

In the lower images of FIGS. 16A and 16B, bright spots appear at the positions indicated by the arrows. This bright spot is a bright spot based on the diffracted light generated by periodically arranging the first phase modulation pattern and the second phase modulation pattern in a mosaic pattern. Since the diffraction angle is smaller as the value of M is larger, the region where the diffracted light is closer to the center of the image when the phase modulation pattern of M = 3 is used than when the phase modulation pattern of M = 1 is used. Will appear in. When M = 3, it was found that the diffracted light was superimposed on a part of the bright spots corresponding to the DH-PSF originally desired to be observed.

Therefore, when the phase modulation pattern of Example 2 is used, it can be said that it is desirable to make the value of M as small as possible. However, when the size of one pixel of the phase modulator 51 is small, it is possible that the diffracted light does not enter the field of view even if the value of M is increased. Further, since the bright spot of the diffracted light is darker than the bright spot corresponding to the DH-PSF originally desired to be observed, there may be no particular problem when calculating the three-dimensional coordinates based on the rotation angle. The value of M capable of appropriately forming an image corresponding to DH-PSF of two types of fluorescence having different center wavelengths is not limited to 1, 3 and 5 as described above.

<Phase modulation pattern of Example 3>
As shown in FIG. 17A, the phase modulation pattern of Example 3 is produced by arranging the first phase modulation pattern and the second phase modulation pattern in a mosaic pattern, similarly to the phase modulation pattern of Example 2. Will be done. However, in the phase modulation pattern of the third embodiment, one region of the first region and one region of the second region are both M pixels in the vertical direction and N pixels in the horizontal direction.

17 (b) to 17 (d) are diagrams showing the phase modulation pattern of Example 3 when (M, N) = (1, 2), (1, 4), (1, 32), respectively. be. The results of verification of these three phase modulation patterns are shown below.

18 (a) to 18 (d) are diagrams showing the results of observing fluorescence using the phase modulation pattern of Example 3 of (M, N) = (1, 2). As shown in FIGS. 18A and 18B, when the first fluorescence was observed, the two bright spot images became appropriate, and the relationship between the scan position and the angle also became a curve corresponding to one-to-one. As shown in FIGS. 18 (c) and 18 (d), even when the second fluorescence was observed, the two bright spot images became appropriate, and the relationship between the scan position and the angle became a curve corresponding to one-to-one. .. Therefore, it was found that by using the phase modulation pattern of Example 3 of (M, N) = (1, 2), an image corresponding to both the first fluorescence and the second fluorescence DH-PSF can be appropriately formed. rice field.

18 (e) and 18 (f) are diagrams showing the results of observing the first fluorescence and the second fluorescence using the phase modulation pattern of Example 3 of (M, N) = (1, 32). .. Also in this case, the two bright spot images were appropriate in the case of observing the first fluorescence and the case of observing the second fluorescence. Therefore, it was found that by using the phase modulation pattern of Example 3 of (M, N) = (1, 32), an image corresponding to both the first fluorescence and the second fluorescence DH-PSF can be appropriately formed. rice field.

FIG. 19A is a diagram showing the results of observing the first fluorescence in a wide field of view using the phase modulation pattern of Example 3 of (M, N) = (1, 2), (1, 4). be. FIG. 19B is a diagram showing the results of observing the second fluorescence in a wide field of view using the phase modulation pattern of Example 3 of (M, N) = (1, 2), (1, 4). be. In FIGS. 19A and 19B, the image on the left corresponds to the phase modulation pattern of (M, N) = (1, 2), and the image on the right corresponds to (M, N) = (1, 4). Corresponds to the phase modulation pattern.

In the lower images of FIGS. 19 (a) and 19 (b), bright spots appear at the positions indicated by the arrows, as in the case of FIGS. 16 (a) and 16 (b). The vertical direction and the horizontal direction of FIG. 17A correspond to the horizontal direction and the vertical direction, respectively, in the images of FIGS. 19A and 19B. In this verification, since the value of M is fixed at 1, the diffracted light appears at the same position at both ends in the left-right direction as shown in the lower images of FIGS. 19A and 19B. On the other hand, when the value of N becomes large, the diffracted light approaches the center of the image in the vertical direction while maintaining the position in the horizontal direction. Further, also when the phase modulation pattern of the rectangular shape as shown by the condition (M, N) = (2, 4) is used, as can be seen from the verification of the phase modulation pattern of the second embodiment. As the value of M increases from 1 to 2, it is assumed that the diffracted light approaches the center of the image in the left-right direction.

Therefore, when the phase modulation pattern of Example 3 is used, it can be said that it is desirable to make the values of M and N as small as possible. The values of M and N capable of appropriately forming an image corresponding to DH-PSF of two types of fluorescence having different center wavelengths are (1, 2), (1, 4), (1, 32) as described above. ).

<Phase modulation pattern of Example 4>
As shown in FIG. 20A, the phase modulation pattern of Example 4 is produced by arranging the first phase modulation pattern and the second phase modulation pattern in a striped pattern. One region of the first region and one region of the second region both have a stripe shape of M pixels in the left-right direction. Each region of the first region and each region of the second region extends from one end to the other end of the incident region where the first fluorescence and the second fluorescence are incident. That is, the length in the vertical direction is the maximum pixel value of the length that can be set.

20 (b) to 20 (d) are diagrams showing the phase modulation pattern of Example 4 when M = 1, 2, and 16, respectively. The results of verification of these three phase modulation patterns are shown below.

21 (a) to 21 (d) are diagrams showing the results of observing fluorescence using the phase modulation pattern of Example 4 of M = 1. As shown in FIGS. 21 (a) and 21 (b), when the first fluorescence was observed, the two bright spot images became appropriate, and the relationship between the scan position and the angle also became a curve corresponding to one-to-one. As shown in FIGS. 21 (c) and 21 (d), even when the second fluorescence was observed, the two bright spot images became appropriate, and the relationship between the scan position and the angle became a curve corresponding to one-to-one. .. Therefore, it was found that by using the phase modulation pattern of Example 4 with M = 1, an image corresponding to both the first fluorescence and the second fluorescence DH-PSF can be appropriately formed.

21 (e) and 21 (f) are diagrams showing the results of observing the first fluorescence and the second fluorescence using the phase modulation pattern of Example 4 of M = 16. Also in this case, the two bright spot images were appropriate in the case of observing the first fluorescence and the case of observing the second fluorescence. Therefore, it was found that by using the phase modulation pattern of Example 4 of M = 16, an image corresponding to both the first fluorescence and the second fluorescence DH-PSF can be appropriately formed.

FIG. 22A is a diagram showing the results of observing the first fluorescence in a wide field of view using the phase modulation pattern of Example 4 of M = 1 and 2. FIG. 22B is a diagram showing the results of observing the second fluorescence in a wide field of view using the phase modulation pattern of Example 4 of M = 1 and 2. In FIGS. 22 (a) and 22 (b), the image on the left side corresponds to the phase modulation pattern of M = 1, and the image on the right side corresponds to the phase modulation pattern of M = 2.

In the lower images of FIGS. 22 (a) and 22 (b), bright spots appear at the positions indicated by the arrows, as in the case of FIGS. 16 (a) and 16 (b). However, unlike the case of the phase modulation pattern of Example 2, the diffracted light appeared only in the vertical direction as shown in the lower images of FIGS. 22 (a) and 22 (b). On the other hand, when the value of M becomes large, the diffracted light approaches the center of the image in the vertical direction, as in the phase modulation pattern of Example 2.

Therefore, when the phase modulation pattern of Example 4 is used, it is desirable to make the value of M as small as possible. The value of M capable of appropriately forming an image corresponding to DH-PSF of two types of fluorescence having different center wavelengths is not limited to 1, 2 and 16 as described above.

<Phase modulation pattern of Example 5>
As shown in FIG. 23A, the phase modulation pattern of Example 5 is produced by arranging the first phase modulation pattern and the second phase modulation pattern concentrically. In other words, each region of the first region and each region of the second region have a concentric ring shape. The center of each region of the first region and the center of each region of the second region may be deviated from each other. One region of the first region and one region of the second region are arranged so as to be interchanged with each other as the distance from the center increases by M pixels.

23 (b) to 23 (e) are diagrams showing the phase modulation patterns of Example 5 when M = 1, 2, 5, and 60, respectively. The results of verification of these four phase modulation patterns are shown below.

24 (a) to 24 (d) are diagrams showing the results of observing fluorescence using the phase modulation pattern of Example 5 of M = 1. As shown in FIGS. 24 (a) and 24 (b), when the first fluorescence was observed, the two bright spot images became appropriate, and the relationship between the scan position and the angle also became a curve corresponding to one-to-one. As shown in FIGS. 24 (c) and 24 (d), even when the second fluorescence was observed, the two bright spot images became appropriate, and the relationship between the scan position and the angle became a curve corresponding to one-to-one. .. Therefore, it was found that by using the phase modulation pattern of Example 5 with M = 1, an image corresponding to both the first fluorescence and the second fluorescence DH-PSF can be appropriately formed.

24 (e) and 24 (f) are diagrams showing the results of observing the first fluorescence and the second fluorescence using the phase modulation pattern of Example 5 of M = 5. Also in this case, the two bright spot images were appropriate in the case of observing the first fluorescence and the case of observing the second fluorescence. Therefore, it was found that by using the phase modulation pattern of Example 5 of M = 5, an image corresponding to both the first fluorescence and the second fluorescence DH-PSF can be appropriately formed.

24 (g) and 24 (h) are diagrams showing the results of observing the first fluorescence and the second fluorescence using the phase modulation pattern of Example 5 of M = 60. In this case, it was confirmed that the two bright spot images were distorted in the case of observing the first fluorescence and the case of observing the second fluorescence. Therefore, it was found that if the value of M becomes too large, images corresponding to the DH-PSF of the first fluorescence and the second fluorescence cannot be properly formed.

FIG. 25A is a diagram showing the results of observing the first fluorescence in a wide field of view using the phase modulation pattern of Example 5 of M = 2 and 5. FIG. 25B is a diagram showing the results of observing the second fluorescence in a wide field of view using the phase modulation pattern of Example 5 of M = 2 and 5. In FIGS. 25 (a) and 25 (b), the image on the left side corresponds to the phase modulation pattern of M = 2, and the image on the right side corresponds to the phase modulation pattern of M = 5.

In the lower images of FIGS. 25 (a) and 25 (b), a ring-shaped diffracted light surrounding the bright spot image appears as shown by an arrow. With reference to the lower images of M = 2 and 5, this diffracted light appears as a ring-shaped diffracted light with a smaller diameter as the value of M increases, and the bright spot corresponds to the DH-PSF originally desired to be observed. It turned out to be approaching. Therefore, when the phase modulation pattern of Example 5 is used, it can be said that it is desirable to make the value of M as small as possible. The value of M that can appropriately form an image corresponding to DH-PSF of two types of fluorescence having different center wavelengths is not limited to 1, 2, and 5 as described above.

Further, according to the phase modulation pattern of Example 5, the diffracted light is dispersed in a wider range along the circle as compared with the phase modulation pattern of Examples 2 to 4, so that the brightness of the diffracted light becomes smaller. Therefore, when the phase modulation pattern of Example 5 is used, even if the diffracted light overlaps the bright spot corresponding to the DH-PSF originally desired to be observed, it does not cause a big problem. From this, the phase modulation pattern of Example 5 in which the value of M is small more appropriately obtains an image corresponding to both DH-PSF of the first fluorescence and the second fluorescence than the phase modulation pattern of Examples 2 to 4. It can be said that it can be formed.

<Phase modulation pattern of Example 6>
In the phase modulation pattern of the sixth embodiment, the first phase modulation pattern and the second phase modulation pattern whose positions and sizes are changed are arranged in a square-shaped mosaic like the phase modulation pattern of the second embodiment. Is made by.

FIG. 26 is a diagram showing eight types of phase modulation patterns of the fifth embodiment. The "mosaicROTATE5" is a phase modulation pattern in which a first phase modulation pattern rotated 5 degrees around the center and a second phase modulation pattern are arranged in a mosaic pattern. The "mosaicROTATE30" is a phase modulation pattern in which a first phase modulation pattern rotated 30 degrees around the center and a second phase modulation pattern are arranged in a mosaic pattern. The "mosaicSHIFT5" is a phase modulation pattern in which a first phase modulation pattern shifted by 5 pixels to the right and a second phase modulation pattern shifted by 5 pixels to the left are arranged in a mosaic pattern. The "mosaicSHIFT10" is a phase modulation pattern in which a first phase modulation pattern shifted by 10 pixels to the right and a second phase modulation pattern shifted by 10 pixels to the left are arranged in a mosaic pattern.

The "mosaicEXPAND10" is a phase modulation pattern in which a first phase modulation pattern whose diameter is expanded by 10 pixels and a second phase modulation pattern are arranged in a mosaic pattern. The "mosaicEXPAND40" is a phase modulation pattern in which a first phase modulation pattern whose diameter is expanded by 40 pixels and a second phase modulation pattern are arranged in a mosaic pattern. The "mosaicREDUCE20" is a phase modulation pattern in which a first phase modulation pattern whose diameter is reduced by 20 pixels and a second phase modulation pattern are arranged in a mosaic pattern. The "mosaicREDUCE40" is a phase modulation pattern in which a first phase modulation pattern whose diameter is reduced by 40 pixels and a second phase modulation pattern are arranged in a mosaic pattern. The results of verification of these eight phase modulation patterns are shown below.

27 (a) and 27 (b) are diagrams showing the results of observing fluorescence using "mosaicROTATE5". In this case, the two bright spot images were appropriate in the case of observing the first fluorescence and the case of observing the second fluorescence. 27 (c) and 27 (d) are diagrams showing the results of observing fluorescence using "mosaicROTATE30". In this case, it was confirmed that the two bright spot images were distorted in the case of observing the first fluorescence and the case of observing the second fluorescence. Therefore, it was found that if the difference between the rotation angle of the first phase modulation pattern and the rotation angle of the second phase modulation pattern becomes large, it becomes impossible to properly form an image corresponding to the DH-PSF of the first fluorescence and the second fluorescence. ..

27 (e) and 27 (f) are diagrams showing the results of observing fluorescence using "mosaicSHIFT5". In this case, FIGS. 27 (g) and 27 (h) show the results of observing the fluorescence using "mosaicSHIFT10" in the case of observing the first fluorescence and the case of observing the second fluorescence. .. In this case, the two bright spot images were appropriate in the case of observing the first fluorescence and the case of observing the second fluorescence, but the bright spot image was slightly distorted as compared with "mosaicSHIFT5".

The reason why the bright spot image of "mosaicSHIFT10" is slightly distorted compared to "mosaicSHIFT5" is that the fluorescence of the center wavelength is incident on the phase modulation pattern optimal for only one center wavelength. it is obvious. That is, even if the fluorescence incident on the phase modulation pattern is the optimum fluorescence for the phase modulation pattern, the shape of the bright spot image is deformed as the center of the incident beam moves away from the center of the phase modulation pattern. Therefore, in "mosaicSHIFT10" where the amount of shift from the center is large, the bright spot image tends to collapse.

28 (a) and 28 (b) are diagrams showing the results of observing fluorescence using "mosaicEXPAND10". In this case, the two bright spot images were appropriate in the case of observing the first fluorescence and the case of observing the second fluorescence. 28 (c) and 28 (d) are diagrams showing the results of observing fluorescence using "mosaicEXPAND40". In this case, the two bright spot images were appropriate in the case of observing the first fluorescence and the case of observing the second fluorescence, but the bright spot image was slightly distorted as compared with "mosaicEXPAND10".

The reason why the bright spot image of "mosaicEXPAND40" is slightly distorted compared to "mosaicEXPAND10" is that the fluorescence of the center wavelength is incident on the phase modulation pattern optimal for only one center wavelength. it is obvious. That is, even if the fluorescence incident on the phase modulation pattern is the optimum fluorescence for the phase modulation pattern, the shape of the bright spot image is deformed as the diameter of the incident beam is farther from the diameter of the phase modulation pattern. Therefore, in "mosaicEXPAND40" where the amount of deviation in diameter is large, the bright spot image is liable to collapse.

28 (e) and 28 (f) are diagrams showing the results of observing fluorescence using "mosaicREDUCE20". In this case, the two bright spot images were appropriate in the case of observing the first fluorescence and the case of observing the second fluorescence. 28 (g) and 28 (h) are diagrams showing the results of observing fluorescence using "mosaicREDUCE40". In this case, the two bright spot images were appropriate in the case of observing the first fluorescence and the case of observing the second fluorescence, but the bright spot image was slightly distorted as compared with "mosaicREDUCE20". The reason why the bright spot image of "mosaicREDUCE40" is slightly collapsed compared to "mosaicREDUCE20" is the same as the reason why it is collapsed in the case of "mosaicEXPAND40".

<Application to phase plates made of transparent members>
An example of applying the phase modulation pattern shown in the first embodiment to a phase plate made of a transparent member will be described with reference to FIGS. 29 (a) to 29 (f).

FIG. 29A is a phase plate 91 manufactured so as to correspond to the first phase modulation pattern shown in FIG. 7A. FIG. 29 (b) is a phase plate 92 manufactured so as to correspond to the second phase modulation pattern shown in FIG. 7 (b). The phase plates 91 and 92 are made of the same material as the phase plate 52. In the phase plate 91, the portion having a large thickness corresponds to a region close to white in the first phase modulation pattern, and the portion having a small thickness corresponds to a region close to black in the first phase modulation pattern. Similarly, in the phase plate 92, the portion having a large thickness corresponds to a region close to white in the second phase modulation pattern, and the portion having a small thickness corresponds to a region close to black in the second phase modulation pattern.

The maximum thickness of the phase plate 91 is designed so that the phase of the first fluorescence incident on the maximum thickness portion is shifted by one wavelength. Similarly, the maximum thickness of the phase plate 92 is designed so that the phase of the second fluorescence incident on the maximum thickness portion is shifted by one wavelength. The phase plate 91 applies the first phase modulation to the light of the first wavelength, that is, the first fluorescence. The phase plate 92 applies the second phase modulation to the light of the second wavelength, that is, the second fluorescence.

Such phase plates 91 and 92 are synthesized in the same manner as in the phase modulation pattern of Example 1 to produce a phase plate 52 as shown in FIG. 29 (c). The thickness of the phase plate 52 is the thickness between the thickness of the phase plate for light of the first wavelength and the thickness of the phase plate for light of the second wavelength, that is, the thickness of the phase plate 91 optimal for the first fluorescence. It has a thickness between the thickness and the thickness of the phase plate 92 that is optimal for the second fluorescence. The phase plate 52 is made of a transparent member such as acrylic resin. The transparent member constituting the phase plate 52 does not necessarily have to be transparent, as long as it can transmit light.

The thickness T1 of the phase plate 91 and the thickness T2 of the phase plate 92 are calculated by the following equations. In the following equation, n1 is the refractive index around the phase plates 91 and 92, that is, the refractive index of air. n2 is the refractive index of the phase plates 91 and 92, that is, the refractive index of the phase plate 52 to be manufactured. λ1 is the center wavelength of the first fluorescence, and λ2 is the center wavelength of the second fluorescence. θ is the amount of phase shift.

Thickness T1 = λ1 × θ / {2π (n2-n1)}
Thickness T2 = λ2 × θ / {2π (n2-n1)}
In addition, T2 / T1 = λ2 / λ1.

For example, in the phase plates 91 and 92, when the maximum shift amount of the phases of the first fluorescence and the second fluorescence is θmax, the thicknesses T1 and T2 obtained by substituting θmax into the above equation are the phase plates 91 and T2, respectively. Corresponds to the maximum thickness of 92. The thickness of the phase plate 52 is set to a thickness between the maximum thickness of the phase plate 91 and the maximum thickness of the phase plate 92. Similarly, even in a region other than the region where the phase plates 91 and 92 have the maximum thickness, the thickness of the phase plate 52 is set to a thickness between the thickness T1 and the thickness T2 obtained based on the phase shift amount.

In the formula for calculating the thickness, when the range of θ is 2 (m-1) π <θ≤2mπ (m is a positive integer), the following formula is preferable. As a result, it is possible to suppress a decrease in light transmittance.

Thickness T1 = λ1 {θ-2 (m-1) π} / {2π (n2-n1)}
Thickness T2 = λ2 {θ-2 (m-1) π} / {2π (n2-n1)}

29 (d) to 29 (f) are diagrams schematically showing the cut surface when the region surrounded by the dotted line in FIGS. 29 (a) to 29 (c) is cut by a plane parallel to the thickness direction. .. The thickness of the region surrounded by the dotted line is assumed to be the first thickness H1 in the case of the phase plate 91 and the second thickness H2 in the case of the phase plate 92. The first thickness H1 is a thickness for appropriately forming an image corresponding to the DH-PSF of the first fluorescence, and the second thickness H2 is a thickness appropriately corresponding to the DH-PSF of the second fluorescence. It is the thickness for forming. Here, the phase plate 52 is manufactured based on the same method as in the case of manufacturing the phase modulation pattern of the first embodiment, and the thickness of the phase plate 52 is the thickness of the phase plate 91 and the thickness of the phase plate 92. It is distributed in the thickness between and. Therefore, assuming that the thickness of the phase plate 52 in the region surrounded by the dotted line is the third thickness H3, the third thickness H3 is calculated by the following formula.

Third thickness H3 = (first thickness H1 × a + second thickness H2 × b) / (a + b)

The phase plate 52 produced in this way appropriately obtains an image corresponding to both the first fluorescence and the second fluorescence DH-PSF, as in the case where the phase modulation pattern of the first embodiment is set in the phase modulator 51. Can be formed into. Similar to the verification result in the phase modulation pattern of Example 1, the closer the value of a and the value of b are, the more appropriately an image corresponding to both the first fluorescence and the second fluorescence can be formed. ..

Next, an example in which the phase modulation pattern shown in the second embodiment is applied to a phase plate made of a transparent member will be described with reference to FIGS. 30 (a) to 30 (i).

The phase plate 91 of FIG. 30 (a) is the same as the phase plate of FIG. 29 (a), and the phase plate 92 of FIG. 30 (b) is the same as the phase plate of FIG. 29 (b). Such phase plates 91 and 92 are arranged in a mosaic pattern in the same manner as in the phase modulation pattern of the second embodiment, and the phase plate 52 is manufactured as shown in FIG. 30 (c).

As shown in light gray in FIG. 30 (d), the phase plate 91 is manufactured so as to have an optimum thickness only for the first fluorescence in the entire region. As shown in dark gray in FIG. 30 (e), the phase plate 92 is manufactured so as to have an optimum thickness only for the second fluorescence in the entire region. That is, the phase plate 91 is set in the same manner as the first phase modulation pattern, and the phase plate 92 is set in the same manner as in the second phase modulation pattern. As shown by the mosaic shape of light gray and dark gray in FIG. 30 (f), the phase plate 52 shown in FIG. 30 (c) has a first region having an optimum thickness for the first fluorescence and an optimum thickness for the second fluorescence. The second area of thickness is arranged in a mosaic pattern.

In other words, the first region indicated by light gray in FIG. 30 (f) is a region configured to give the first phase modulation to the light of the first wavelength, that is, the first fluorescence in the incident region. The second region indicated by the dark gray in FIG. 30 (f) is a region configured to give the second phase modulation to the light of the second wavelength, that is, the second fluorescence in the incident region.

30 (g) to 30 (i) are diagrams schematically showing the cut surface when the region surrounded by the dotted line in FIGS. 30 (d) to 30 (f) is cut by a plane parallel to the thickness direction. Hereinafter, for convenience, the value of M is set to 1 in the phase modulation pattern of the second embodiment. It is assumed that the thickness of the region surrounded by the dotted line in the four pixel section is the first thickness H11 to H14 in the case of the phase plate 91 and the second thickness H21 to H24 in the case of the phase plate 92. In this case, the thickness of the phase plate 52 is such that the thickness of the phase plate 91 and the thickness of the phase plate 92 appear alternately, for example, as shown in FIG. 30 (i).

In other words, the configuration of the phase plate 52 shown in FIG. 30 (i) is as follows. As shown in FIG. 30 (f), the phase plate 52 includes a first region composed of a light gray region and a second region composed of a dark gray region in the incident region. Each region of the first region and each region of the second region are adjacent to each other. The same thickness as that of the phase plate 91 is set in the first region. A thickness similar to that of the phase plate 92 is set in the second region. As shown in FIG. 30 (i), the thicknesses of the positions corresponding to the first region are the first thicknesses H11 and H13, and the thicknesses of the positions corresponding to the second regions are the second thicknesses H22 and H24. Will be done.

The phase plate 52 produced in this way appropriately obtains an image corresponding to both the first fluorescence and the second fluorescence DH-PSF, as in the case where the phase modulation pattern of the first embodiment is set in the phase modulator 51. Can be formed into.

As described above, as a configuration example of the phase modulation mask 50 shown in FIG. 1, a phase plate 52 made of a transparent member and a phase modulator 51 having a liquid crystal panel 51a have been described. The configuration example is not limited to this. For example, the phase modulation mask 50 may be a deformable mirror provided with a minute mirror that reflects light according to the setting at different positions in the incident direction at each position in the incident surface. Alternatively, the phase modulation mask 50 may be a reflective member that reflects light at different positions in the incident direction at each position in the incident surface.

10 Optical equipment 20 Light source unit 30 Irradiation optical system 40 Stage 36 Objective lens 50 Phase modulation mask 51 Phase modulator 52 Phase plate 60 Condensing optical system 62 Imaging unit 62a Imaging surface 111 Processing unit

Claims (9)

A common phase modulation mask that provides phase modulation for light of the first wavelength and light of the second wavelength,
An irradiation optical system for incident the light having the first wavelength and the light having the second wavelength into the same incident region on the phase modulation mask.
A condensing optical system that condenses the light of the first wavelength and the light of the second wavelength that are phase-modulated by the phase modulation mask to form an image corresponding to the point image distribution function on the imaging surface. Prepare,
The phase modulation mask is a phase plate and is
The phase plate has an optimum thickness T1 for forming the image corresponding to the light of the first wavelength on the imaging surface at each incident position in the same incident region, and the second wavelength. Both the light of the first wavelength and the light of the second wavelength are point images with a thickness between T2, which is the optimum thickness for forming the image corresponding to the light on the imaging surface. have a thickness T3 which forms an image in accordance with the distribution function,
The thickness T3 is an optical instrument in which b / a is set to be in the range of 2/8 or more and 8/2 or less in the following formula.
T3 = (T1 × a + T2 × b) / (a + b)
In the above equation, both a and b are positive real numbers. The optical device according to claim 1, wherein the light having the first wavelength and the light having the second wavelength are the first fluorescence and the second fluorescence generated from a fluorescent substance, respectively. The wavelength band of the first fluorescence and the wavelength band of the second fluorescence each have a spread, and a part of the wavelength band of the first fluorescence and a part of the wavelength band of the second fluorescence overlap each other. , The optical device according to claim 2. The light having the first wavelength is light having an intensity peak at the first wavelength.
The optical device according to any one of claims 1 to 3, wherein the light having the second wavelength is light having an intensity peak at the second wavelength. The irradiation optical system includes an objective lens and has an objective lens.
The phase modulation mask, a bright spot of the first of the bright spot and the second wavelength of light having a wavelength of the light, is focused on two points on each of the imaging surface, and, with the objective lens, wherein Two bright spot images of the first wavelength light and the second bright spot image of the first wavelength light on the imaging surface, depending on the distance between the bright spot of the light of the first wavelength and the bright spot of the light of the second wavelength. In any one of claims 1 to 4, the phases of the first fluorescence and the second fluorescence are modulated so as to form the point image distribution function in which two bright spot images of light having a wavelength of the above are rotated. Described optical equipment. The optical device according to any one of claims 1 to 5, wherein the condensing optical system includes an image pickup unit that captures the image of the light of the first wavelength and the image of the light of the second wavelength. device. Equipped with a stage for installing the sample,
The irradiation optical system irradiates the sample with the light emitted from the light source unit.
The optical device according to claim 6, wherein the condensing optical system condenses the light having the first wavelength and the light having the second wavelength generated from a substance contained in the sample. The sample contains a first fluorescent dye and a second fluorescent dye.
The light source unit irradiates the sample with light so that the active state and the inactive state of the first fluorescent dye and the second fluorescent dye are repeated.
The imaging unit includes the image of the light of the first wavelength generated from the first fluorescent substance and the second image of the light generated from the first fluorescent substance while the first fluorescent substance and the second fluorescent substance repeat an active state and an inactive state. The optical device according to claim 7, wherein the image of the light having the second wavelength generated from the fluorescent substance is imaged. It is a method of forming an image corresponding to the point spread function from the light of the first wavelength and the light of the second wavelength.
The light of the first wavelength and the light of the second wavelength are placed in the same incident region on a common phase plate that applies phase modulation to the light of the first wavelength and the light of the second wavelength. Make it incident
The light of the first wavelength and the light of the second wavelength, which are phase-modulated by the phase plate, are condensed to form an image corresponding to the point image distribution function on the imaging surface .
The phase plate has an optimum thickness T1 for forming the image corresponding to the light of the first wavelength on the imaging surface at each incident position in the same incident region, and the second wavelength. Both the light of the first wavelength and the light of the second wavelength are point images with a thickness between T2, which is the optimum thickness for forming the image corresponding to the light on the imaging surface. have a thickness T3 which forms an image in accordance with the distribution function,
The thickness T3 is an image forming method in which b / a is set to be in the range of 2/8 or more and 8/2 or less in the following formula.
T3 = (T1 × a + T2 × b) / (a + b)
In the above equation, both a and b are positive real numbers.

Images