JP7770682B2 - 3D image capturing device - Google Patents

Description

The present invention relates to a three-dimensional image capturing device.

In imaging flow cytometry, it takes a long time to acquire three-dimensional images, so it was not possible to obtain three-dimensional images of cells flowing at the flow speeds (1-10 meters per second) used in general flow cytometry.
A three-dimensional microscopy method is known in which a cross section of an object is imaged using a light sheet tilted in the scanning direction, thereby capturing a three-dimensional image of the object at high speed (Non-Patent Document 1). Also, a method is known in which a three-dimensional image of an object is captured in an extremely short imaging time of the order of femtoseconds to picoseconds by stroboscopic photography using light of multiple wavelengths (Non-Patent Document 2).

"NATURE PHOTONICS", January 19, 2015, Volume 9, February 2015 issue, p. 113-119 "NATURE PHOTONICS", August 10, 2014, Volume 8, September 2014 issue, p. 695-700

In the 3D microscopy method described in Non-Patent Document 1, one frame of image is captured using a single light sheet, so multiple frames are required to capture multiple cross sections. Therefore, a high frame rate is required when capturing images of cross sections of fast-flowing cells, as in flow cytometry. On the other hand, the imaging method described in Non-Patent Document 2 acquires signals for each wavelength, making it unsuitable for capturing fluorescence images.

To implement 3D imaging in general flow cytometry, it is necessary to capture 3D images of cells flowing at the flow speeds used in general flow cytometry (1-10 meters per second) using the fluorescence emitted from the cells.
As described above, when imaging using fluorescence, it is required to be able to capture three-dimensional images at high speed.

The present invention was made in consideration of the above points, and provides a three-dimensional image capturing device that can capture three-dimensional images at high speed when capturing images using fluorescence.

The present invention has been made to solve the above-mentioned problems, and one aspect of the present invention is a three-dimensional image capturing device comprising: an irradiation unit that irradiates an object to be observed with band-shaped excitation light at predetermined time intervals; a relative position change unit that changes the relative position between the object to be observed and the excitation light in a predetermined scanning direction; and an imaging unit that captures the multiple portions with a single exposure by irradiating the excitation light onto each of multiple portions of the object to be observed at predetermined time intervals by the irradiation unit while changing the relative position by the relative position change unit, thereby changing the portion of the object to be observed that is irradiated with the excitation light in the scanning direction, and forming images of the fluorescence emitted from each of the multiple portions at different positions on an imaging element.

In another aspect of the present invention, in the above-described three-dimensional image capturing device, the relative position change unit changes the relative position in the scanning direction by moving the object to be observed in a movement direction that is different from the scanning direction, and tilting the longitudinal direction of the strip-shaped excitation light irradiated by the irradiation unit by a predetermined angle with respect to the movement direction when viewed from the lateral direction of the strip-shaped excitation light.

In another aspect of the present invention, in the above-mentioned three-dimensional image capturing device, the imaging element is a collection of elements arranged in a matrix, with signals read out in units of rows or columns, and the imaging element is arranged such that the row or column direction in which the signals of the plurality of elements are read out corresponds to the direction in which different positions on the imaging element are aligned where the fluorescence emitted from each of the plurality of portions is imaged.

In another aspect of the present invention, in the above-mentioned three-dimensional image capturing device, the relative position change unit changes the relative position in the scanning direction by changing the direction in which the excitation light is irradiated onto the fixed observation object.

The present invention enables high-speed capture of three-dimensional images when imaging using fluorescence.

1 is a diagram showing the external configuration of a cell measuring system according to a first embodiment of the present invention. 1 is a diagram illustrating an example of the functional configuration of an imaging flow cytometer according to a first embodiment of the present invention. A figure showing an example of light sheet illumination light irradiated onto a flow path in the first embodiment of the present invention. A figure showing an example of the positional relationship between light sheet illumination light and cells in the first embodiment of the present invention. FIG. 2 is a diagram showing an example of a captured image captured by an imaging section according to the first embodiment of the present invention. FIG. 2 is a diagram showing an example of an imaging result according to the first embodiment of the present invention. FIG. 2 is a diagram showing an example of a three-dimensional image generated by a three-dimensional image generating unit according to the first embodiment of the present invention. FIG. 10 is a diagram showing an example of the configuration of a cell measuring system according to a second embodiment of the present invention. A figure showing an example of the positional relationship between light sheet illumination light and cells in the second embodiment of the present invention.

(First embodiment)
A first embodiment of the present invention will be described below with reference to the drawings.
FIG. 1 is a diagram showing the external configuration of a cell measuring system 1. The cell measuring system 1 includes an imaging flow cytometer 20 and a display unit 10. The imaging flow cytometer 20 includes at least one flow path through which an object to be observed flows. Note that, although the present embodiment describes a case in which the object to be observed is a cell, the object to be observed is not limited to a cell. The object to be observed may be anything that transmits light. The cell in question is a fluorescently stained cell.

The imaging flow cytometer 20 generates a three-dimensional image of cells flowing through a flow channel. The display unit 10 displays the three-dimensional image generated by the imaging flow cytometer 20. The display unit 10 is equipped with, for example, a liquid crystal display, and displays a variety of images. The images displayed on the display unit 10 include the three-dimensional image of cells generated by the imaging flow cytometer 20.

[Functional configuration of imaging flow cytometer 20]
Next, the functional configuration of the imaging flow cytometer 20 will be described with reference to FIG.
2 is a diagram showing an example of the functional configuration of an imaging flow cytometer 20 according to this embodiment. The imaging flow cytometer 20 includes the above-described flow channel 21, an irradiation unit 22, an optical system 23, an imaging unit 24, and a control unit 25.

The flow channel 21 moves the cells C1 in the direction of the flow velocity by flowing the cells C1 together with the fluid. The flow velocity of the fluid is, for example, 1-10 meters per second. The fluid is, for example, sheath fluid.

The irradiation unit 22 irradiates the flow path 21 with excitation light P1. The irradiation unit 22 includes a light source 221. The light source generates band-shaped coherent light as the excitation light P1. The excitation light P1 is excitation light that has been formed into a band by narrowing the coherent light. The specific shape of the band-shaped excitation light will be described later.

The irradiation unit 22 irradiates the cell C1 with a band of excitation light at a predetermined time interval. The imaging flow cytometer 20 performs stroboscopic photography by irradiating the cell C1 with a band of excitation light at a predetermined time interval.

In this embodiment, the light source 221 is, for example, a pulsed laser. In this embodiment, the irradiation unit 22 irradiates the cell C1 with pulsed light as excitation light P1. Each of the multiple pulses contained in the pulsed excitation light P1 is a band-shaped excitation light. When the irradiation unit 22 irradiates pulsed light, the predetermined time interval is a predetermined period. The band-shaped excitation light is referred to as light sheet illumination light LS1.

The light source 221 may be a CW (Continuous Wave) laser. When the light source 221 is a CW laser, the irradiation unit 22 irradiates the cell C1 with light obtained by modulating continuous wave laser light at a predetermined period as excitation light P1.

The optical system 23 irradiates the excitation light P1 generated by the light source 221 at a predetermined position in the flow path 21. The optical system 23 tilts the longitudinal direction of the band-shaped light sheet illumination light LS1 irradiated by the irradiation unit 22 by a predetermined angle relative to the direction of the flow velocity in the flow path 21 when viewed from the side of the flow path 21. In this embodiment, the predetermined position in the flow path 21 at which the excitation light P1 is irradiated is determined in advance and does not change over time. The optical system 23 includes various lenses, including, for example, a dichroic mirror and an objective lens.

In the following description, the position in the flow channel 21 where the excitation light P1 is irradiated will also be referred to as the irradiation position. When a cell C1 passes through this irradiation position, the fluorescent molecules are excited by the excitation light P1, causing the cell C1 to emit light. The light resulting from this emission is fluorescence F1.

The imaging unit 24 images the fluorescence F1 from the cell C1 that has passed through the position in the flow path 21 where the excitation light P1 is irradiated. In this way, the imaging unit 24 images a certain portion of the cell C1. Here, the certain portion refers to a portion where fluorescent molecules are excited by the excitation light P1. The certain portion has a thickness approximately equal to the thickness of the excitation light, as the band of excitation light is irradiated onto the cell C1. The top or bottom surface of the certain portion is also referred to as the cross section of the cell C1.

The imaging unit 24 images multiple portions of the cell C1 with a single exposure. The imaging unit 24 includes, for example, a camera. The camera is preferably capable of high-speed imaging. Examples of the camera include a CCD (Charge Coupled Device) camera and a CMOS (Complementary MOS) camera. The camera includes an imaging element 241. The imaging element 241 is a solid-state imaging element such as a CCD or CMOS.

In this embodiment, the image sensor 241 is a collection of elements arranged in a matrix, with signals read out row by row or column by column. The image sensor 241 has a rectangular shape with more horizontal pixels than vertical pixels. The readout circuit of the image sensor 241 is configured to acquire the detected light intensity for each horizontal pixel column of the image sensor 241. As described below, the image sensor 241 is oriented so that the horizontal direction (row direction) in which signals from the multiple elements are read out coincides with the direction in which different positions on the image sensor 241 are aligned and in which fluorescence F1 emitted from each of the multiple portions of the cell C1 is imaged.

In this example, the image sensor 241 is an image sensor configured with an sCMOS (Scientific CMOS) etc. The sCMOS can capture images faster and with better image quality than conventional image sensors configured with a CCD or CMOS.
The image capturing unit 24 includes an optical system for collecting the fluorescence F1 from the cell C1 onto the image capturing element 241. A part of the optical system may also serve as a part of the optical system 23 described above.

The control unit 25 includes, for example, a CPU, a GPU (Graphics Processing Unit), an FPGA (Field-Programmable Gate Array), and other components, and performs various calculations and information exchange. The control unit 25 includes a 3D image generation unit 251 as one of its functional units.

The three-dimensional image generation unit 251 acquires an image from the image sensor 241. The three-dimensional image generation unit 251 generates a three-dimensional image of the cell C1 based on multiple cross-sectional images included in the acquired image. Generating a three-dimensional image is also referred to as reconstructing a three-dimensional image.

[Image capture method]
3 to 5, an imaging method using the imaging flow cytometer 20 will be described. Fig. 3 is a diagram showing an example of the light sheet illumination light LS1 irradiated onto the flow path 21 according to this embodiment. Fig. 4 is a diagram showing an example of the positional relationship between the light sheet illumination light LS1 and the cell C1 according to this embodiment.

Figures 3 and 4 show an xyz coordinate system as a three-dimensional Cartesian coordinate system. In this embodiment, the x-axis direction is the length direction of the flow channel 21. The x-axis direction is the direction in which cell C1 flows in the flow channel 21. Cell C1 flows in the +x direction of the x-axis. The y-axis direction is the width direction (also called the depth direction) of the flow channel 21. The z-axis direction is the height direction of the flow channel 21. In the following description, in the width direction of the flow channel 21, the positive side of the y-axis direction may be referred to as the "right side," and the negative side of the y-axis direction may be referred to as the "left side." The positive side of the z-axis direction may be referred to as the "upper side," and the negative side of the z-axis direction may be referred to as the "lower side."

Figure 3 (A) shows the flow channel 21 when viewed from above. Figure 3 (B) shows the flow channel 21 when viewed from the side (e.g., the left side). In this embodiment, the fluid flowing through the flow channel 21 is maintained in a steady flow state. Therefore, while the cell C1 flows through the flow channel 21, at least during the course of imaging by the imaging unit 24, the position of the cell C1 in the width direction of the flow channel 21, the position of the cell C1 in the height direction of the flow channel 21, and the orientation of the cell C1 relative to the top or bottom surface of the flow channel 21 do not change. In other words, the cell C1 flows straight through the flow channel 21 from upstream to downstream (+x direction) without rotating, at least during the course of imaging by the imaging unit 24.

As shown in Figure 3 (B), the light sheet illumination light LS1 is rotated from a direction parallel to the bottom surface of the flow path 21 (parallel to the xy plane) to a direction parallel to the width direction of the flow path 21 (y-axis direction) as the rotation axis, and is irradiated at a predetermined angle relative to the direction in which the cell C1 flows in the flow path 21 (x-axis direction).
The optical system provided in the imaging flow cytometer 20 tilts the direction of the surface of the light sheet illumination light LS1 emitted by the irradiation unit 22 by a predetermined angle with respect to the direction of the flow velocity.

Note that Figure 3(B) describes an example in which the light sheet illumination light LS1 is tilted in a direction in which the height of the upstream side is lower than the height of the downstream side when viewed from the side of the flow path 21, but this is not limited to this. The light sheet illumination light LS1 may also be tilted in a direction in which the height of the upstream side is higher than the height of the downstream side when viewed from the side of the flow path 21.

As described above, in this embodiment, the excitation light P1 is pulsed light. The light sheet illumination light LS1 is one pulse contained in the excitation light P1. Therefore, when the pulse contained in the excitation light P1 is irradiated onto the flow path 21, the flow path 21 is in a state where it is irradiated with the light sheet illumination light LS1. Figure 4 corresponds to the case where the light sheet illumination light LS1 is irradiated onto the flow path 21. On the other hand, when the pulse is not irradiated onto the flow path 21, the flow path 21 is in a state where it is not irradiated with the light sheet illumination light LS1.

The flow path 21 and the optical system 23 are an example of a relative position change unit. In other words, in this embodiment, the relative position change unit moves the cell C1 through the flow path 21 in the direction of the flow velocity of the flow path 21, which is a direction different from the scanning direction, and changes the relative position between the cell C1 and the light sheet illumination light LS1 in the scanning direction by tilting the longitudinal direction of the strip-shaped light sheet illumination light LS1 emitted by the irradiation unit 22 by a predetermined angle with respect to the direction of the flow velocity of the flow path 21 when viewed from the short side of the strip-shaped light sheet illumination light LS1. Therefore, the relative position change unit changes the relative position between the cell C1 and the light sheet illumination light LS1 in the predetermined scanning direction. Note that in this embodiment, the short side of the strip-shaped light sheet illumination light LS1 coincides with the width direction (depth direction or y-axis direction) of the flow path 21.

In this embodiment, scanning refers to irradiating multiple portions (cross sections) of the object to be observed with excitation light at predetermined time intervals, and capturing images of the multiple portions based on the fluorescence emitted from each of the multiple portions. The scanning direction is the direction in which the multiple portions (cross sections) of the object to be observed that are irradiated with excitation light at predetermined time intervals are lined up.

Figure 4 shows a time series of the light emitted by each part of the cell C1 flowing through the flow path 21 at multiple times when the light sheet illumination light LS1 is irradiated onto the flow path 21. The multiple times are indicated by six times, "t=1" to "t=6," in Figure 4. In this embodiment, the excitation light P1 is pulsed light. In other words, the light sheet illumination light LS1 is irradiated onto the flow path 21 at a predetermined cycle.

As described above, the light sheet illumination light LS1 is tilted at a predetermined angle relative to the direction in which the cell C1 flows in the flow channel 21 (the x-axis direction). Therefore, the portion of the cell C1 irradiated with the light sheet illumination light LS1 changes in a predetermined scanning direction as the cell C1 flows through the flow channel 21. The scanning direction is the normal direction to the plane of the light sheet illumination light LS1. The multiple pulses of the light sheet illumination light LS1 contained in the excitation light P1 are each irradiated to a different portion of the cell C1. In other words, the portion of the cell C1 flowing through the flow channel 21 is scanned in the normal direction to the plane of the strip-shaped light sheet illumination light LS1.

In this embodiment, the cell C1 flowing through the flow channel 21 passes through an area irradiated with the light sheet illumination light LS1, whose position does not change relative to the flow channel 21, at a constant speed. Therefore, each portion (cross section) of the cell C1 is irradiated with the light sheet illumination light LS1 at equal intervals in the scanning direction.

The area irradiated with the light sheet illumination light LS1 is a planar area where the strip-shaped light sheet illumination light LS1 and the cell C1 intersect. The part of the cell C1 irradiated with the light sheet illumination light LS1 emits light. Because the area irradiated with the light sheet illumination light LS1 changes as the cell C1 flows through the flow path 21, the part of the cell C1 that emits light changes as the cell C1 flows through the flow path 21.

As described above, the light sheet illumination light LS1 is a strip of excitation light. The width direction of the light sheet illumination light LS1 coincides with the width direction of the flow path 21. The length direction of the light sheet illumination light LS1 is inclined at a predetermined angle relative to the length direction of the flow path 21. The thickness (height) direction of the light sheet illumination light LS1 is inclined at a predetermined angle relative to the height direction of the flow path 21 when viewed from the side. The predetermined angle is, for example, an angle greater than 0 degrees and less than 45 degrees.

The width of the light sheet illumination light LS1 is sufficiently longer than the diameter of the cell C1, which is the object of observation. In this embodiment, the width of the light sheet illumination light LS1 is longer than the width of the flow path 21. The height (thickness) of the light sheet illumination light LS1 is sufficiently shorter than the diameter of the cell C1. The position of the intersection between the light sheet illumination light LS1 and the cell C1 changes as the cell C1 flows through the flow path 21, but the area of the light sheet illumination light LS1 consisting of these parts is narrow enough to be included in the vertical angle of view of the image sensor 241 included in the imaging unit 24. The direction of the vertical angle of view of the image sensor 241 is the length direction of the light sheet illumination light LS1.

The imaging unit 24 is installed so that the direction of the optical axis of the camera lens coincides with the direction of the normal to the surface of the light sheet illumination light LS1. Note that the direction of the optical axis of the camera lens provided in the imaging unit 24 may be tilted from the direction of the normal to the surface of the light sheet illumination light LS1.
The imaging unit 24 images the multiple parts of the cell C1 with a single exposure by irradiating the light sheet illumination light LS1 onto each of the multiple parts of the cell C1 at a predetermined time interval and forming images of the fluorescence emitted from each of the multiple parts of the cell C1 at different positions on the imaging element 241. The imaging unit 24 maintains the exposure state while imaging the multiple parts of one cell C1.

The image capturing unit 24 preferably opens and closes the shutter based on, for example, the following shutter opening and closing conditions. However, the following shutter opening and closing conditions are not essential. For example, the image capturing unit 24 opens the shutter to start exposure when the cell C1 flowing through the flow path 21 passes a predetermined position (exposure start position) in the flow path 21. The exposure start position is, for example, a position upstream of the irradiation position of the excitation light P1 in the flow path 21 and outside the angle of view of the image capturing element 241. The image capturing unit 24 closes the shutter and ends the exposure when a predetermined time has elapsed since the start of exposure. However, the image capturing unit 24 may also end exposure when the cell C1 passes a predetermined position (exposure end position) in the flow path 21. The exposure end position is, for example, a position downstream of the irradiation position of the excitation light P1 in the flow path 21 and outside the angle of view of the image capturing element 241.
The imaging unit 24 opens and closes the shutter in accordance with the position of the cell C1 flowing through the flow channel 21. Therefore, the imaging unit 24 opens and closes the shutter in synchronization with the movement of the object to be observed.

Multiple cells C1 may be flowed through the flow path 21, and the imaging unit 24 may capture an image of each of the multiple cells C1. In this case, the imaging unit 24 captures an image of one cell C1 per exposure. The imaging unit 24 repeatedly opens and closes the shutter to sequentially capture an image of each of the multiple cells C1 flowing through the flow path 21.

When multiple cells C1 are flowed through the flow path 21, the distance between the multiple cells C1 is set to a level that prevents another cell C1 from entering the field of view of the imaging element 241 while the imaging unit 24 is imaging a certain cell C1.
If the width of the flow channel 21 is sufficiently wide, images of a plurality of cells flowing at different positions in the width direction of the flow channel 21 may be captured in a single exposure.

Figure 5 is a diagram showing an example of an image T1 captured by the imaging unit 24 according to this embodiment. Figure 5 shows an XY coordinate system as a two-dimensional Cartesian coordinate system. In this embodiment, the X direction is the horizontal direction of the imaging element 241. The X direction coincides with the direction of the flow velocity of the flow channel 21 (the x-axis direction in Figures 3 and 4). The Y direction is the vertical direction of the imaging element 241.

In Figure 5, cross-sectional images, which are images of portions of cell C1 that have emitted light at multiple times, are captured as one frame of image at different positions on the image sensor 241. In captured image T1, fluorescence emitted from portions of cell C1 at six times, respectively indicated by "t=1" to "t=6," is focused at different positions on the image sensor 241 and captured as one frame of image.

The imaging unit 24 images multiple parts of the cell C1 with a single exposure by irradiating each of the multiple parts of the cell C1 with the light sheet illumination light LS1 at predetermined time intervals using the irradiation unit 22 while changing the relative position of the imaging unit 24 with the light sheet illumination light LS1 in the scanning direction by changing the portion of the cell C1 irradiated with the light sheet illumination light LS1 using the flow path 21 and the optical system 23, and forming images of the fluorescence emitted from each of the multiple parts of the cell C1 at different positions on the imaging element 241. In other words, the imaging unit 24 images multiple parts of the cell C1 in a single frame without requiring multiple frames.

As described above, the image sensor 241 simultaneously acquires the detected light intensity for each pixel row in the horizontal direction (X direction). Furthermore, the image sensor 241 simultaneously acquires the light intensity for all horizontal pixel rows aligned in the vertical direction (Y direction). In other words, the camera provided in the imaging unit 24 uses a global shutter. Therefore, the imaging time required to capture one captured image T1 increases in proportion to the time required to scan in the vertical direction (Y direction). In other words, the imaging time required to capture one captured image T1 increases in proportion to the number of pixels in the vertical direction. Therefore, the number of images that the image sensor 241 can capture per unit time is inversely proportional to the number of pixels in the vertical direction and is independent of the number of pixels in the horizontal direction.

In this embodiment, the image sensor 241 is oriented so that the horizontal direction (X direction) in which signals from multiple elements are read out coincides with the direction in which different positions on the image sensor 241 are aligned, where the fluorescence F1 emitted from each of the multiple portions of the cell C1 is imaged. Therefore, the imaging speed of the imaging flow cytometer 20 does not depend on the number of pixels in the horizontal direction of the image sensor 241. The imaging speed of the imaging flow cytometer 20 is not limited by the imaging speed of the camera itself used in the imaging unit 24.

The image sensor 241 may be oriented in a direction other than the direction in which the horizontal direction (X direction) in which signals from the multiple elements are read out coincides with the direction in which different positions on the image sensor 241 are aligned, where the fluorescence F1 emitted from each of the multiple portions of the cell C1 is imaged. Even in this case, the image sensor 241 is preferably oriented such that the row or column direction in which signals from the multiple elements are read out coincides with the direction in which different positions on the image sensor 241 are aligned, where the fluorescence F1 emitted from each of the multiple portions of the cell C1 is imaged. The direction in which the different positions are aligned is, for example, a direction tilted at an angle greater than 0 degrees and less than a predetermined angle (e.g., 45 degrees) from the direction in which the different positions are aligned.

In addition, in this embodiment, the image sensor 241 is a collection of elements in which multiple elements are arranged in a matrix and signals are read out in units of rows or columns, and an example has been described in which the image sensor 241 has a rectangular shape in which the number of pixels arranged horizontally is greater than the number of pixels arranged vertically, but this is not limited to this. The image sensor 241 may also be square in shape, in which the number of pixels arranged horizontally and the number of pixels arranged vertically are equal.

In this embodiment, the three-dimensional image generation unit 251 combines multiple cross-sectional images of the cell C1 captured in the captured image T1 in an order based on the scanning direction. The three-dimensional image generation unit 251 acquires information in advance indicating the relationship between the scanning direction and the vertical and horizontal directions of the image sensor 241. Based on the information indicating this relationship, the three-dimensional image generation unit 251 determines the order in which to combine the multiple cross-sectional images captured in the captured image T1. The three-dimensional image generation unit 251 generates a three-dimensional image by combining the multiple cross-sectional images based on the determined order.

The tilt of the light sheet illumination light LS1 is determined based on, for example, the flow rate of the fluid flowing through the flow path 21, the pulse period contained in the excitation light P1, and the desired resolution in the depth (thickness) direction of the cell C1 portion. The depth (thickness) direction of the cell C1 portion is the direction in which the cell C1 is scanned by the light sheet illumination light LS1.

[Image results]
6 is a diagram showing an example of an imaging result according to this embodiment. Note that the captured image shown in FIG. 6 shows a scale of "30 μm."
The captured image T10 is an image captured when the flow velocity of the fluid flowing through the flow channel 21 is 1.3 m/s and a CW laser modulated at a frequency of 50 kHz is used as the light source 221.
The captured image T20 is the result of imaging when the flow velocity of the fluid flowing through the flow channel 21 is 11.8 m/s and a CW laser modulated at a frequency of 500 kHz is used as the light source 221.
The captured image T30 is the result of capturing an image when the flow velocity of the fluid flowing through the flow channel 21 is 0.98 m/s and a Q-switched laser with a pulse frequency of 48 kHz is used as the light source 221.

Comparing the imaging conditions for captured image T20 with those for captured images T10 and T30, the flow velocity under the imaging conditions for captured image T20 is approximately 10 times that under the imaging conditions for captured images T10 and T30. Accordingly, the frequency of the CW laser used to capture captured image T20 is set to approximately 10 times the frequency of the CW laser used to capture image T10 or the frequency of the Q-switched laser used to capture image T30.

In general, a Q-switched laser emits a greater amount of light per pulse than a CW laser. Therefore, when a Q-switched laser is used as the light source 221, the contrast of the cross-sectional image of cell C1 captured in the cross-sectional image can be increased compared to when a CW laser is used as the light source 221.

Figure 7 is a diagram showing an example of a three-dimensional image generated by the three-dimensional image generation unit 251 according to this embodiment. Figure 7 shows the result of reconstructing a three-dimensional image of cell C1.

In this embodiment, an example has been described in which the optical system 23 tilts the plane of the light sheet illumination light LS1 by a predetermined angle relative to the direction of the flow velocity of the flow path 21, but this is not limited to this. The optical system 23 may make the plane of the light sheet illumination light LS1 parallel to the upper surface (or lower surface) of the flow path 21, and change the position of the light sheet illumination light LS1 in the depth direction.

In this embodiment, an example has been described in which the irradiating unit 22 irradiates the cell C1 with the light sheet illumination light LS1 at a predetermined period, but this is not limited to this. The irradiating unit 22 may also irradiate the cell C1 with the light sheet illumination light LS1 at non-periodic time intervals. For example, as described above, when the surface of the light sheet illumination light LS1 is parallel to the upper surface (or lower surface) of the flow channel 21 and the position of the light sheet is changed in the depth direction, the irradiating unit 22 may irradiate the cell C1 with the light sheet illumination light LS1 at non-periodic time intervals.

Furthermore, for example, if the movement speed of cell C1 changes while passing through an area irradiated with light sheet illumination light LS1 whose position does not change, the irradiation unit 22 may irradiate light sheet illumination light LS1 onto cell C1 at non-periodic time intervals in accordance with the change in movement speed.

In this embodiment, an example has been described in which cell C1 flows straight through flow channel 21 from upstream to downstream (+x direction) without rotating, at least during imaging by imaging unit 24, but this is not limited to this. For example, control unit 25 may include a detection unit that detects a deviation in the orientation of cell C1 from before the rotation when cell C1 rotates. In the process of reconstructing a three-dimensional image, three-dimensional image generation unit 251 corrects the deviation caused by the rotation of cell C1 based on the deviation detected by the detection unit.

As described above, the three-dimensional image capturing device according to this embodiment (in this embodiment, the imaging flow cytometer 20) comprises an irradiation unit 22, a relative position change unit (in this embodiment, the flow path 21 and the optical system 23), and an imaging unit 24.
The irradiation unit 22 irradiates the observation object (in this embodiment, cell C1) with band-shaped excitation light (in this embodiment, light sheet illumination light LS1) at a predetermined time interval.
The relative position change unit (in this embodiment, the flow path 21 and the optical system 23) changes the relative position between the object to be observed (in this embodiment, the cell C1) and the excitation light (in this embodiment, the light sheet illumination light LS1) in a predetermined scanning direction (in this embodiment, the normal direction of the surface of the light sheet illumination light LS1).
The imaging unit 24 images multiple parts (in this embodiment, multiple parts of the cell C1) with a single exposure by changing the relative position (in this embodiment, the relative position between the cell C1 and the light sheet illumination light LS1) using a relative position change unit (in this embodiment, the flow path 21 and the optical system 23), thereby changing the part of the object to be observed (in this embodiment, the cell C1) to be irradiated with the excitation light (in this embodiment, the light sheet illumination light LS1) in the scanning direction (in this embodiment, the normal direction of the surface of the light sheet illumination light LS1), and by irradiating the excitation light (in this embodiment, the light sheet illumination light LS1) to each of multiple parts of the object to be observed (in this embodiment, the cell C1) at a predetermined time interval using the irradiation unit 22, and imaging the fluorescence emitted from each of the multiple parts (in this embodiment, multiple parts of the cell C1) at different positions on the imaging element 241.

With this configuration, the three-dimensional imaging device according to this embodiment (in this embodiment, the imaging flow cytometer 20) can image multiple portions of an object to be observed with a single exposure, allowing for high-speed imaging of three-dimensional images when imaging using fluorescence. Here, "high-speed" means faster than when multiple portions of an object to be observed are each imaged in multiple frames. The three-dimensional imaging device according to this embodiment does not require a high frame rate because it can image multiple portions of an object to be observed with a single exposure.

In addition, in the three-dimensional image capturing device according to this embodiment (in this embodiment, the imaging flow cytometer 20), the relative position change unit (in this embodiment, the flow path 21 and the optical system 23) moves the object to be observed (in this embodiment, the cell C1) in a movement direction (in this embodiment, the direction of the flow velocity of the flow path 21), which is different from the scanning direction (in this embodiment, the direction normal to the surface of the light sheet illumination light LS1), and changes the relative position (in this embodiment, the relative position between the cell C1 and the light sheet illumination light LS1) in the scanning direction (in this embodiment, the direction normal to the surface of the light sheet illumination light LS1) by tilting the longitudinal direction of the band-shaped excitation light (in this embodiment, the light sheet illumination light LS1) emitted by the irradiation unit 22 by a predetermined angle with respect to the movement direction (in this embodiment, the direction of the flow velocity of the flow path 21) when viewed from the short side of the band-shaped excitation light (in this embodiment, the light sheet illumination light LS1).

With this configuration, the 3D image capturing device according to this embodiment (in this embodiment, the imaging flow cytometer 20) can capture multiple portions of a moving object with a single exposure when capturing a 3D image of the object. This allows for high-speed capture of 3D images when using fluorescence imaging. Here, "high-speed" means faster than capturing multiple portions of a moving object in multiple frames.

As explained above, the three-dimensional image capturing device according to this embodiment is suitable for use as an imaging flow cytometer 20. In flow cytometry using a sheath flow, cells flow through a flow channel at a flow velocity of 1-10 m/s. With the above configuration, the imaging flow cytometer 20 can capture images of cells flowing at a flow velocity of 1-10 m/s.
Furthermore, the imaging flow cytometer 20 can instantly acquire three-dimensional images of cells flowing through the flow channel 21, making it possible to instantly distinguish cells using machine learning and to implement a cell sorter.

In this embodiment, an example has been described in which the cell C1, which is the object to be observed, is moved in a movement direction that is different from the scanning direction by flowing the cell C1 through the flow channel 21, but this is not limited to this. As long as the object to be observed can be moved in a movement direction that is different from the scanning direction, a mechanism other than a flow channel may be used.

In addition, in the three-dimensional image capturing device according to this embodiment (in this embodiment, the imaging flow cytometer 20), the image capturing element 241 is a collection of elements in which multiple elements are arranged in a matrix and signals are read out in units of rows or columns, and the image capturing element 241 is arranged so that the row or column direction in which signals from the multiple elements are read out corresponds to the direction in which different positions on the image capturing element 241 are aligned at which fluorescence F1 emitted from each of multiple portions of the observation object (in this embodiment, the cell C1) is imaged.

With this configuration, in the three-dimensional image capturing device according to this embodiment (in this embodiment, the imaging flow cytometer 20), the imaging speed is not limited by the readout speed of the imaging element 241 compared to when the imaging element 241 is not arranged so that the row or column direction in which signals from multiple elements are read out corresponds to the direction in which different positions on the imaging element 241 at which images of the fluorescence F1 emitted from each of the multiple portions of the observation object (in this embodiment, the cell C1) are formed are aligned. This allows for faster imaging of three-dimensional images. In particular, when the imaging element 241 is arranged so that the row or column direction in which signals from multiple elements are read out corresponds to the direction in which different positions on the imaging element 241 at which images of the fluorescence F1 emitted from each of the multiple portions of the observation object (in this embodiment, the cell C1) are aligned, the imaging speed is not limited by the readout speed of the imaging element 241.

Second Embodiment
The second embodiment of the present invention will be described in detail below with reference to the drawings.
In the first embodiment, the three-dimensional image capturing device was described as a case where the relative position of the observation object and the excitation light (light sheet illumination light) changes as the observation object flows through the flow channel. In this embodiment, the position of the observation object is fixed, and a polygonal mirror is used to change the relative position of the excitation light with respect to the observation object.
The cell measuring system according to this embodiment is referred to as a cell measuring system 1a.

Fig. 8 is a diagram showing an example of the configuration of a cell measurement system 1a according to this embodiment. Fig. 9 is a diagram showing an example of the positional relationship between the light sheet illumination light LS2 and the cell C2 according to this embodiment.
8 and 9 show an xyz coordinate system as a three-dimensional Cartesian coordinate system. The z-axis is oriented vertically upward. The x-axis and y-axis are parallel to the horizontal plane.

The cell measurement system 1a includes a light source D1, a camera CM1, and an optical system O1. The optical system O1 includes various lenses, various mirrors, an aperture A1, and the like. The various lenses included in the optical system O1 include lens L1, lens L2, lens L3, objective lens L4, lens L5, lens L6, and lens L7. The various mirrors included in the optical system O1 include a dichroic mirror M1, a polygonal mirror M2, a half mirror M3, and a tilted mirror M4.

Lens L1, lens L2, lens L3, lens L5, and lens L7 are, for example, tube lenses and achromatic lenses. An achromatic lens is a lens made by bonding together two optical elements, for example, crown glass, which has a low refractive index, and flint glass, which has a high refractive index.

The light source D1 emits excitation light E1. The excitation light E1 is pulsed light. Each pulse contained in the excitation light E1 is a band-shaped coherent light. The light source D1 is, for example, a pulse laser.
The excitation light E1 emitted from the light source D1 is reflected by the dichroic mirror M1. The excitation light E1 reflected by the dichroic mirror M1 is collected by the lens L1.

Excitation light E1 focused by lens L1 is reflected by polygonal mirror M2 as excitation light E2. The excitation light E2 reflected by polygonal mirror M2 is focused by lens L2, passes through aperture A1, and then focused by lens L3 to enter objective lens L4. Each of the multiple pulses contained in excitation light E2 is irradiated via objective lens L4 as light sheet illumination light LS2 onto cell C2, the object of observation.

Here, the polygonal mirror M2 has multiple reflecting surfaces. In the example shown in Figure 8, the polygonal mirror M2 has eight reflecting surfaces. The polygonal mirror M2 has a rotation mechanism and can rotate at high speed around a rotation axis parallel to the y-axis direction.

Polygonal mirror M2 reflects the multiple pulses contained in excitation light E2 from one of its reflecting surfaces while rotating around its axis of rotation, changing the angle of the reflecting surface relative to the incident direction of excitation light E2. As the multiple pulses are reflected by the reflecting surface, the angle of the reflecting surface relative to the incident direction of excitation light E2 changes, so the multiple pulses are reflected in different directions. Polygonal mirror M2 rotates at a rotational speed such that each of the multiple pulses is irradiated onto a different part of cell C2, which is the object of observation.

Figure 9 shows an example of the positional relationship between the light sheet illumination light LS2 and cell C2 according to this embodiment. The light sheet illumination light LS2 corresponds to one of the multiple pulses contained in the excitation light E2. As the orientation of the reflective surface changes with the rotation of the polygonal mirror M2, the portion of cell C2 irradiated with the light sheet illumination light LS2 changes in a predetermined direction. This predetermined direction is, for example, the normal direction of the surface of the light sheet illumination light LS2 (the direction indicated by the arrow AR1 in Figure 9). In other words, the portion of cell C2 is scanned in a predetermined direction (scanning direction).

Polygonal mirror M2 is an example of a relative position changer. In other words, polygonal mirror M2 changes the direction in which excitation light is irradiated onto a fixed observation object, thereby changing the relative position between the observation object and the excitation light in the scanning direction.

Returning to FIG. 8, the description of the configuration of the cell measuring system 1a will be continued.
The light sheet illumination light LS2 is irradiated onto a portion of the cell C2, and the fluorescence F1 emitted from that portion is incident on the polygonal mirror M2 via the objective lens L4, the lens L3, the aperture A1, and the lens L2. The fluorescence F1 incident on the polygonal mirror M2 is reflected by the reflective surface of the polygonal mirror M2 toward the lens L1.

Fluorescence F1 reflected by the reflective surface of polygonal mirror M2 and collected by lens L1 enters dichroic mirror M1 as fluorescence F2. Dichroic mirror M1 transmits the incident fluorescence F2 toward lens L5. Fluorescence F2 that passes through dichroic mirror M1 is collected by lens L5 and enters half mirror M3 as fluorescence F3. Half mirror M3 transmits the incident fluorescence F3 toward lens L6.

Lens L6 focuses the incident fluorescence F3 to form an image of a portion (cross section) of cell C2. The fluorescence F3 imaged by lens L6 is reflected by tilted mirror M4, which is tilted at a predetermined angle (e.g., 45 degrees) relative to the optical axis of lens L6.

Fluorescence F3 reflected by tilted mirror M4 enters half mirror M3 via lens L6 as fluorescence F4. Fluorescence F4 that enters half mirror M3 is reflected by half mirror M3 toward lens L7. Lens L7 collects fluorescence F4 reflected by half mirror M3. Fluorescence F4 collected by lens L7 forms an image on the image sensor of camera CM1.

Camera CM1 captures multiple parts of cell C2 with a single exposure by irradiating light sheet illumination light LS2 at predetermined time intervals and imaging the fluorescence emitted from multiple parts of cell C2 at different positions on the imaging element. Camera CM1 maintains the exposure state while capturing images of multiple parts of a single cell C2.

As described above, in the three-dimensional image capturing device according to this embodiment (in this embodiment, cell measurement system 1a), the relative position change unit (in this embodiment, polygonal mirror M2) changes the direction in which excitation light (in this embodiment, light sheet illumination light LS2) is irradiated onto a fixed observation object (in this embodiment, cell C2), thereby changing the relative position (in this embodiment, the relative position between cell C2 and light sheet illumination light LS2) in the scanning direction (in this embodiment, the normal direction to the surface of light sheet illumination light LS2).

With this configuration, the 3D image capturing device according to this embodiment (in this embodiment, cell measurement system 1a) can capture 3D images of a stationary object of observation with a single exposure, thereby capturing 3D images faster than when capturing images of multiple portions of a stationary object of observation in multiple frames each.

In the above-described embodiments, an example has been described in which the scanning direction when irradiating multiple portions of the observation object with band-like excitation light is a direction indicated by a straight line, but this is not limited to this. The scanning direction may also be an angular direction. In other words, excitation light may be irradiated at predetermined time intervals onto multiple portions (cross sections) of the observation object viewed from different directions in the angular direction of the observation object, and cross-sectional images of the multiple portions in the different directions may be captured.

In each of the above-described embodiments, the order in which multiple portions of the observation object are scanned does not have to be consecutive in the scanning direction. For example, in the first embodiment, when the surface of the light sheet illumination light LS1 is not tilted with respect to the upper surface (lower surface) of the flow channel 21 and the position of the light sheet illumination light LS1 is changed in the depth direction, the order in which the multiple portions of the cell C1 are irradiated with the light sheet illumination light LS1 at predetermined time intervals does not have to be consecutive in the height direction (z-axis direction), which is the scanning direction.

In each of the above-described embodiments, an example has been described in which the observation object, which is the object for capturing a three-dimensional image, is a cell, but this is not limited to this. The observation object may be a particle, a macroscopic object, or the like, as long as it is a substance that emits fluorescence when irradiated with excitation light.

Note that each of the above-mentioned devices has an internal computer. The processing steps of each of the above-mentioned devices are stored in the form of a program on a computer-readable recording medium, and the computer reads and executes this program to perform the above-mentioned processing. Here, computer-readable recording medium refers to a magnetic disk, magneto-optical disk, CD-ROM, DVD-ROM, semiconductor memory, etc. Alternatively, the computer program may be distributed to a computer via a communication line, and the computer that receives the program may execute the program.

The program may also be for realizing part of the above-mentioned functions.
Furthermore, the above-mentioned functions may be realized in combination with a program already recorded in the computer system, that is, a so-called differential file (differential program).

One embodiment of the present invention has been described in detail above with reference to the drawings, but the specific configuration is not limited to that described above, and various design modifications can be made without departing from the spirit of the present invention.

1, 1a... cell measurement system, 20... imaging flow cytometer, 21... flow path, 22... illumination unit, 23... optical system, 24... imaging unit, 241... imaging element, LS1, LS2... light sheet illumination light, C1, C2... cells

Claims (4)

an irradiation unit that irradiates the object to be observed with a strip of excitation light at predetermined time intervals;
a relative position change unit that changes the relative position between the object to be observed and the excitation light in a predetermined scanning direction;
an imaging unit that images the plurality of portions with a single exposure by irradiating the excitation light onto each of the plurality of portions of the observation object at predetermined time intervals using the irradiation unit while changing the relative position using the relative position change unit to change the portion of the observation object that is irradiated with the excitation light in the scanning direction, and forming images of fluorescence emitted from each of the plurality of portions at different positions on an imaging element;
A three-dimensional image capturing device comprising: 2. The three-dimensional image capturing device according to claim 1, wherein the relative position changing unit changes the relative position in the scanning direction by moving the object to be observed in a movement direction that is different from the scanning direction and tilting a longitudinal direction of the strip-shaped excitation light irradiated by the irradiation unit by a predetermined angle with respect to the movement direction when viewed from a lateral direction of the strip-shaped excitation light. The imaging element is a group of elements in which a plurality of elements are arranged in a matrix and signals are read out in units of rows or columns,
3. The three-dimensional image capturing device according to claim 2, wherein the imaging element is arranged such that a row or column direction in which signals from the plurality of elements are read out corresponds to a direction in which different positions on the imaging element at which fluorescence emitted from each of the plurality of portions is imaged are aligned. The three-dimensional image capturing device according to claim 1 , wherein the relative position changing unit changes the relative position in the scanning direction by changing a direction in which the excitation light is irradiated onto the fixed observation target.