JP7838232B2 - Microscope system, imaging method, and program - Google Patents

Description

This disclosure relates to a microscope system, an imaging method, and a program.

Patent Document 1 describes an optical image separation device connected to the confocal image extraction port of a Nipow disk type confocal scanner.

Patent Document 2 describes a confocal scanner that performs optical scanning by rotating a pinhole disk, which has multiple types of pinholes of different diameters arranged on it, allowing light to selectively pass through only one of the pinholes of a particular diameter.

Patent Document 3 describes a drug discovery screening apparatus comprising means for acquiring the fluorescence signal from a sample as a fluorescence image and means for acquiring the fluorescence signal from a sample as a confocal image, and performing optical path switching to obtain either the fluorescence image or the confocal image.

Patent Document 4 describes a microscope system that includes a confocal scanner for acquiring slice images of a sample as confocal images, and an actuator that moves the focal position of the microscope's objective lens in the optical axis direction, thereby performing imaging while moving the focal position of the objective lens in the optical axis direction.

Japanese Patent Publication No. 2002-62480 Japanese Patent Publication No. 2011-90145 Japanese Patent Publication No. 2008-64671 Japanese Patent Publication No. 2005-70689

When imaging cells under a microscope, well plates are commonly used as observation containers. The frame and bottom surface of well plates can warp due to distortions during molding (the distortion of the frame is approximately several hundred micrometers, and the distortion of the bottom surface is approximately several tens of micrometers), which can result in the sample surface being tilted relative to the microscope's imaging plane.

Therefore, when confocal imaging is performed using the configuration of Patent Document 1 with the sample surface tilted relative to the imaging plane, the shallow depth of field results in a significant decrease in brightness in a portion of the field of view. While imaging using the configurations of Patent Documents 2 to 4 can prevent this decrease in brightness in a portion of the field of view, it presents challenges such as limitations in mechanical design, increased required image storage capacity, and increased imaging time.

The purpose of this disclosure is to provide a microscope system, imaging method, and program that can capture clear images of a sample without partial brightness reduction within the field of view, even when the sample surface is tilted, without mechanical design limitations or significant reductions in imaging speed.

Some embodiments of the microscope system are microscope systems for acquiring an image of a sample to be observed, and include a microscope having a light-emitting device for irradiating illumination light, a confocal scanner having a scanning unit for scanning the illumination light on the sample, an objective lens for projecting an image of the illumination light scanned by the confocal scanner onto the sample, and a position sensor for detecting the position of a container on which the sample is placed relative to the focal position of the objective lens; an imaging device having an imaging surface positioned conjugate to the scanning unit for detecting the light to be measured from the sample irradiated with the illumination light; and a processing unit for controlling the operation of the microscope and the imaging device, wherein the processing unit transmits control signals to the microscope and the imaging device for performing imaging by the imaging device, while displacing the relative position of the focal point of the objective lens with respect to the sample in a direction parallel to the optical axis of the illumination light, based on the position of the container relative to the focal position of the objective lens. In this way, by performing confocal imaging while displacing the focal position of the objective lens on the sample in a direction parallel to the optical axis of the illumination light, it is possible to acquire a clear image without localized brightness reduction within the field of view, even when the sample surface is tilted. The microscope system does not require a complex configuration, and therefore does not impose limitations on mechanical design. Since the microscope system acquires the image in a single scan, the imaging speed is not significantly reduced. Therefore, even when the sample surface is tilted, it is possible to acquire a clear image without localized brightness reduction within the field of view, without mechanical design limitations or significant reductions in imaging speed.

In one embodiment, the confocal scanner rotates a pinhole disk, which has a plurality of pinholes arranged on it, as the scanning unit, to scan the illumination light emitted from the light-emitting device onto the sample. The objective lens projects the images of the plurality of pinholes, which are scanned by the illumination light from the confocal scanner, onto the sample. The imaging device may have its imaging surface positioned conjugate to the plurality of pinholes. In this way, by rotating the pinhole disk at high speed and scanning the illumination light onto the sample, a confocal image of the sample can be formed on the imaging surface of the imaging device in a short time. Therefore, high-speed imaging is possible.

In one embodiment, the microscope may further include a drive device capable of displacing the objective lens in a direction parallel to the optical axis of the illumination light, and the processing device may control the microscope and the imaging device so that the imaging device performs imaging while the drive device displaces the objective lens.

In one embodiment, the processing apparatus may control the microscope and the imaging device so that imaging is performed by the imaging device while the relative position of the focal point of the objective lens with respect to the sample is displaced at a constant velocity. In this way, because the relative position of the focal point of the objective lens with respect to the sample is displaced at a constant velocity during imaging, it is possible to prevent the relative position of the sample with respect to the objective lens from moving in a direction perpendicular to the optical axis of the illumination light, thus preventing distortion of the captured image.

In one embodiment, the processing apparatus may detect the local tilt of the container based on the position of the container relative to the focal position of the objective lens, determine the displacement range of the relative position of the focal point of the objective lens with respect to the sample during imaging by the imaging device based on the detected local tilt of the container, and control the microscope and the imaging device to perform imaging while displacing the relative position of the focal point of the objective lens with respect to the sample by the determined displacement range. In this way, by determining the displacement range of the relative position of the focal point of the objective lens with respect to the sample without excess or deficiency based on the local tilt of the container during imaging, it is possible to perform imaging by appropriately displacing the relative position of the focal point of the objective lens with respect to the sample and obtain a clear image.

In one embodiment, the local tilt of the container may be detected, and based on this local tilt, a process of determining the displacement range of the relative position of the focal point of the objective lens relative to the sample may be performed. This process may be performed alternately to control the microscope and the imaging device so that imaging is performed by the imaging device while displacing the relative position of the focal point of the objective lens by the determined displacement range. By performing this process alternately, each of the multiple samples placed in the container may be imaged. In this way, by alternating between determining the displacement range and imaging the samples, it is possible to perform high-speed imaging with appropriate displacement corresponding to the local tilt of the container in just one scan.

In one embodiment, the processing apparatus may control the microscope and the imaging device so that imaging is performed by the imaging device while displacing the relative position of the focal point of the objective lens with respect to the sample by a displacement range specified by the user. In this way, because imaging is performed while displacing the relative position of the focal point of the objective lens with respect to the sample by a displacement range specified by the user, imaging can be performed with an appropriate displacement of the relative position of the focal point of the objective lens with respect to the sample, thereby obtaining a clear image.

An imaging method for a microscope system according to several embodiments is an imaging method for a microscope system comprising a light-emitting device, a confocal scanner, a microscope, an imaging device, and a processing device for acquiring an image of a sample to be observed, the method comprising the steps of: the light-emitting device irradiating illumination light, the scanning unit of the confocal scanner scanning the illumination light onto the sample; the objective lens of the microscope projecting the image of the illumination light scanned by the confocal scanner onto the sample; the position sensor of the microscope detecting the position of the container on which the sample is placed relative to the focal position of the objective lens; and the imaging device, whose imaging surface is positioned conjugate to the scanning unit, detecting the light to be measured from the sample irradiated with the illumination light, the processing device transmitting control signals to the microscope and the imaging device for imaging by the imaging device, while displacing the relative position of the focal point of the objective lens with respect to the sample in a direction parallel to the optical axis of the illumination light, based on the position of the container relative to the focal position of the objective lens. In this way, by performing confocal imaging while displacing the focal position of the objective lens on the sample in a direction parallel to the optical axis of the illumination light, it is possible to acquire a clear image without localized brightness reduction within the field of view, even when the sample surface is tilted. The imaging method of the microscope system does not require a complex configuration and therefore does not impose limitations on mechanical design. Since the imaging method of the microscope system acquires the image in a single scan, the imaging speed is not significantly reduced. Therefore, even when the sample surface is tilted, it is possible to acquire a clear image without localized brightness reduction within the field of view, without mechanical design limitations or significant reductions in imaging speed.

Some embodiments of the program operate the computer as the processing unit of the microscope system. In this way, confocal imaging is performed while displacing the focal position of the objective lens on the sample in a direction parallel to the optical axis of the illumination light. Therefore, even when the sample surface is tilted, confocal imaging makes it possible to acquire a clear image without partial brightness reduction within the field of view. The program does not require a complex configuration and therefore does not impose limitations on mechanical design. Since the program acquires the image in a single scan, the imaging speed is not significantly reduced. Therefore, even when the sample surface is tilted, it is possible to acquire a clear image without partial brightness reduction within the field of view, without mechanical design limitations or significant reductions in imaging speed.

According to one embodiment of this disclosure, even when the sample surface is tilted, it is possible to capture a clear image of the sample without partial brightness reduction within the field of view, without mechanical design limitations or a significant decrease in imaging speed.

This figure shows the configuration of a microscope system according to one embodiment. Figure 1 is a top view of the well plate. Figure 2A is a cross-sectional view of the well plate. This diagram schematically shows a side view of the imaged sample. This diagram schematically shows a side view of the imaged sample. This diagram schematically shows a side view of the imaged sample. This diagram schematically illustrates the control of the objective lens's focal position according to the depth of the imaged sample. This flowchart shows the operation of the imaging process performed by the microscope system. This diagram schematically shows the deflection of the bottom surface of a well plate. This flowchart shows the operation of the imaging process performed by the microscope system. This flowchart shows the operation of the imaging process performed by the microscope system. This flowchart shows the operation of the imaging process performed by the microscope system. This flowchart shows the operation of the imaging process performed by the microscope system.

<Comparative Example>
Patent Document 1 describes an optical image separation device connected to the confocal image extraction port of a Nipow disk type confocal scanner. The configuration of Patent Document 1 separates the reflected light from the sample emitted from the confocal image extraction port of the confocal scanner into light in multiple wavelength regions using a dichroic mirror. As a result, the configuration of Patent Document 1 can acquire confocal images at high speed for each arbitrarily separated wavelength region or for multiple portions of the same wavelength region.

When confocal imaging is performed using the configuration described in Patent Document 1, if the sample surface is aligned with the imaging plane of the microscope, an image with uniform brightness across the entire field of view can be obtained. However, if confocal imaging is performed with the sample surface tilted relative to the imaging plane of the microscope, the brightness of the image will be significantly reduced in a portion of the field of view due to the shallow depth of field.

To prevent a significant decrease in brightness in a portion of the field of view even when the sample surface is tilted relative to the imaging plane, it is conceivable to enlarge the pinhole diameter of the confocal scanner using the method described in Patent Document 2. This would expand the depth of focus on the sample surface and prevent a partial decrease in brightness of the sample image. However, using this method would reduce the scanning speed due to the decrease in the number of pinhole rows on the Nipow disk. Furthermore, the addition of a pinhole switching mechanism would limit the mechanical design and increase equipment costs. Additionally, enlarging the pinhole diameter would reduce the resolution in the XYZ directions. The XY directions are two mutually orthogonal directions in a plane perpendicular to the optical axis of the excitation beam 11, while the Z direction is parallel to the optical axis of the excitation beam 11.

Furthermore, as described in Patent Document 3, in addition to the means for acquiring the fluorescence signal from the sample as a confocal image, it is conceivable to provide a switchable means for acquiring the fluorescence signal from the sample as a fluorescence image, so that when the tilt of the sample surface relative to the imaging plane is significant, a fluorescence image can be acquired. Because epifluorescence observation has a deep depth of field, even if the sample surface is tilted, the sample can be captured within that deep depth of field, preventing partial brightness reduction of the sample image. However, adopting such a configuration requires a mechanism for switching the optical path, which limits the design of the mechanism and increases equipment costs. Also, misalignment of the confocal optical path and the epifluorescence optical path necessitates adjustment to correct the mismatch in the field of view. Furthermore, in the configuration of Patent Document 3, mismatch in magnification between the relay lens in the confocal optical path and the relay lens in the epifluorescence optical path necessitates image processing to correct the mismatch in imaging magnification.

Furthermore, as described in Patent Document 4, it is conceivable to configure a system that includes a confocal scanner for acquiring slice images of a sample as confocal images, and an actuator that moves the focal position of the microscope's objective lens in the optical axis direction, thereby enabling the acquisition of slice images in the depth direction of the sample. However, in the configuration of Patent Document 4, the objective lens drive waveform is generated based on the video signal output from a video-rate camera, so a separate camera capable of outputting a video signal is required. Also, the configuration of Patent Document 4 generates the objective lens drive waveform based on an analog signal, the video signal. Therefore, if the objective lens is driven by a stepping motor or the like, it becomes necessary to perform AD (Analog-to-Digital) conversion of the video signal, calculation of the objective lens drive pattern, and conversion of the objective lens drive pattern to a pulse waveform, resulting in the disadvantage of a time lag.

<Implementation>
The embodiments of this disclosure enable the acquisition of clear images without partial brightness reduction within the field of view, even when the sample surface is tilted, without mechanical design limitations or a significant decrease in imaging speed.

Hereinafter, an embodiment of this disclosure will be described with reference to the drawings. In each drawing, parts having the same configuration or function are denoted by the same reference numerals. In the description of this embodiment, redundant descriptions of identical parts may be omitted or simplified as appropriate.

Figure 1 is a schematic diagram showing the configuration of a microscope system 1 according to one embodiment of this disclosure. The microscope system 1 comprises a light-emitting device 10, a microscope 20, a confocal scanner 30, a camera 40, and a processing device 50.

In Figure 1, the light-emitting device 10 irradiates the sample 6, placed on the well plate 60, with an excitation luminous beam 11 as illumination light. The sample 6 has a fluorescent reagent attached to it, and when irradiated with the excitation luminous beam 11, the sample 6 emits a fluorescence signal 12 as the light to be measured.

The microscope 20 comprises an objective lens 21, a drive unit 22, a relay lens 23, and a position sensor 24. The objective lens 21 and the relay lens 23 constitute an infinity-correcting optical system and optically interact with the excitation light beam 11 emitted from the light-emitting device 10 and the fluorescence signal 12 emitted from the sample 6. The drive unit 22 can displace the objective lens 21 in a direction parallel to the optical axis of the excitation light beam 11 based on a control signal transmitted from the processing unit 50.

The position sensor 24 detects the Z-direction position of the well plate (container) 60 on which the sample 6 is placed, for example, by a focus detection method, using a method such as focusing on the Z-direction of a predetermined number of reference points on the well plate 60. Such reference points may include the position of the bottom surface of the well plate 60. The position sensor 24 includes a light source 241, spectral mirrors 242, 243, and a light sensor 244. When the position sensor 24 emits a focusing light 25, the focusing light 25 is reflected by the spectral mirrors 242, 243 and guided to the objective lens 21. The spectral mirror 242 may be, for example, a half mirror or a polarizing beam splitter. The spectral mirror 243 may be, for example, a dichroic mirror. The focusing light 25 is focused by the objective lens 21 and irradiated onto the bottom surface of the well plate 60. The focusing light 25 reflected from the bottom surface of the well plate 60 passes through the objective lens 21 and returns to the position sensor 24 again. The reflected focusing light 25 is reflected by the spectral mirror 243 in the position sensor 24, passes through the spectral mirror 242, and is detected by the light sensor 244. The focusing light 25 detected by the light sensor 244 is output to the processing unit 50 as a focus error signal reflecting the error from the focal position of the objective lens 21. The processing unit 50 detects the position of the reference point of the well plate 60 in the Z direction based on the focus error signal. Alternatively, the position sensor 24 may detect the Z-direction position of the objective lens 21 where its focal plane is located at the reference point of the well plate 60, and then detect the Z-direction position of the reference point based on the Z-direction position of the objective lens 21. Note that the position sensor 24 may also detect the Z-direction position of the well plate 60 using any focus detection method other than the astigmatism method, such as the knife-edge method, confocal method, and triangulation method.

The confocal scanner 30 is optically connected to the microscope 20 and the light-emitting device 10. For example, the confocal scanner 30 is attached to both the microscope 20 and the light-emitting device 10. The confocal scanner 30 comprises a pinhole array disk (hereinafter referred to as "Nipow disk") 31, a component 32, a microlens array disk (hereinafter referred to as "ML disk") 33, a dichroic mirror (hereinafter referred to as "DM") 34, a relay lens 35, a bandpass filter 36, and a relay lens 37.

The DM34 is designed to transmit the excitation light beam 11 and reflect the desired fluorescence signal 12. The ML disk 33 has multiple focusing optical elements (microlenses) arranged, for example, in a helical pattern. The Nipou disk 31 has multiple pinholes positioned at the focal points of the excitation light beam 11 by the multiple focusing optical elements of the ML disk 33. Based on a control signal transmitted from the processing unit 50, the confocal scanner 30 can rotate the Nipou disk 31 and the ML disk 33 at high speed around a rotation axis 39 using a motor or the like, while they are mechanically connected to each other by a member 32. Here, the individual microlenses and pinholes are arranged so that each pinhole formed on the Nipou disk 31 sweeps the surface of the sample 6. Therefore, the Nipou disk 31 functions as a scanning unit that scans the illumination light (excitation light beam 11) over the sample 6. The Nipou disk 31 rotates the multiple pinholes in a plane substantially perpendicular to the optical axis of the excitation light beam 11. During the imaging operation, the ML disk 33 and the Nipow disk 31 are constantly rotating.

Most of the excitation light beam 11 that passes through the ML disk 33 also passes through the Nipou disk 31. Therefore, by using the ML disk 33 and Nipou disk 31 in combination, the irradiation intensity of the excitation light beam 11 onto the sample 6 is increased compared to using the Nipou disk 31 alone. Furthermore, reflection of the excitation light beam 11 from areas other than the pinhole in the Nipou disk 31 is suppressed. Consequently, the signal-to-noise ratio (SNR) of the sample 6 image is increased.

The excitation light beam 11 is focused into individual beams by the ML disk 33, passes through the DM 34, and then through the individual pinholes of the Nipow disk 31. After passing through the relay lens 23 of the microscope 20, the excitation light beam 11 is focused by the objective lens 21 onto the sample 6 placed in the wells (holes) of the well plate 60.

As described above, a fluorescent reagent is added to each of the samples 6 in the well plate 60. The fluorescent signals 12 emitted by each of the fluorescent reagents in the samples 6 pass again through the objective lens 21 and relay lens 23 and are focused onto the individual pinholes of the Nipow disk 31.

The fluorescence signal 12 that passes through the pinhole of the Nipow disk 31 is reflected by the DM 34. The confocal scanner 30 has relay lenses 35 and 37 of an infinity-correcting optical system that images the fluorescence signal 12 reflected by the DM 34 onto the two-dimensional sensor (image sensor) 41 of the camera 40, which acts as an imaging device. The confocal scanner 30 also has a bandpass filter 36 between the relay lenses 35 and 37. The bandpass filter 36 may be implemented as a barrier filter (absorption filter) that selectively transmits the fluorescence contained in the fluorescence signal 12.

The plane on which the pinholes of the Nipow disk 31 are aligned, the surface of the sample 6, and the light-receiving surface of the camera 40's two-dimensional sensor 41 are arranged in an optically conjugate relationship. Therefore, an optical cross-sectional image of the sample 6, i.e., a confocal image, is formed on the camera 40's two-dimensional sensor 41. Furthermore, as described above, the objective lens 21 drive device 22 is driven based on a control signal from the processing device 50 to displace the objective lens 21 in a direction parallel to the optical axis of the excitation light beam 11. Since the objective lens 21 and the relay lens 23 constitute an infinity correction optical system, the displacement of the objective lens 21 can displace the focal plane of the objective lens 21 on the sample 6 while maintaining an optically conjugate relationship with the Nipow disk 31 and the light-receiving surface of the two-dimensional sensor 41. The focal plane of the objective lens 21 constitutes the imaging plane of the microscope 20. Based on the exposure signal from the processing device 50, the camera 40 exposes the image projected onto the light-receiving surface of its two-dimensional sensor 41 for a specified time and converts it into digital image data. In other words, the camera 40 performs exposure while receiving an ON exposure signal from the processing unit 50, forming an image on the two-dimensional sensor 41. The digital image data is transferred to the processing unit 50 and stored in the processing unit 52.

The processing unit 50 is connected to each component of the microscope system 1 and controls the operation of the entire microscope system 1 by transmitting control signals to each component. However, the processing unit 50 also transmits an exposure signal to the camera 40 to control exposure during imaging. The processing unit 50 also provides a user interface for the user to operate the microscope system 1 and perform observations. The processing unit 50 is, for example, a computer device and includes any device such as a PC (Personal Computer), tablet PC, smartphone and feature phone, and a personal digital assistant (PDA).

The processing unit 50 comprises a control unit 51, a storage unit 52, and an input/output unit 53. The control unit 51 includes one or more processors. In one embodiment, the "processor" is a general-purpose processor or a dedicated processor specialized for a specific process, but is not limited to these. The control unit 51 is communicatively connected to each component constituting the processing unit 50 and controls the operation of the entire processing unit 50.

The storage unit 52 includes any storage module, including an HDD (Hard Disk Drive), SSD (Solid State Drive), ROM (Read-Only Memory), and RAM (Random Access Memory). The storage unit 52 may function, for example, as main memory, auxiliary memory, and cache memory. The storage unit 52 stores any information used in the operation of the processing unit 50 or obtained as a result of the operation of the processing unit 50. For example, the storage unit 52 stores programs such as system programs and application programs for operating the processing unit 50, as well as image data of images captured by the camera 40.

The functions of the processing unit 50 can be realized by executing a program (computer program) that can be used to operate the microscope system 1 according to this embodiment, using the processor included in the control unit 51. In other words, the functions of the processing unit 50 can be realized by software. The program causes the computer to execute the processing steps included in the operation of the processing unit 50, thereby realizing the functions corresponding to the processing of each step. That is, the program is a program that causes the computer to function as the processing unit 50 according to this embodiment.

The program can be recorded on a computer-readable recording medium. Examples of computer-readable recording media include magnetic recording devices, optical discs, magneto-optical recording media, or semiconductor memory. The program can be distributed, for example, by selling, transferring, or leasing portable recording media such as DVDs (Digital Versatile Discs) or CD-ROMs (Compact Disc ROMs) on which the program is recorded. The program may also be distributed by storing it in server storage and transferring it from the server to other computers via a network. The program may also be provided as a program product.

A computer, for example, stores a program recorded on a portable storage medium or a program transferred from a server in its main memory. Then, the computer reads the program stored in main memory using its processor and executes the processing according to the read program. Alternatively, the computer may directly read the program from the portable storage medium and execute the processing according to the program. The computer may also execute the processing according to the received program sequentially each time a program is transferred from a server to the computer. Such processing may also be performed by a so-called ASP (Application Service Provider) type service, which realizes its function only through execution instructions and result retrieval, without transferring the program from the server to the computer. A program includes information used for processing by an electronic computer that is equivalent to a program. For example, data that is not a direct instruction to the computer but has the nature of defining the computer's processing falls under the category of "information equivalent to a program."

Some or all of the functions of the processing unit 50 may be implemented by a dedicated circuit included in the control unit 51. That is, some or all of the functions of the processing unit 50 may be implemented by hardware. Furthermore, the processing unit 50 may be implemented by a single information processing unit, or by the cooperation of multiple information processing units.

The input/output unit 53 includes an input unit that receives user input and inputs information based on that input, and an output unit that outputs the calculation results of the processing unit 50 and the captured images taken by the camera 40. The input unit may be, for example, a physical key, a capacitive key, a pointing device, a touchscreen integrated with the output unit's display, or a microphone that accepts voice input. The output unit may be, for example, a display that outputs information as an image, or a speaker that outputs information as sound.

The microscope system 1, having the configuration described above, performs imaging while displacing the focal plane of the objective lens 21 in a direction parallel to the optical axis of the excitation light beam 11 relative to the sample 6. This makes it possible to capture a focused image across the entire field of view, even when the sample 6 is tilted.

Figure 2A is a top view of the well plate 60, which serves as a container on which the sample 6 is placed during microscopic cell imaging. As shown in Figure 2A, the well plate 60 is provided with numerous wells 68 for placing the sample 6.

Figure 2B is a cross-sectional view of the well plate 60 shown in Figure 2A, obtained by cutting along surface AA. As shown in Figure 2B, the well plate 60 comprises a frame 61 and a bottom surface 62.

The frame 61 is made of resin, and the bottom surface 62, which serves as the sample surface, is generally made of a thin sheet of resin or glass. Therefore, due to distortion (approximately several hundred μm) during the molding of the frame 61, the frame 61 may not be positioned horizontally on the sample stage of the microscope 20, resulting in the sample surface being tilted relative to the focal plane of the microscope 20. Similarly, distortion (approximately several tens of μm) during the molding of the bottom surface 62 may cause the bottom surface 62 to bend, also resulting in the sample surface being tilted relative to the focal plane of the microscope 20.

Figures 3 to 5 schematically show a side view of the sample 63 to be imaged. In Figure 3, the cell culture surface 64 is positioned horizontally, and the culture surface 64 of the sample 63 to be imaged aligns with the focal plane 65 of the objective lens 21 of the microscope 20. Therefore, when imaging is performed in this state, the camera 40 acquires an image that is in focus across the entire field of view.

In Figure 4, the cell culture surface 64 is tilted relative to the focal plane 65 of the objective lens 21. Therefore, if imaging is performed in this state, an image with significantly reduced brightness in a portion of the field of view will be obtained. In Figure 5, numerous samples 63 have gathered to form a thick mass 66. Therefore, if imaging is performed using a confocal microscope with the focal plane 65 of the microscope 20 passing through the center of the mass 66, only a portion of the thick mass 66 will be reflected in the image.

Figure 6 is a schematic diagram showing the control of the focal position of the objective lens 21 according to the depth of the sample to be imaged 63, as performed by the microscope system 1 according to this embodiment. In Figure 6, the culture surface 64 of the sample to be imaged 63 is inclined with respect to the focal plane. Therefore, the processing device 50 controls the camera 40 to perform imaging while displacing the relative position of the focal point of the objective lens 21 with respect to the sample 6 in a direction parallel to the optical axis of the excitation light beam 11. Specifically, in this embodiment, the processing device 50 controls the camera 40 to perform imaging while the drive device 22 displaces the objective lens 21 without moving the sample 6. The displacement width 71 of the objective lens 21 may be the width of the range in which the sample 6 is distributed in the well plate 60 in a direction parallel to the optical axis of the excitation light beam 11. As shown on the right of Figure 6, the processing device 50 may transmit an exposure signal to the camera 40 and perform exposure while displacing the relative position of the focal point of the objective lens 21 with respect to the sample 6 at a constant velocity as time t progresses. This allows for the acquisition of an image covering the depth range in which sample 6 is distributed, making it possible to obtain a clear image.

(Example 1)
Next, with reference to Figures 7 and 8, an embodiment 1 of the operation of the microscope system 1 will be described. Figure 7 is a flowchart showing the operation of the imaging process performed by the microscope system 1. Figure 8 is a schematic diagram showing the deflection of the bottom surface 62 of the well plate 60. The operation of the microscope system 1 described with reference to Figures 7 and 8 corresponds to one of the imaging methods according to embodiment 1. The operation of each step in Figure 7 is performed based on the control of the control unit 51 of the processing device 50. The program for causing a computer to execute the imaging method according to this embodiment includes each step shown in Figure 7. As a prerequisite for the following process, the well plate 60 on which the sample 6 is placed is set to a measurement position for measurement.

In step S001, the control unit 51 receives from the user the setting of measurement conditions related to imaging by the microscope system 1. Examples of such measurement conditions include the following:
- Displacement range Δz _a of the relative position of the focal point of the objective lens 21 with respect to the sample 6 (range of movement of the focal point of the objective lens 21)
・Exposure time T _exp
- A list of XY positions on the well plate 60 on which imaging is performed. Here, the displacement range Δz _a is, for example, 10 μm to several hundred μm, and is a value that indicates the range in the Z direction (parallel to the optical axis of the excitation light beam 11) in which the sample 6 exists, taking into account the thickness of the sample 6 and the inclination and deflection of the well plate 60. The exposure time T _exp may be a value of about 50 ms to 1 s. The control unit 51 may also accept input of dimensional information of the well plate 60.

The XY position is the position on the well plate 60. As shown in Figure 8, the XY position can be expressed as (x _i , y _j ), for example, where i is an integer between 1 and m and j is an integer between 1 and n. Each of x _i and y _j may correspond to A, B, C, ... and 1, 2, 3, ... which identify the well 68, as shown in Figure 2A. If there is no tilting or distortion in the well plate 60, the position in the Z direction of the bottom surface 62 of the well plate 60 is the same for each of the XY positions (x _i , y _j ). However, if tilting or distortion occurs, the position in the Z direction of the bottom surface 62 of the well plate 60 will no longer be the same for each of the XY positions (x _i , y _j ). As shown in Figure 8, the positions in the Z direction of the bottom surface 62 of the well plate 60 at the XY positions (x _i , y _j ) are represented as z _i,j .

In step S002, the control unit 51 controls each component of the microscope system 1 to align each component and moves the well plate 60 in the XY direction so that the sample 6 at the XY position where the first image is to be taken is within the imaging range of the excitation light beam 11. Subsequently, the control unit 51 performs the processes in steps S003 to S009 for each of the XY positions included in the list that was set in step S001. Hereinafter, the XY position to be processed will be represented as (x _i , y _j ).

In step S003, the control unit 51 controls the drive unit 22 to move the objective lens 21 in the Z direction, reads the signal from the position sensor 24, and searches for the position z0 _(i,j) of the objective lens 21 where the focal point of the objective lens 21 coincides with the bottom surface 62 of the well plate 60.

In step S004, the control unit 51 controls the drive unit 22 to move the objective lens 21 to the position z0 _(i,j) found in step S003.

In step S005, the control unit 51 controls the confocal scanner 30 and the light-emitting device 10 to rotate the ML disk 33 and the Nipow disk 31 while emitting an excitation light beam 11 from the light-emitting device 10. As a result, the excitation light beam 11 is focused into individual beams by the ML disk 33, passes through the DM 34, passes through individual pinholes in the Nipow disk 31, and is focused by the objective lens 21 of the microscope 20 onto the sample 6 placed in the well 68 of the well plate 60. The fluorescence signal 12 emitted by the fluorescent reagent in the sample 6 due to the excitation light beam 11 passes through the objective lens 21 again and is focused onto individual pinholes in the Nipow disk 31. The fluorescence signal 12 that has passed through the individual pinholes is reflected by the DM 34, passes through relay lenses 35 and 37, and is emitted from the aperture (confocal image extraction port) of the confocal scanner 30 via the bandpass filter 36 to be imaged by the camera 40. Meanwhile, the control unit 51 moves the objective lens 21 from z _0(i,j) to z _0(i,j) + Δz _a at a constant velocity, for example, while exposing the camera 40 for an exposure time T _exp and acquiring an image. Specifically, the control unit 51 transmits a control signal to the drive unit 22 to displace the objective lens 21, while simultaneously transmitting an exposure signal to the camera 40 to expose the 2D sensor 41 and perform imaging.

Once the objective lens 21 has been driven and the camera 40 has finished exposing the subject, in step S006, the control unit 51 terminates the emission of the excitation light beam 11 from the light-emitting device 10.

In step S007, the control unit 51 transfers the image data acquired by the camera 40 to the storage unit 52 of the processing unit 50.

In step S008, the control unit 51 determines whether imaging has been completed for all XY positions specified by the user in step S001. If imaging has not been completed for all XY positions, i.e., if there are XY positions specified by the user that have not yet been imaged (NO in step S008), the control unit 51 proceeds to step S009. If imaging has been completed for all XY positions (YES in step S008), the control unit 51 terminates the series of operations.

In step S009, the control unit 51 moves the well plate 60 in the XY direction so that the sample 6 at the XY position for the next imaging is within the imaging range of the excitation light beam 11, and then returns to step S003.

As described above, in Embodiment 1, the control unit 51 detects the position z0 _(i,j) of the objective lens 21 corresponding to the bottom surface 62 of the well plate 60 at the XY position (x _i , y _j ) where imaging is performed. Then, the control unit 51 moves the objective lens 21 in the z direction by a displacement width Δz _a set by the user from that position z0 _(i,j) and images the sample 6. In this way, in Embodiment 1, confocal imaging is performed while displacing the focal position of the objective lens 21 on the sample 6 in a direction parallel to the optical axis of the excitation light beam 11. For this reason, even if the sample surface is tilted, confocal imaging makes it possible to capture a clear image without partial brightness reduction in the field of view. Furthermore, since the microscope system 1 does not require a complex configuration, there are no limitations in mechanical design. Since the microscope system 1 acquires the image in a single imaging pass, the imaging speed is not significantly reduced. Therefore, even when the sample surface is tilted, it is possible to acquire a clear image without partial brightness reduction within the field of view, without limitations in the mechanical design or a significant decrease in imaging speed.

(Example 2)
Next, with reference to Figures 8, 9A, and 9B, an embodiment 2 of the operation of the microscope system 1 will be described. In embodiment 1, for each XY position on the well plate 60, the position of the bottom surface 62 of the well plate 60 was detected, and an example was described in which imaging was performed while displacing the bottom surface 62 by a displacement width Δz _a set by the user. In this embodiment, an example will be described in which the displacement width Δz _c(i,j) of the range in which the sample 6 exists, corresponding to the local inclination of the well plate 60 at each XY position, is detected, and imaging is performed in an imaging range that reflects this. Figures 9A and 9B are flowcharts showing the operation of the imaging process performed by the microscope system 1. The operation of the microscope system 1 described with reference to Figures 9A and 9B corresponds to one of the imaging methods according to embodiment 2. The operation of each step in Figures 9A and 9B is performed based on the control of the control unit 51 of the processing device 50. The program for causing a computer to execute the imaging method according to this embodiment includes the steps shown in Figures 9A and 9B. As a prerequisite for the following processing, the well plate 60 on which the sample 6 is placed is set to a measurement position for measurement.

In step S101, the control unit 51 receives from the user the setting of measurement conditions related to imaging by the microscope system 1. Examples of such measurement conditions include the following:
- Displacement range Δz _b of the relative position of the focal point of the objective lens 21 with respect to the sample 6 (range of movement of the focal position of the objective lens 21)
・Exposure time T _exp
- A list of XY positions on the well plate 60 on which imaging is performed. Here, the displacement width Δz _b is, for example, 10 μm to several hundred μm, and is a value indicating the width in the Z direction that is entered by the user based on the thickness of the sample 6. The exposure time T _exp may be a value of about 50 ms to 1 s. The control unit 51 may also accept input of dimensional information of the well plate 60.

In step S102, the control unit 51 sets the XY positions ( _x1 , _y1 ) to ( _xm , _yn ) for pre-scanning based on a list of XY positions of the well plate 60 on which imaging is to be performed.

In step S103, the control unit 51 controls each component of the microscope system 1 to align each component and moves the well plate 60 in the XY direction so that the sample 6 at the XY position where the first pre-scan is performed is located within the imaging range of the excitation light beam 11. Pre-scanning is the operation of searching for the position z0 _(i,j) in the Z direction at each XY position (x _i , y _j ) where the focal point of the objective lens 21 coincides with the bottom surface 62 of the well plate 60. Hereafter, the control unit 51 executes the processes in steps S104 to S106 for each of the XY positions set in step S102. Hereafter, the XY position to be processed will be represented as (x _i , y _j ).

In step S104, the control unit 51 controls the drive unit 22 to move the objective lens 21 in the Z direction, while reading the signal from the position sensor 24 to find the Z-direction position z0 _(i,j) of the objective lens 21 where the focal point of the objective lens 21 coincides with the bottom surface 62 of the well plate 60. Here, position _z0(i,j) is the Z-direction position of the objective lens 21 with respect to the XY position (x _i , y _j ).

In step S105, the control unit 51 determines whether the search for _z0(i,j) has been completed at all XY positions (x _i , y _j ) where the pre-scan is to be performed. If the pre-scan is not completed for all XY positions where the pre-scan is to be performed, that is, if there are XY positions specified by the user where the pre-scan has not been completed (NO in step S105), the control unit 51 proceeds to step S106. If the pre-scan is completed at all XY positions where the pre-scan is to be performed (YES in step S105), the control unit 51 proceeds to step 107.

In step S106, the control unit 51 moves the well plate 60 in the XY direction so that the sample 6 at the XY position for the next pre-scan is within the imaging range of the excitation light beam 11, and then returns to step S104.

In step S107, the control unit 51 calculates the inclination grad(zi, _j ) of the bottom _{surface 62} of the well plate 60 at each XY position ( _xi , y _j ) where imaging is performed, based on the prescan results ( _x1 , _y1 , z0(1,1)) to ( _xm , yn, _z0(m,n) ). The inclination grad(zi _{, j} ) is a vector quantity having grad(zi _,j ) _x which indicates the inclination in the X direction and grad(zi _,j ) _y which indicates the inclination in the Y direction. Grad(zi _,j ) _x and grad(zi _,j ) _y may be calculated, for example, by the following equation 1.
[Formula 1]
grad(z _i,j ) _x = (z _i+1,j −z _i-1,j )/(x _i+1 − _{x i-1} )
grad(z _i,j ) _y = (z _i,j+1 −z _i,j-1 )/(y _j+1 −y _j-1 )

In step S108, the control unit 51 calculates the imaging Z range Δz c( _{i,j) at each XY position where imaging is performed, based on the inclination grad(z i,j} ) of the bottom surface 62 of the well plate 60 at the XY position where imaging is performed and the size of the two-dimensional sensor 41 of the camera 40. Here, Δz _{c(i,j )} is the range in the Z direction where the sample 6 exists due to the inclination of the well plate 60. For example, suppose the two-dimensional sensor 41 has a rectangle, and the length corresponding to the imaging range in the X direction is _L1 , and the length corresponding to the imaging range in the Y direction is _L2 . In this case, the displacement in the Z direction due to the inclination in the X direction is _L1 × grad(z _i,j ) _x . The displacement in the Z direction due to the inclination in the Y direction is _L2 × grad(z _i,j ) _y . Therefore, the control unit 51 may calculate the imaging Z range Δz _c(i,j) using, for example, the following equation 2.
[Formula 2]
Δz _c(i,j) =L ₁ ×grad(z _i,j ) _x +L ₂ ×grad(z _i,j ) _y

In step S109, the control unit 51 controls each component of the microscope system 1 to align each component and moves the well plate 60 in the XY direction so that the sample 6 at the XY position where the first image is to be taken is within the imaging range of the excitation light beam 11. Subsequently, the control unit 51 performs the processing in steps S110 to S116 for each of the XY positions included in the list that was set in step S001. Hereinafter, the XY position to be processed will be represented as (x _i , y _j ).

In step S110, the control unit 51 controls the drive unit 22 to move the objective lens 21 in the Z direction, reads the signal from the position sensor 24, and searches for the position z0 _(i,j) where the focal point of the objective lens 21 coincides with the bottom surface 62 of the well plate 60.

In step S111, the control unit 51 controls the drive device 22 to move the objective lens 21 to the position z0 _(i,j) found in step S110.

In step S112, the control unit 51 controls the confocal scanner 30 and the light-emitting device 10 to rotate the ML disk 33 and the Nipow disk 31 while emitting an excitation light beam 11 from the light-emitting device 10. As a result, the excitation light beam 11 is focused into individual beams by the ML disk 33, passes through the DM 34, passes through individual pinholes in the Nipow disk 31, and is focused by the objective lens 21 of the microscope 20 onto the sample 6 placed in the well 68 of the well plate 60. The fluorescence signal 12 emitted by the fluorescent reagent in the sample 6 due to the excitation light beam 11 passes through the objective lens 21 again and is focused onto individual pinholes in the Nipow disk 31. The fluorescence signal 12 that has passed through the individual pinholes is reflected by the DM 34, passes through relay lenses 35 and 37, and is emitted from the aperture (confocal image extraction port) of the confocal scanner 30 so that it is imaged to the camera 40 via the bandpass filter 36. Meanwhile, the control unit 51 moves the objective lens 21 from z _0(i,j) to z _0(i,j) + Δz _b + Δz _c(i,j) at a constant velocity, for example, while exposing the camera 40 for an exposure time T _exp and acquiring an image. Specifically, the control unit 51 transmits a control signal to the drive unit 22 to displace the objective lens 21, while simultaneously transmitting an exposure signal to the camera 40 to expose the 2D sensor 41 and perform imaging.

Once the objective lens 21 has been driven and the camera 40 has finished exposing the subject, in step S113, the control unit 51 terminates the emission of the excitation light beam 11 from the light-emitting device 10.

In step S114, the control unit 51 transfers the image data acquired by the camera 40 to the storage unit 52 of the processing unit 50.

In step S115, the control unit 51 determines whether imaging has been completed for all XY positions specified by the user in step S101. If imaging has not been completed for all XY positions, i.e., if there are XY positions specified by the user that have not yet been imaged (NO in step S115), the control unit 51 proceeds to step S116. If imaging has been completed for all XY positions (YES in step S115), the control unit 51 terminates the series of operations.

In step S116, the control unit 51 moves the well plate 60 in the XY direction so that the sample 6 at the XY position for the next imaging is within the imaging range of the excitation light beam 11, and then returns to step S113.

As described above, the microscope system 1 according to Example 2 performs a pre-scan in addition to the processing in Example 1 to detect the local tilt of the well plate 60 at each XY position. The microscope system 1 then detects the displacement width Δz _c(i,j) of the range where the sample 6 exists, corresponding to the local tilt of the well plate 60, and performs imaging in the imaging range that reflects this. Therefore, according to this embodiment, it is possible to predict the tilt of the well plate 60 at the XY positions where imaging is performed and the range in the Z direction where the sample 6 exists, making it possible to set an appropriate imaging Z range and perform imaging.

(Example 3)
Next, with reference to Figures 8, 10A, and 10B, an embodiment 3 of the operation of the microscope system 1 will be described. In embodiment 2, a pre-scan was performed to detect the displacement width Δz _c(i,j) corresponding to the local tilt of the well plate 60 for each XY position on the well plate 60, and then a scan was performed again for each XY position to image the sample 6 at each XY position. In this embodiment, an example will be described in which the detection of the displacement width Δz _c(i,j) corresponding to the local tilt of the well plate 60 at each XY position and the imaging of the sample 6 are performed in a single scan. Figures 10A and 10B are flowcharts showing the operation of the imaging process performed by the microscope system 1. The operation of the microscope system 1 described with reference to Figures 10A and 10B corresponds to one of the imaging methods according to embodiment 3. The operation of each step in Figures 10A and 10B is performed based on the control of the control unit 51 of the processing device 50. The program for causing a computer to execute the imaging method according to this embodiment includes the steps shown in Figures 10A and 10B. As a prerequisite for the following process, the well plate 60 on which the sample 6 is placed is set to the measurement position for measurement.

In step S201, the control unit 51 receives from the user the setting of measurement conditions related to imaging by the microscope system 1. Examples of such measurement conditions include the following:
- Displacement range Δz _b and Δz _d of the relative position of the focal point of the objective lens 21 with respect to the sample 6 (range of movement of the focal point of the objective lens 21)
・Exposure time T _exp
- A list of XY positions on the well plate 60 on which imaging is performed ( _x1 , _y1 ) to ( _xm , _yn )
Here, the displacement width Δz _b is, for example, 10 μm to several hundred μm, and is a value indicating the width in the Z direction that the user inputs based on the thickness of the sample 6. The displacement width Δz _d is, for example, several μm to several tens of μm, and is a value that the user inputs by predicting the range in the Z direction where the sample 6 exists due to the inclination of the well plate 60 at the XY position where imaging is performed. The exposure time T _exp may be a value of about 50 ms to 1 s. The control unit 51 may also accept input of dimensional information of the well plate 60.

In step S202, the control unit 51 moves the well plate 60 in the XY direction so that the sample 6 at the first XY position to be imaged is within the imaging range of the excitation light beam 11. Hereafter, the control unit 51 executes the processes in steps S203 to S213 for each of the XY positions set in step S201. Hereafter, the XY positions to be processed are represented as (x _i , y _j ). Hereafter, an example will be described in which the control unit 51 executes the processes in the order of (x ₁ , y ₁ ), ( _{x 2 ,} y ₁ ), _... , (x _m , y ₁ ), ( _{x 1} , y ₂ ), ( _{x 2 ,} y ₂ ), ..., (x m, y ₂ ), ..., (x ₁ , yn), (x ₂ , yn), ..., (x m, _yn ), but the processes may be executed in any other order.

In step S203, the control unit 51 controls the drive unit 22 to move the objective lens 21 in the Z direction, while reading the signal from the position sensor 24 to find the Z-direction position z0 _(i,j) of the objective lens 21 where the focal point of the objective lens 21 coincides with the bottom surface 62 of the well plate 60. Here, position _z0(i,j) is the Z-direction position of the objective lens 21 with respect to the XY position (x _i , y _j ).

In step S204, the control unit 51 determines whether the positions z0( _i -1, _j ) and z0(i, _{j -1} ) of the objective lens 21 at nearby XY positions ( _{xi -1} ,yj) and ( _{xi,yj-1) are known, with respect to the current XY position (xi,yj) .} For example, when processing is performed in the order of ( _x1 , _y1 ), ( _x2 , _y1 ), ..., ( _xm , _y1 ), ( _x1 , _y2 ), ( _x2 , _y2 ), ..., ( _xm , _y2 ), ..., ( _x1 , _yn ), ( _x2 , _yn ), ..., ( _xm , _yn ), if i > 1 and j > 1, the processing at (x _i-1 , y _j ) and (x _i , y _j-1 ) is completed, so _{z0(i-1, j)} and z0 _{(i, j-1)} are known. If they are known (YES in step S204), the control unit 51 proceeds to step S206. If they are not known (NO in step S204), the control unit 51 proceeds to step S205.

In step S205, the control unit 51 determines Δz c(i,j), which is the inclination of the well plate 60 at the XY position (x _i , y _j ) where imaging is performed and the resulting range in the Z direction where the sample 6 exists, to be Δz _{d ,} which was specified by the user in step S201. Then, the control unit 51 proceeds to step S208.

In step S206, the control unit 51 calculates the inclination grad(zi _,j ) of the bottom surface 62 of the well plate 60 at the current XY position (x _i , y _j ). The control unit 51 may calculate grad(zi _,j ) _x and grad(zi _,j ) _y using, for example, the following equation 3.
[Formula 3]
grad(z _i,j ) _x = (z _i,j −z _i-1,j )/(x _i −x _i-1 )
grad(z _i,j ) _y = (z _i,j −z _i,j-1 )/(y _j −y _j-1 )

In step S207, the control unit 51 calculates the imaging Z range Δz _c(i,j) at the XY position (x _i , y _j ) based on grad(z _i,j ) and the size of the two-dimensional sensor 41 of the camera 40. For example, if the two-dimensional sensor 41 is a rectangle having a length _L1 corresponding to the imaging range in the X direction and a length _L2 corresponding to the imaging range in the Y direction, the control unit 51 may calculate the imaging Z range Δz _c(i,j) using the following equation 4, similar to S108 in Figure 9A.
[Formula 4]
L ₁ ×grad(z _i,j ) _x +L ₂ ×grad(z _i,j ) _y
Then, the control unit 51 proceeds to step S208.

In step S208, the control unit 51 controls the drive unit 22 to move the objective lens 21 to the position z0 _(i,j) found in step S203.

In step S209, the control unit 51 controls the confocal scanner 30 and the light-emitting device 10 to rotate the ML disk 33 and the Nipow disk 31 while emitting an excitation light beam 11 from the light-emitting device 10. As a result, the excitation light beam 11 is focused into individual beams by the ML disk 33, passes through the DM 34, passes through individual pinholes in the Nipow disk 31, and is focused by the objective lens 21 of the microscope 20 onto the sample 6 placed in the well 68 of the well plate 60. The fluorescence signal 12 emitted by the fluorescent reagent in the sample 6 due to the excitation light beam 11 passes through the objective lens 21 again and is focused onto individual pinholes in the Nipow disk 31. The fluorescence signal 12 that has passed through the individual pinholes is reflected by the DM 34, passes through relay lenses 35 and 37, and is emitted from the aperture (confocal image extraction port) of the confocal scanner 30 via the bandpass filter 36 to be imaged by the camera 40. Meanwhile, the control unit 51 moves the objective lens 21 from z _0(i,j) to z _0(i,j) + Δz _b + Δz _c(i,j) at a constant velocity, for example, while exposing the camera 40 for an exposure time T _exp and acquiring an image. Specifically, the control unit 51 transmits a control signal to the drive unit 22 to displace the objective lens 21, while simultaneously transmitting an exposure signal to the camera 40 to expose the 2D sensor 41 and perform imaging.

Once the objective lens 21 has been driven and the camera 40 has finished exposing the subject, in step S210, the control unit 51 terminates the emission of the excitation light beam 11 from the light-emitting device 10.

In step S211, the control unit 51 transfers the image data acquired by the camera 40 to the storage unit 52 of the processing unit 50.

In step S212, the control unit 51 determines whether imaging has been completed for all XY positions specified by the user in step S201. If imaging has not been completed for all XY positions, i.e., if there are XY positions specified by the user for which imaging has not been completed (NO in step S212), the control unit 51 proceeds to step S213. If imaging has been completed for all XY positions (YES in step S212), the control unit 51 terminates the series of operations.

In step S213, the control unit 51 moves the well plate 60 so that the sample 6 at the XY position for the next imaging is positioned within the imaging range in the XY direction by the excitation light beam 11, and then returns to step S203.

As described above, the microscope system 1 according to Example 3 calculates the displacement width Δz c(i,j) in the Z direction where the sample 6 exists, which reflects the local inclination of the well plate 60 at the XY position where imaging is performed, while performing the imaging operation without performing a pre-scan. Therefore, according to Example 3, it is possible to set an imaging Z range that is neither too large _nor too small, and to perform imaging in that imaging Z range at high speed.

Furthermore, the processing unit 50 may control the microscope 20 and camera 40 so that imaging is performed by the camera 40 while displacing the relative position of the focal point of the objective lens 21 with respect to the sample 6 at a constant velocity specified by the user. In this way, since the relative position of the focal point of the objective lens 21 with respect to the sample 6 is displaced at a constant velocity during imaging, it is possible to prevent the relative position of the sample 6 with respect to the objective lens 21 from moving in a direction perpendicular to the optical axis of the excitation light beam 11 (XY direction) during imaging, thereby preventing distortion of the captured image.

Furthermore, the processing device 50 may determine the speed at which it displaces the relative position of the focal point of the objective lens 21 with respect to the sample 6 based on the exposure time T _exp and the relative position displacement width Δz _a specified by the user.

As described above, in this embodiment, the microscope system 1 for acquiring an image of the sample 6 to be observed comprises a light-emitting device 10, a microscope 20, a confocal scanner 30, a camera 40, and a processing device 50. The light-emitting device 10 irradiates with an excitation light beam 11. The confocal scanner 30 has a scanning unit that scans the excitation light beam 11 with respect to the sample 6. The microscope 20 has an objective lens 21 that projects the image of the excitation light beam 11 scanned by the confocal scanner 30 onto the sample 6, and a position sensor 24 that detects the position of the well plate 60 on which the sample 6 is placed relative to the focal position of the objective lens 21. The camera 40 has an imaging surface positioned conjugate to the scanning unit and detects a fluorescence signal 12 from the sample 6 irradiated with the excitation light beam 11. The processing device 50 controls the operation of the microscope 20 and the camera 40. Here, the processing unit 50 transmits a control signal to the microscope 20 and camera 40 for imaging by the camera 40, while displacing the relative position of the focal point of the objective lens 21 with respect to the sample 6 in a direction parallel to the optical axis of the excitation light beam 11, based on the position of the well plate 60 with respect to the focal position of the objective lens 21. In this way, the microscope system 1 according to this embodiment performs confocal imaging while displacing the focal position of the objective lens 21 in the sample 6 in a direction parallel to the optical axis of the excitation light beam 11. Therefore, even when the sample surface is tilted, the microscope system 1 can capture a clear image without partial brightness reduction in the field of view by confocal imaging. Furthermore, since the microscope system 1 does not require a complex configuration, it is not subject to limitations in mechanical design. Moreover, since the microscope system 1 acquires the image in a single imaging, it does not require a large load in terms of calculation and storage, nor does it significantly reduce the imaging speed. Therefore, even when the sample surface is tilted, it is possible to acquire clear images without partial brightness reduction within the field of view, without limitations in the mechanical design or a significant decrease in imaging speed.

Furthermore, the confocal scanner 30, acting as a scanning unit, rotates a pinhole disk on which multiple pinholes are arranged, scanning the excitation light beam 11 irradiated from the light-emitting device 10 onto the sample 6. The objective lens 21 projects images of the multiple pinholes, created by the excitation light beam 11 scanned by the confocal scanner 30, onto the sample 6. The camera 40's imaging surface is positioned conjugate to the multiple pinholes. In this way, the microscope system 1 can quickly form a confocal image of the sample 6 on the light-receiving surface of the camera 40's two-dimensional sensor 41 by rapidly rotating the ML disk 33 and the Nipow disk 31. Therefore, it is possible to rapidly acquire confocal images of all samples in a well plate 60, on which numerous samples to be examined are arranged in a matrix, while relatively displacing it in a direction perpendicular to the optical axis of the excitation light beam 11 relative to the microscope 20 and the confocal scanner 30.

Furthermore, the microscope 20 is equipped with a drive device 22 capable of displacing the objective lens 21 in a direction parallel to the optical axis of the excitation light beam 11. The processing device 50 controls the microscope 20 and the camera 40 so that the camera 40 performs imaging while the drive device 22 displaces the objective lens 21.

Furthermore, the processing unit 50 may control the microscope 20 and camera 40 so that imaging is performed by the camera 40 while displacing the relative position of the focal point of the objective lens 21 with respect to the sample 6 at a constant velocity. In this way, since the relative position of the focal point of the objective lens 21 with respect to the sample 6 is displaced at a constant velocity during imaging, it is possible to prevent the relative position of the sample 6 with respect to the objective lens 21 from moving in a direction perpendicular to the optical axis of the excitation light beam 11 during imaging, thereby preventing distortion of the captured image.

Furthermore, the processing unit 50 determines the displacement range of the relative position of the focal point of the objective lens 21 relative to the sample 6 during imaging by the camera 40, based on the position of the container relative to the focal point of the objective lens 21. The processing unit 50 controls the microscope 20 and camera 40 to perform imaging by the camera 40 while displacing the relative position of the focal point of the objective lens 21 relative to the sample 6 by the determined displacement range. In this way, the processing unit 50 determines the displacement range of the relative position of the focal point of the objective lens 21 relative to the sample 6 during imaging, based on the position of the container relative to the objective lens 21 of the microscope 20. Therefore, by appropriately displacing the relative position of the focal point of the objective lens 21 relative to the sample 6, imaging can be performed, and a clear image can be obtained.

Here, as in Examples 2 and 3, the processing device 50 may detect the local tilt of the container, for example, the bottom surface 62, based on the position of the container relative to the focal position of the objective lens 21. Furthermore, the processing device 50 may determine the displacement range of the relative position of the focal point of the objective lens 21 with respect to the sample 6 during imaging by the camera 40, based on the detected local tilt of the container. By utilizing the local tilt of the container in this way, it is possible to determine the displacement range precisely and accurately, thereby displacing the relative position of the focal point of the objective lens 21 with respect to the sample 6 and performing imaging.

Furthermore, as in Example 3, the processing apparatus 50 may alternately perform the following steps: detecting a local tilt of, for example, the bottom surface 62 of the container and determining the corresponding displacement width; and photographing the sample 6 at the location where the local tilt was detected; thereby imaging each of the multiple samples placed on the container. By alternating between determining the displacement width and photographing the sample 6 in this way, it is possible to perform high-speed imaging with the correct displacement corresponding to the local tilt of the container by performing only one scan, without the need for a pre-scan to determine the displacement width at each XY position to be photographed.

Furthermore, the processing unit 50 may control the microscope 20 and camera 40 so that imaging is performed by the camera 40 while displacing the relative position of the focal point of the objective lens 21 with respect to the sample 6 by a displacement range specified by the user. In this way, the processing unit 50 performs imaging while displacing the relative position of the focal point of the objective lens 21 with respect to the sample 6 by a displacement range specified by the user. Therefore, imaging can be performed by appropriately displacing the relative position of the focal point of the objective lens 21 with respect to the sample 6, and a clear image can be obtained.

Furthermore, the processing unit 50 may determine a first displacement range, which is the displacement range of the relative position of the focal point of the objective lens 21 with respect to the sample 6, based on the position of the container relative to the focal point of the objective lens 21. The processing unit 50 may also acquire a second displacement range, which is the displacement range of the relative position of the focal point of the objective lens 21 with respect to the sample 6, as specified by the user. Then, the processing unit 50 may control the microscope 20 and camera 40 to perform imaging with the camera 40 while displacing the relative position of the focal point of the objective lens 21 with respect to the sample 6 by the smaller of the first and second displacement ranges. In this way, the processing unit 50 may perform imaging while displacing the relative position of the focal point of the objective lens 21 with respect to the sample 6 by the smaller of the displacement range determined based on the position of the container relative to the objective lens 21 of the microscope 20 and the displacement range specified by the user. Therefore, by appropriately displacing the relative position of the focal point of the objective lens 21 with respect to the sample 6, imaging can be performed and a clear image can be obtained.

Furthermore, the processing unit 50 may determine the start and end positions, which are the relative positions of the focal point of the objective lens 21 relative to the sample 6, based on the dimensional information of the well plate 60 on which the sample 6 is placed. In addition, the processing unit 50 may control the microscope 20 and camera 40 to perform imaging by the camera 40 while displacing the relative position of the focal point of the objective lens 21 relative to the sample 6 from the start position to the end position. In this way, the processing unit 50 determines the start and end positions of the focal point of the objective lens 21 relative to the sample 6 during imaging, based on the dimensional information of the container on which the sample 6 is placed. Therefore, by appropriately displacing the relative position of the focal point of the objective lens 21 relative to the sample 6, imaging can be performed and a clear image can be obtained.

Furthermore, while Figure 7 illustrates an example where, in step S5, the control unit 51 compares the length of the movement range calculated in step S4 with the length of the movement range set by the user in step S1 and selects the shorter movement range, the configuration is not limited to this. For example, the control unit 51 may select a range that is included in both the movement range calculated in step S4 and the movement range set by the user in step S1, or a range that is included in at least one of them.

In the above embodiment, an example was described in which the ML disk 33 is mechanically connected to the Nipow disk 31 by a member 32, but the ML disk 33 and member 32 are not required. Furthermore, in the above embodiment, the microscope system 1 was described as having a confocal imaging system as the imaging system and a single camera 40 for imaging, but the system is not limited to this configuration. For example, the microscope system 1 may have an epi-illumination optical system, or multiple cameras 40 may be used to simultaneously image multiple fluorescence wavelengths.

Furthermore, in the above embodiment, an example was described in which the objective lens 21 is displaced by the drive device 22 without the sample 6 being displaced in order to displace the relative position of the focal point of the objective lens 21 with respect to the sample 6. However, the configuration is not limited to this. For example, the well plate 60, or both the objective lens 21 and the well plate 60, may be moved in a direction parallel to the optical axis of the excitation light beam 11.

Furthermore, although the above embodiment describes an example in which a well plate 60 is used as the container on which the sample 6 is placed, other sample containers such as a Petri dish, cell culture flask, microscope slide, or coverslip chamber may be used instead. Also, while imaging of the sample 6 is being performed, it is preferable to set the drive pattern so that the drive state of the drive device 22 for the focal plane of the objective lens 21 is constant speed, but this is not required. Furthermore, while imaging of the sample 6 is being performed, the direction of displacement of the focal plane of the objective lens 21 may be either in the direction that decreases the distance between the sample 6 and the objective lens 21 or in the direction that increases it.

Furthermore, while the explanation referring to Figure 6 describes an example where the displacement of the focal plane of the objective lens 21 occurs only once in one direction during imaging of the sample 6, this is not the only example. For example, the focal plane may be displaced back and forth or displaced multiple times. Also, Figure 1 shows an example where the microscope 20 is an inverted microscope, but the microscope 20 may be in other configurations such as an upright microscope.

Furthermore, the processing unit 50 may determine the position of the lower end of the displacement of the objective lens 21 based on the design information of the well plate 60. Alternatively, the processing unit 50 may determine the position by detecting the position of the bottom surface 62 of the well plate 60 using the position sensor 24.

This disclosure is not limited to the embodiments described above. For example, multiple blocks shown in the block diagram may be combined, or a single block may be divided. Multiple steps shown in the flowchart may be performed in parallel or in a different order, depending on the processing capacity of the device performing each step, or as necessary, instead of being performed chronologically as described. Other modifications are possible without departing from the spirit of this disclosure.

1. Microscope System 6: Sample 10, Light Emitting Device 11, Excitation Beam 12, Fluorescence Signal 20, Microscope 21, Objective Lens 22, Drive Device 23, Relay Lens 24, Position Sensor 241, Light Source 242, Spectroscopic Mirror 243, Spectroscopic Mirror 244, Light Sensor 30, Confocal Scanner 31, Pinhole Array Disk (Nipow Disk)
32 Components 33 Microlens array disk 34 Dichroic mirror 35 Relay lens 36 Bandpass filter 37 Relay lens 39 Rotation center axis 40 Camera 41 Two-dimensional sensor 50 Processing unit 51 Control unit 52 Storage unit 53 Input/output unit 60 Well plate 61 Frame 62 Bottom surface 63 Sample to be imaged 64 Cell culture surface 65 Imaging plane 66 Sample to be imaged 68 Well 71 Imaging Z width

Claims (9)

A microscope system for acquiring an image of a sample to be observed,
A light-emitting device that emits illumination light,
A confocal scanner having a scanning unit that scans the illumination light over the sample,
A microscope comprising: an objective lens that projects an image of the illumination light scanned by the confocal scanner onto the sample; and a position sensor that detects the position of the container on which the sample is placed relative to the focal position of the objective lens;
An imaging device, wherein the imaging surface is positioned conjugate to the scanning unit, detects the light to be measured from the sample irradiated with the aforementioned illumination light.
A processing unit that controls the operation of the microscope and the imaging device,
Equipped with,
The aforementioned processing apparatus is
Based on the local tilt of the container, the displacement range of the relative position of the focal point of the objective lens with respect to the sample is determined.
Based on the position of the container relative to the focal position of the objective lens, a control signal is transmitted to the microscope and the imaging device to continue exposure and imaging while changing the relative position of the focal point of the objective lens with respect to the sample by the determined displacement amount in a direction parallel to the optical axis of the illumination light.
Microscope system. The confocal scanner, as the scanning unit, rotates a pinhole disk on which a plurality of pinholes are arranged, and scans the illumination light irradiated from the light-emitting device onto the sample.
The objective lens projects images of the multiple pinholes, scanned by the illumination light from the confocal scanner, onto the sample.
The imaging device is configured such that the imaging surface is positioned conjugate to the plurality of pinholes.
The microscope system according to claim 1. The microscope further comprises a drive device capable of displacing the objective lens in a direction parallel to the optical axis of the illumination light,
The processing apparatus controls the microscope and the imaging device so that the drive device displaces the objective lens while the imaging device continues exposure and imaging.
The microscope system according to claim 1 or 2. The microscope system according to any one of claims 1 to 3, wherein the processing device controls the microscope and the imaging device so as to continuously perform exposure and imaging in the imaging device while changing the relative position of the focal point of the objective lens with respect to the sample at a constant velocity. The aforementioned processing apparatus is
Based on the position of the container relative to the focal position of the objective lens, the local tilt of the container is detected.
Based on the local tilt of the container detected, the displacement range is determined.
The microscope system according to any one of claims 1 to 4. The microscope system according to claim 5, wherein the processing apparatus alternately performs the following steps: detecting a local tilt of the container and determining the displacement range of the relative position of the focal point of the objective lens with respect to the sample based on the local tilt; and controlling the microscope and the imaging device so as to continue exposure and imaging in the imaging device while changing the relative position of the focal point of the objective lens by the determined displacement range, thereby imaging each of the plurality of samples placed in the container. The microscope system according to any one of claims 1 to 4, wherein the processing apparatus controls the microscope and the imaging device so as to continue exposure and imaging in the imaging device while further changing the relative position of the focal point of the objective lens with respect to the sample by a displacement range specified by the user. An imaging method for a microscope system comprising a light-emitting device, a confocal scanner, a microscope, an imaging device, and a processing device for acquiring an image of a sample to be observed,
The light-emitting device emits illumination light,
The scanning unit of the confocal scanner performs the step of scanning the illumination light with the sample,
The objective lens of the microscope projects the image of the illumination light scanned by the confocal scanner onto the sample.
The microscope's position sensor includes the step of detecting the position of the container on which the sample is placed with respect to the focal position of the objective lens,
The imaging device, whose imaging surface is positioned conjugate to the scanning unit, performs the step of detecting the light to be measured from the sample irradiated with the illumination light,
It has,
The aforementioned processing apparatus is
Based on the local tilt of the container, the displacement range of the relative position of the focal point of the objective lens with respect to the sample is determined.
Based on the position of the container relative to the focal position of the objective lens, a control signal is transmitted to the microscope and the imaging device to continue exposure and imaging while changing the relative position of the focal point of the objective lens with respect to the sample by the determined displacement amount in a direction parallel to the optical axis of the illumination light.
Imaging methods for microscope systems. A program that causes a computer to operate as a processing unit included in the microscope system described in any one of claims 1 to 7.