JP7254686B2 - Imaging system and image processing method - Google Patents

Description

The disclosed technology relates to an imaging system and an image processing method.

The following techniques are known as techniques related to an imaging system having a plurality of microscope units that respectively capture images of an object mounted on a stage. For example, Patent Document 1 discloses a stage for mounting a sample that is movable in at least one direction, ie, one axis direction or the other axis direction within a plane, and a stage that is provided in a bridge-like manner and that is mutually connected. A plurality of support members provided with adjustable intervals, and a head group comprising a plurality of microscope heads provided on each of the plurality of support members and provided with adjustable mutual intervals. A multi-head microscope apparatus is described which is characterized by:

JP-A-8-171057

Particles of about 1 μm present in a laminate obtained by laminating a layer with a thickness of about several tens to 100 μm on one side or both sides of a light-transmitting support such as a plastic film are removed from the laminate mounted on the XY stage . , are extracted by photographing each position in the XY and Z directions using a microscope unit including a microscope and an imaging device. The imaging by the microscope unit is performed a plurality of times while sequentially moving the focal position along the XY and Z directions. Therefore, as the area of the laminated plate increases, the number of times of photographing increases and the photographing time becomes longer. Moreover, the moving distance of the XY stage becomes long, and a more expensive XY stage is required.

As a method of solving this problem, a method of using a plurality of microscope units is conceivable. By having a plurality of microscope units share the imaging range and operate in parallel, the imaging time can be shortened. Also, the moving distance of the XY stage can be shortened. However, in this case, in order to extract the particles present in the laminate from the images taken by each of the plurality of microscope units, the positional relationship of the imaging fields of each of the plurality of microscope units (i.e., the It is necessary to specify the positional relationship of the images captured by each with micron-order accuracy.

Although it is possible to specify the positional relationship of the imaging fields of the microscope units from the mounting positions of the microscope units, it is difficult to specify with micron-order accuracy. In this case, the images photographed by each of the plurality of microscope units cannot be appropriately spliced together, making it difficult to extract particles present in the laminated plate .

The disclosed technique has been made in view of the above points, and aims to specify the positional relationship of the imaging fields of each of the plurality of microscope units with high accuracy.

The imaging system according to the disclosed technology is arranged with a stage movable in a first direction and a second direction orthogonal to the first direction along at least one of the first direction and the second direction, A plurality of microscope units, each equipped with a microscope for obtaining an image of an object to be photographed mounted on a stage and an imaging device for photographing the image obtained by the microscope, and a mark arranged at a predetermined position on the stage. is moved from within the imaging field of view of the first microscope unit out of the plurality of microscope units to within the imaging field of view of the second microscope unit out of the plurality of microscope units. a specifying unit that specifies the positional relationship between the field of view of the first microscope unit and the field of view of the second microscope unit based on the amount of movement in the two directions.

The specifying unit may derive coordinate values in a common coordinate system for images captured by each of the plurality of microscope units, based on the specified positional relationship.

Based on the position coordinates of the stage when the mark is positioned within the imaging field of view of the first microscope unit and the position coordinates of the stage when the mark is positioned within the imaging field of view of the second microscope unit , the positional relationship between the field of view of the first microscope unit and the field of view of the second microscope unit may be specified.

The imaging system derives an inclination angle of the coordinate axis in the imaging field of view with respect to the first direction or the second direction for each of the plurality of microscope units, and calculates the inclination angle for the image photographed by each of the plurality of microscope units. It may further include a corrector that performs rotation correction accordingly.

The correction unit corrects the microscope unit based on the moving direction of the mark arranged on the stage within the imaging field of the microscope unit when the stage is moved in the first direction or the second direction. A tilt angle may be derived.

The arrangement interval of the plurality of microscope units arranged side by side along the first direction is obtained by dividing the length of the object to be photographed in the first direction by the number of the plurality of microscope units arranged side by side along the first direction. preferably corresponds to the value

The imaging system may further include a movement mechanism that moves the focal position of each of the multiple microscope units along a third direction orthogonal to both the first direction and the second direction.

The movement mechanism may include a first movement mechanism that integrally moves the focal positions of the plurality of microscope units along the third direction. Also, the moving mechanism may include a second moving mechanism that individually moves the focal position of each of the plurality of microscope units along the third direction.

The imaging system may further include a control unit that controls each microscope unit to instruct the microscope unit to perform imaging based on the focal position of the microscope unit in the third direction.

The imaging system further includes an illumination device that irradiates illumination light to the imaging position of each of the plurality of microscope units of the imaging target from the side opposite to each of the plurality of microscope units with the imaging target interposed therebetween. You can

The illumination device may include a single light source and an optical system configured to split light from the light source and irradiate the imaging positions of each of the plurality of microscope units with the illumination light.

The imaging system may further include a detector provided for each of the plurality of microscope units and detecting the height of the object to be imaged at the imaging position of the corresponding microscope unit.

It is preferable that the microscopes included in each of the plurality of microscope units have the same optical performance, and the imaging devices included in each of the plurality of microscope units have the same imaging performance.

The plurality of microscope units includes a first microscope, a second microscope spaced in a first direction from the first microscope, and a second microscope spaced in a second direction from the first microscope. a third microscope and a fourth microscope spaced from the second microscope in a second direction and spaced from the third microscope in a first direction good.

In the image processing method of the technology disclosed herein, a stage movable in a first direction and a second direction orthogonal to the first direction is juxtaposed along at least one of the first direction and the second direction, A plurality of microscope units of an imaging system including a plurality of microscope units each equipped with a microscope for obtaining an image of an object to be photographed mounted on a stage and an imaging device for photographing the image obtained by the microscope wherein a mark arranged at a predetermined position on the stage is detected from within the imaging field of view of a first microscope unit out of the plurality of microscope units Imaging of the field of view of the first microscope unit and the field of view of the second microscope unit based on the amount of movement of the stage in the first direction and the second direction when the stage is moved within the field of view of the second microscope unit. Including specifying the positional relationship of the field of view.

The image processing method according to the technology disclosed may further include deriving coordinate values in a common coordinate system for images captured by each of the plurality of microscope units based on the specified positional relationship.

The image processing method of the disclosed technology derives the inclination angle of the coordinate axes in the photographed image with respect to the first direction or the second direction for each of the plurality of microscope units, It may further include performing a rotation correction on the tilted image according to the tilt angle.

According to the disclosed technology, it is possible to specify the positional relationship of the field of view of each of the plurality of microscope units with high accuracy.

1 is a cross-sectional view showing an example of a configuration of a nuclear emulsion plate according to an embodiment of technology disclosed; FIG. 1 is a perspective view showing an example of a configuration of an imaging system according to an embodiment of technology disclosed herein; FIG. FIG. 4 is a plan view showing an example of a positional relationship between first and second microscope units according to an embodiment of technology disclosed herein; 1 is a diagram showing an example of a configuration of first and second microscope units and an illumination device according to an embodiment of technology disclosed herein; FIG. 1 is a diagram illustrating an example of a hardware configuration of an information processing device according to an embodiment of technology disclosed herein; FIG. 1 is a functional block diagram showing an example of a functional configuration of an information processing device according to an embodiment of technology disclosed herein; FIG. FIG. 4 is a cross-sectional view showing movement paths of focal positions of the first and second microscope units according to the embodiment of the disclosed technique; FIG. 4 is a plan view showing movement paths of focal positions of the first and second microscope units according to the embodiment of the disclosed technique; FIG. 2 is a diagram showing an example of a mark according to an embodiment of technology disclosed; FIG. 2 is a diagram showing an example of a mark according to an embodiment of technology disclosed; FIG. 4 is a plan view showing a preferred arrangement of marks on a film bearing marks according to an embodiment of the disclosed technique; 4 is a flowchart showing an example of the flow of positional relationship identification processing according to an embodiment of technology disclosed herein; 4 is a diagram showing a state in which the coordinate axes of the fields of view of the first and second microscope units according to the embodiment of the technology disclosed are not tilted with respect to the X direction and the Y direction; FIG. 4 is a diagram showing a state in which the coordinate axes of the field of view of the first and second microscope units according to the embodiment of the disclosed technique are tilted at an inclination angle θ with respect to the X direction and the Y direction; FIG. 6 is a flow chart showing an example of the flow of image correction processing according to an embodiment of the disclosed technology; FIG. 5 is a diagram showing movement of a mark within the imaging field of view of the first microscope unit according to the embodiment of the disclosed technology; FIG. 5 is a diagram showing movement of a mark within the imaging field of view of the first microscope unit according to the embodiment of the disclosed technique; FIG. 10 is a plan view showing an example of the positional relationship of four microscope units included in an imaging system according to another embodiment of the disclosed technique; FIG. 4 is a plan view showing a preferred arrangement of marks according to another embodiment of the disclosed technology; FIG. 10 is a diagram illustrating an example of a configuration of a lighting device according to another embodiment of technology disclosed herein;

Embodiments of the technology disclosed herein will be described below with reference to the drawings. In each drawing, substantially the same or equivalent components or portions are given the same reference numerals.

Cosmic-ray radiography uses charged particles such as muons (μ particles), which have higher penetrability than the X-rays contained in cosmic rays, to non-destructively examine the interior of objects that cannot be observed with X-rays. It is an imaging technology. When obtaining a muon image, a nuclear emulsion plate for detecting muons is placed around the object to be observed, and tracks of muons transmitted through the object to be observed are recorded on the nuclear emulsion plate. A nuclear emulsion plate is obtained by laminating a recording layer having a thickness of several tens to 100 μm on one or both sides of a light-transmitting support such as a plastic film. As a material constituting the recording layer, a gelatin film in which silver halide crystals are uniformly dispersed is generally used.

When a charged particle such as a muon passes through a silver halide crystal, its track is retained by the crystal. Crystals that retain tracks of charged particles remain in the nuclear emulsion as silver particles (black spots) with a size of about 1 μm after development processing. Tracks of charged particles passing through a nuclear emulsion plate are recorded as three-dimensional arrays of silver particles (black dots). This three-dimensional arrangement of silver particles (black dots) is extracted by observation using an optical microscope. By specifying the positional relationship between the tracks of charged particles recorded on the nuclear emulsion plate, the image of the observed object can be derived.

The tracks of the charged particles recorded on the nuclear emulsion plate are recorded at each position in the XY and Z directions of the recording layer of the nuclear emulsion plate mounted on the XY stage using a microscope unit comprising a microscope and an imaging device. It is extracted by shooting with The imaging by the microscope unit is performed a plurality of times while sequentially moving the focal position along the XY and Z directions. Therefore, as the area of the nuclear emulsion plate becomes larger, the number of times of photographing increases and the photographing time becomes longer. Moreover, the moving distance of the XY stage becomes long, and a more expensive XY stage is required.

As a method of solving this problem, a method of using a plurality of microscope units is conceivable. By having a plurality of microscope units share the imaging range and operate in parallel, the imaging time can be shortened. Also, the moving distance of the XY stage can be shortened. However, in this case, in order to extract the tracks of the charged particles recorded on the nuclear emulsion plate from the images captured by each of the plurality of microscope units, the positional relationship of the imaging fields of each of the plurality of microscope units (i.e., positional relationship between images captured by each of the microscope units) must be specified with micron-order accuracy.

Although it is possible to specify the positional relationship of the imaging fields of the microscope units from the mounting positions of the microscope units, it is difficult to specify with micron-order accuracy. In this case, the images photographed by each of the plurality of microscope units cannot be appropriately stitched together, making it difficult to extract tracks of charged particles.

A nuclear emulsion as an object to be imaged in the imaging system 1 (see FIG. 2) according to the embodiment of the disclosed technique will be described. FIG. 1 is a cross-sectional view showing an example of the configuration of a nuclear emulsion plate NP.

The nuclear emulsion plate NP is obtained by laminating track recording layers EL having a thickness of several tens to 100 μm on both sides of a light-transmitting support FB such as a plastic film. The track-recording layer EL is a layer that records tracks of charged particles such as muons as latent images through the passage of the charged particles. As a recording material constituting the track recording layer EL, it is possible to use a gelatin film in which silver halide crystals are uniformly dispersed. When charged particles pass through the track-recording layer containing silver halide crystals, their tracks are recorded as latent images in the track-recording layer EL. By developing the track recording layer EL on which the latent image is recorded, tracks of the charged particles are retained in the track recording layer EL as a three-dimensional arrangement of silver particles (black dots).

As light-sensitive materials other than silver halide, for example, iron salt light-sensitive materials containing ferric oxalate or diazo light-sensitive materials containing aromatic diazonium salts can be used. Iron salt photosensitive materials have the property of absorbing photons and developing colors, and are used, for example, in blueprint photosensitive paper. The diazo photosensitive material is a so-called positive type photosensitive material in which the unexposed portion develops color by development, and is used, for example, in diazo photosensitive paper. When the nuclear emulsion NP is used for the purpose of extracting tracks of charged particles, silver halide is most preferable from the viewpoint of sensitivity to charged particles.

Although non-electric gamma rays themselves do not form tracks on the nuclear emulsion plate NP, since gamma rays change into electrons and positrons when they react with substances, they record their tracks as latent images on the nuclear emulsion plate NP. Neutrons do not directly sensitize the nuclear emulsion plate, but when they collide with protons in the material, the tracks of the protons are recorded on the nuclear emulsion plate NP. That is, the tracks of charged particles recorded on the nuclear emulsion plate NP include tracks of charged particles secondary to collisions with gamma rays or neutron beams.

In a non-destructive inspection using a nuclear emulsion plate NP, the nuclear emulsion plate NP is placed around an object to be inspected, and after a predetermined period of time such as several weeks, the nuclear emulsion plate NP is recovered. Next, the recovered nuclear emulsion plate NP is developed, and tracks of charged particles such as muons recorded on the nuclear emulsion plate NP are extracted from a plurality of tomographic images obtained by photographing the developed nuclear emulsion plate NP. Then, the density distribution of the object to be inspected is obtained by analysis processing using the extracted tracks.

FIG. 2 is a perspective view showing an example of the configuration of the imaging system 1 according to the embodiment of technology disclosed. The imaging system 1 includes an XY stage 20, a first microscope unit 10A, a second microscope unit 10B, a first moving mechanism 31, a second moving mechanism 35, a detector 40, an illumination device 50 (see FIG. 4), and It is configured including an information processing device 60 . The imaging system 1 is used for imaging a nuclear emulsion plate NP as an imaging target mounted on the main surface 20S of the XY stage 20 with the first microscope unit 10A and the second microscope unit 10B.

The main surface 20S of the XY stage is movable in the X and Y directions. The X direction and the Y direction are orthogonal to each other, parallel to the main surface 20S of the XY stage 20, and therefore parallel to the main surface of the nuclear emulsion plate NP mounted on the XY stage 20, respectively. The XY stage 20 includes a motor 21 for moving the main surface 20S in the X direction and a motor 22 for moving the main surface 20S in the Y direction. The XY stage 20 can use, for example, precision feed screws, and can be positioned with an accuracy of 0.1 μm. The XY stage 20 is provided with a holder 23 for vacuum-sucking the nuclear emulsion plate NP. In the mounting area of the nuclear emulsion plate NP of the holder 23, illumination light from an illumination device 50 (see FIG. 4) is introduced to the nuclear emulsion plate NP. A transparent glass 24 is attached for viewing.

The first microscope unit 10A and the second microscope unit 10B are arranged side by side along the X direction. The first and second microscope units 10A and 10B are, respectively, a microscope 11 for acquiring an image of the nuclear emulsion plate NP mounted on the XY stage 20, and an imaging device 12 for capturing the image acquired by the microscope 11. and are integrated. The first and second microscope units 10A and 10B share the photographing range and perform parallel operations.

The first and second microscope units 10A and 10B are attached to a support member 32 extending along the X direction, and their positional relationship is fixed. FIG. 3 is a plan view showing an example of the positional relationship between the first and second microscope units 10A and 10B.

Let D be the arrangement interval of the plurality of microscope units juxtaposed along the X direction, W be the length of the nuclear emulsion plate NP in the X direction, and n be the number of microscope units juxtaposed along the X direction. It is preferable to satisfy the following formula (1).
D=W/n (1)

This makes it possible to equalize the photographing ranges shared by each of the plurality of microscope units, so that the moving distance of the XY stage 20 in the X direction can be minimized. In this embodiment, the number of microscope units juxtaposed along the X direction is two (n=2), so the arrangement interval D between the first microscope unit 10A and the second microscope unit 10B is W /2 is preferred.

FIG. 4 is a diagram showing an example of the configuration of the first and second microscope units 10A and 10B and the illumination device 50. As shown in FIG. The first microscope unit 10A and the second microscope unit 10B have the same configuration. That is, the first and second microscope units 10 have the same optical performance of the microscopes 11 and the same imaging performance of the imaging devices 12 provided therein. Note that "same optical characteristics" means that the magnification of the objective lens 11o and the state of occurrence of Seidel's five aberrations are the same. “Equal image pickup performance” means that the number of pixels of the image sensor 12A is equal.

The first and second microscope units 10A and 10B are configured including an objective lens 11o, an imaging lens 11i, and an imaging device 12A. As the objective lens 11o, for example, a plan-type apochromatic objective lens of 20 times liquid immersion can be suitably used. Thereby, desired resolution, depth of field and flatness of the reading image plane can be obtained. The objective lens 11o and the imaging lens 11i constitute an infinity correcting optical system. Light passing through the objective lens 11o enters the imaging lens 11i as a parallel beam at infinity, and passes through the imaging lens 11i to the image sensor 12A. An image is formed on the imaging surface. For example, a monochrome sensor with 5000×5000 pixels can be used as the imaging device 12A.

The illumination device 50 illuminates the imaging position of each of the first and second microscope units 10A and 10B from the side opposite to the first and second microscope units 10A and 10B with the nuclear emulsion plate NP interposed therebetween. to irradiate. The illumination device 50 constitutes a Koehler illumination system and includes a strobe light source 51 , a condenser lens 52 , a field stop 53 , an aperture stop 54 and a condenser lens 55 . As the strobe light source 51, for example, a blue LED (Light Emitting Diode) can be used. As a result, the resolution of the microscope can be further increased, and at the same time, the depth of field can be reduced to increase the resolution in the depth direction of the image of the nuclear emulsion plate NP. The strobe light source 51 emits light in synchronization with photographing timings in the first and second microscope units 10A and 10B. In this embodiment, each component of the illumination device 50 shown in FIG. 4 is provided for each of the first and second microscope units 10A and 10B.

The first moving mechanism 31 and the second moving mechanism 35 move the focal positions of the first and second microscope units 10A and 10B in directions perpendicular to both the X direction and the Y direction (that is, the XY stage 20 It is moved along the Z direction (the direction perpendicular to the main surface 20S).

The first moving mechanism 31 includes a support member 32 extending along the X direction, and a pair of struts 33 connected to one end and the other end of the support member 32 and extending along the Z direction. consists of The support member 32 is attached with the first and second microscope units 10A and 10B. The support member 32 is movable in the Z direction along the support 33 . The first moving mechanism 31 is configured such that when the support member 32 moves in the Z direction along the column 33, the positions of one end and the other end of the support member 32 in the Z direction are always the same. there is The focal positions of the first and second microscope units 10A and 10B are integrally moved along the Z direction by the support member 32 moving along the Z direction.

A second moving mechanism 35 is provided in each of the first and second microscope units 10A and 10B. The second moving mechanism 35 individually moves the objective lenses 11o of the first and second microscope units 10A and 10B, thereby individually moving their focal positions along the Z direction. Since the first and second microscope units 10A and 10B constitute an infinity correction optical system, the light emitted from the objective lens 11o is parallel light, and the movement of the objective lens 11o shifts the focal position in the optical axis direction. It is possible to move along The second moving mechanism 35 may be configured to individually move the entire first and second microscope units 10A and 10B along the Z direction.

As described above, the imaging system 1 includes two types of moving mechanisms as mechanisms for moving the focal positions of the first and second microscope units 10A and 10B along the Z direction. As a result, for example, when moving the focus position over a relatively long distance (that is, when performing rough adjustment of the focus position), the first moving mechanism 31 is operated to move the focus position over a relatively short distance. (that is, fine adjustment of the focal position), the second moving mechanism 35 can be operated.

In addition, since the imaging system 1 includes the second moving mechanism 35 for individually moving the focal positions of the first and second microscope units 10A and 10B, for example, when the nuclear emulsion plate NP is warped, The focus positions of the first and second microscope units 10A and 10B can be made different from each other according to the warp. As a result, in each of the first and second microscope units 10A and 10B, the movement range of the focal position along the Z direction can be minimized. shooting time can be shortened.

As a mechanism for moving the focal positions of the first and second microscope units 10A and 10B along the Z direction, only the first moving mechanism 31 or only the second moving mechanism 35 may be provided. good. This makes it possible to simplify the configuration of the device and reduce the cost of the device compared to a configuration including both the first moving mechanism 31 and the second moving mechanism 35 .

Detectors 40 are provided in the first and second microscope units 10A and 10B, respectively. The detector 40 includes, for example, a displacement meter using laser light, and detects the height of the nuclear emulsion plate NP at the imaging position of the corresponding microscope unit. By detecting the height of the nuclear emulsion plate NP by the detector 40, the warp amount of the nuclear emulsion plate at the imaging position can be grasped. Thereby, in the first and second microscope units 10A and 10B, it is possible to adjust the focal position according to the amount of warpage of the nuclear emulsion plate. For example, if the nuclear emulsion plate NP is warped in the direction of approaching the microscope unit, the position away from the nuclear emulsion plate NP in the Z direction by a distance corresponding to the amount of warp with respect to the imaging start reference position in the Z direction is The imaging start position in the Z direction of the microscope unit may be determined.

The information processing device 60 comprehensively controls the imaging system 1 . The information processing device 60 also stores the images captured by the first and second microscope units 10A and 10B. The information processing device 60 also performs image processing on the images captured by the first and second microscope units 10A and 10B.

FIG. 5 is a diagram showing an example of the hardware configuration of the information processing device 60. As shown in FIG. The information processing device 60 includes a CPU (Central Processing Unit) 61 , a memory 62 as a temporary storage area, and a nonvolatile storage section 63 . The information processing device 60 also includes a display unit 64 such as a liquid crystal display, an input unit 65 including a keyboard, a mouse, etc., a network I/F (InterFace) 66 connected to a network, and an external I/F 67 . The information processing device 60 is connected to the first and second microscope units 10A and 10B, the XY stage 20, the first moving mechanism 31, the second moving mechanism 35, the detector 40, and the illumination device via the external I/F 67. 50.

The CPU 61 , memory 62 , storage unit 63 , display unit 64 , input unit 65 , network I/F 66 and external I/F 67 are connected to bus 68 . The information processing device 60 may be, for example, a personal computer or a server computer.

The storage unit 63 is implemented by a HDD (Hard Disk Drive), SSD (Solid State Drive), flash memory, or the like. The storage unit 63 stores a shooting control program 71 , a positional relationship identification program 72 and an image correction program 73 . The CPU 30 reads each of the above programs from the storage unit 63, expands them in the memory 62, and executes the expanded programs. The storage unit 63 also stores images captured by the first and second microscope units 10A and 10B.

FIG. 6 is a functional block diagram showing an example of the functional configuration of the information processing device 60. As shown in FIG. The information processing device 60 includes a control section 81 , a specifying section 82 and a correction section 83 . The information processing device 60 functions as a control section 81 as the CPU 61 executes the imaging control program 71 . The information processing device 60 also functions as the identification unit 82 by the CPU 61 executing the positional relationship identification program 72 . The information processing device 60 also functions as a correction unit 83 as the CPU 61 executes the image correction program 73 .

The control unit 81 controls the driving of the XY stage 20, the first moving mechanism 31, and the second moving mechanism 35, and controls the imaging timing of the first and second microscope units 10A and 10B and the light emission timing of the illumination device 50. to control.

7 and 8 are a cross-sectional view and a plan view, respectively, showing movement paths of focal positions of the first and second microscope units 10A and 10B. In FIGS. 7 and 8, the moving path of the focal position is indicated by dotted arrows. Under imaging control by the control unit 81, the first and second microscope units 10A and 10B each perform imaging of the nuclear emulsion plate NP for one imaging field 90 while changing the focus position downward in the Z direction. The imaging field of view 90 is one imaging area on the XY plane by the first and second microscope units 10A and 10B. Movement of the focus position in the Z direction is performed by the second moving mechanism 35 moving the objective lens 11o along the Z direction. Note that changing the focal position downward in the Z direction means moving the objective lens 11o along the Z direction in a direction approaching the nuclear emulsion plate NP. Further, in FIG. 7, the tracks of the charged particles are indicated by black dots, and the arrangement of the black dots is indicated by a dashed line.

A second moving mechanism 35 provided in each of the first and second microscope units 10A and 10B supplies a position signal indicating the position of the objective lens 11o of each microscope unit to the control section 81 . When the control unit 81 determines that the focal position in the Z direction indicated by this position signal has reached a predetermined position, it supplies a control signal instructing the corresponding microscope unit to take an image. That is, the control section 81 performs control for instructing imaging timing for each microscope unit based on the focal position in the Z direction of each of the first and second microscope units 10A and 10B. In addition, the control section 81 supplies a control signal for instructing light emission to the corresponding illumination device 50 in accordance with the photographing timing of each of the first and second microscope units 10A and 10B. As a result, each of the first and second microscope units 10A and 10B captures an image of the track recording layer EL in each imaging field 90 each time the focal position in the Z direction moves by 1 μm, for example. The strobe light source 51 of the illumination device 50 emits light in synchronization with the photographing timings of the first and second microscope units 10A and 10B.

When imaging the nuclear emulsion plate NP, the detector 40 detects the height of the nuclear emulsion plate NP at the imaging position of the corresponding microscope unit, and supplies a detection signal indicating the detected height to the controller 81 . For example, based on the detection signal supplied from the detector 40 provided in the first microscope unit 10A, the control unit 81 controls the first microscope unit 10A at the photographing position of the first microscope unit 10A of the nuclear emulsion plate NP. 10A, the first microscope unit moves a position away from the nuclear emulsion plate NP in the Z direction by a distance corresponding to the amount of warp with respect to the imaging start reference position in the Z direction. 10A in the Z direction is defined as the imaging start position. Similarly, in the second microscope unit 10B, the imaging start position in the Z direction is determined according to the warp amount of the nuclear emulsion plate NP at the imaging position of the second microscope unit 10B.

In each of the first and second microscope units 10A and 10B, after the tomographic image capturing for one field of view 90 is completed, the XY stage 20 moves in the X direction or Moves by one field of view along the Y direction. As a result, the focal positions of the first and second microscope units 10A and 10B move to the adjacent field of view 90. FIG. The first and second microscope units 10A and 10B photograph the track recording layer EL in the same manner as described above while changing the focus position upward in the Z direction in the new field of view 90 . Note that changing the focal position upward in the Z direction means moving the objective lens 11o along the Z direction in a direction away from the nuclear emulsion plate NP.

The first and second microscope units 10A and 10B execute the above imaging process for each field of view 90 to be photographed. Therefore, a plurality of tomographic images are obtained for each imaging field of view 90 . In the obtained tomographic image, tracks of charged particles appear as black dots. The control unit 81 stores each of the tomographic images captured by the first and second microscope units 10A and 10B in the storage unit 63 . The positional relationship between the first and second microscope units 10A and 10B is maintained constant during the imaging period. The first and second microscope units 10A and 10B share the photographing range and perform parallel operations.

The specifying unit 82 moves a mark 92 (see FIGS. 9A and 9B) placed at a predetermined position on the XY stage 20 from within the imaging field of view of the first microscope unit 10A to the imaging field of view of the second microscope unit 10B. The positional relationship between the field of view of the first microscope unit 10A and the field of view of the second microscope unit 10B is specified based on the amount of movement of the XY stage 20 in the X and Y directions when the XY stage 20 is moved inward. Based on the specified positional relationship, the specifying unit 82 also derives coordinate values in a common coordinate system for images captured by each of the first and second microscope units 10A and 10B.

9A and 9B are diagrams showing an example of marks 92 used for specifying the positional relationship of the fields of view of the first and second microscope units 10A and 10B. A mark 92 is made on a transparent film having the same size as the nuclear emulsion plate NP. The film marked with marks 92 is mounted on the XY stage 20 in place of the nuclear emulsion plate NP and observed by the first and second microscope units 10A and 10B.

As shown in FIG. 9A, the mark 92 may consist of two intersecting straight lines several microns wide. Alternatively, as shown in FIG. 9B, the mark 92 may be composed of a dot with a diameter of several microns and a plurality of straight lines with a width of several microns arranged around the dot. The mark 92 is sized to fit within the fields of view of the first and second microscope units 10A and 10B. The mark 92 may have any shape as long as the position thereof can be recognized by the first and second microscope units 10A and 10B, and is not limited to the shape described above.

FIG. 10 is a plan view showing a preferred arrangement of marks 92 on a film 94 on which marks 92 are marked. As shown in FIG. 10, the mark 92 is located anywhere on a straight line V1 passing through the midpoint of a straight line H1 connecting the optical axes of the first and second microscope units 10A and 10B and perpendicular to the straight line H1. is preferably arranged. By arranging the mark 92 at any position on the straight line V1 and moving the XY stage 20 in the X and Y directions, the mark 92 can be observed in each of the first and second microscope units 10A and 10B. It becomes possible to

The information processing device 60 functions as the specifying unit 82 by executing the positional relationship specifying program 72 by the CPU 61, and performs the positional relationship specifying process described below. FIG. 11 is a flowchart illustrating an example of the flow of positional relationship identification processing.

In step S1, the specifying unit 82 acquires the position coordinates (X1, Y1) of the XY stage 20 when the mark 92 is positioned within the imaging field of view of the first microscope unit 10A (for example, the center of the imaging field of view). In step S2, the specifying unit 82 acquires the position coordinates (X2, Y2) of the XY stage 20 when the mark 92 is positioned within the imaging field of the second microscope unit 10B (for example, the center of the imaging field). The specifying unit 82 may acquire position coordinates output from an encoder attached to the XY stage 20 or may acquire position coordinates input by the user via the input unit 65 . The process of driving the XY stage to position the mark 92 within the field of view of the first and second microscope units 10A and 10B can be performed automatically or manually.

In step S3, the identifying unit 82 determines the positions of the first and second microscope units 10A and 10B based on the position coordinates (X1, Y1) obtained in step S1 and the position coordinates (X2, Y2) obtained in step S2. Identify the positional relationship of the field of view. Specifically, the identifying unit 82 derives the difference (ΔX, ΔY) between (X1, Y1) and (X2, Y2). However, ΔX=X2-X1 and ΔY=Y2-Y1. Note that the position of the mark 92 within the field of view of the first microscope unit 10A when the position coordinates (X1, Y1) are acquired, and the position of the second microscope unit 10B when the position coordinates (X2, Y2) are acquired. If the position of the mark 92 within the field of view is deviated, it is preferable to add or subtract the amount corresponding to the deviation from the difference (ΔX, ΔY).

In step S4, the identifying unit 82 determines the image captured by the first and second microscope units 10A and 10B based on the positional relationship of the imaging fields of the first and second microscope units 10A and 10B identified in step S3. Coordinate values in a common coordinate system (global coordinate system) are derived for the image obtained by

The position of the image captured by the first microscope unit 10A is specified by local coordinate values (Xa, Ya) in a local coordinate system unique to the first microscope unit 10A. Similarly, the position of the image captured by the second microscope unit 10B is specified by local coordinate values (Xb, Yb) in the local coordinate system unique to the second microscope unit 10B. Here, it is assumed that the local coordinate values (Xa, Ya) in the first microscope unit 10A are applied as global coordinate values in the global coordinate system. In this case, the specifying unit 82 adds the difference (ΔX, ΔY) to the local coordinate values (Xb, Yb) in the second microscope unit 10B, so that the image captured by the second microscope unit 10B is represented by the global coordinates Derive the values (Xb+ΔX, Yb+ΔY). That is, the specifying unit 82 derives coordinate values (global coordinate values) in a common coordinate system (global coordinate system) for images captured by the first and second microscope units 10A and 10B. This makes it possible to appropriately join the images captured by each of the first and second microscope units 10A and 10B.

The positional relationship between the first and second microscope units 10A and 10B is specified by the specifying unit 82, for example, by adjusting the inclination of the lens barrels of the first and second microscope units 10A and 10B. This may be performed when there is a change in the positional relationship of the fields of view of the microscope units 10A and 10B.

The correction unit 83 derives the inclination angle of the coordinate axis of the field of view with respect to the X direction or the Y direction for each of the first and second microscope units 10A and 10B, Rotational correction according to the tilt angle is performed on the image photographed by each. Here, the coordinate axes of the imaging fields of the first and second microscope units 10A and 10B are the coordinate axes in the local coordinate system of the microscope units, and are also the coordinate axes of the images captured by these microscope units. The tilt angle with respect to the X direction or the Y direction is the tilt angle with respect to the movement direction of the XY stage 20 (coordinate axes in the coordinate system of the XY stage 20).

FIG. 12 shows a state in which the _coordinate axes XV and _YV of the field of view 90 of the first and second microscope units 10A and 10B are not tilted with respect to the moving directions of the XY stage 20, namely the X and Y directions. It is a diagram. On the other hand, in FIG. 13, the coordinate axes XV and _YV of the field of view 90 of the first and second microscope units 10A and 10B are tilted in the negative direction with respect to the X and _Y directions, which are the moving directions of the XY stage 20. It is a figure which shows the state which inclines at angle (theta). The imaging fields of the first and second microscope units 10A and 10B can have tilt angles different from each other.

When the coordinate axes of the imaging fields of the first and second microscope units 10A and 10B are tilted with respect to the X direction and the Y direction, which are the moving directions of the XY stage 20, the first and second microscope units 10A and 10B If the images captured by each of the above are not rotationally corrected according to the tilt angle, these images cannot be properly spliced together. The correcting unit 83 performs image rotation correction according to the tilt of the field of view of the first and second microscope units 10A and 10B, so that the images can be joined appropriately.

The information processing device 60 functions as the correction unit 83 by executing the image correction program 73 by the CPU 61, and performs image correction processing described below. FIG. 14 is a flowchart showing an example of the flow of image correction processing. In the following, an example of performing image correction processing on an image captured by the first microscope unit 10A will be described.

In step S11, the correction unit 83 acquires the position coordinates (X _V1 , Y _V1 ) of the mark 92 within the imaging field of the first microscope unit 10A. Prior to the processing in this step, the XY stage 20 is driven to position the mark 92 within the field of view of the first microscope unit 10A. Note that the correction unit 83 may acquire the position coordinates (X _V1 , Y _V1 ) input by the user via the input unit 65 .

In step S12, the correction unit 83 moves the XY stage 20 along the X direction by a predetermined distance, and then moves the position coordinates (X _V2 , Y _V2 ). Prior to the processing in this step, the XY stage 20 is driven to move the mark 92 within the imaging field of the first microscope unit 10A. Note that the correction unit 83 may acquire the position coordinates (X _V2 , Y _V2 ) input by the user via the input unit 65 .

In step S13, the correction unit 83 adjusts the inclination angle θ of the coordinate axis of the imaging field of view of the first microscope unit 10A with respect to the X direction based on the position coordinates (X _V1 , Y _V1 ) and (X _V2 , Y _V2 ). derive

Here, FIGS. 15 and 16 show the XY plane when the coordinate axes _XV and YV of the field of view of the first microscope unit 10A are tilted with respect to the coordinate axes X and _Y of the XY stage 20 at an inclination angle θ. FIG. 10 is a diagram showing movement of a mark 92 within an imaging field 90 of the first microscope unit 10A when the stage 20 is moved along the X direction; As shown in FIG. 15, when the coordinate axes XV _and YV of the field of view 90 of the first microscope unit 10A are tilted with respect to the coordinate axes X and _Y of the XY stage 20, the XY stage 20 is moved in the X direction, the mark 92 moves in a direction inclined by _.theta . That is, by deriving the movement direction of the mark 92 within the imaging field 90 of the first microscope unit 10A, the coordinate axes _XV and _YV of the imaging field 90 are aligned with the coordinate axes X and Y of the XY stage 20 (that is, the XY stage 20 ) can be derived. The movement direction of the mark 92 can be derived from the position coordinates (X _V1 , Y _V1 ) before movement and one coordinate (X _V2 , Y _V2 ) after movement. That is, the tilt angle θ can be obtained by the following formula (2).
θ=arctan {(Y _V2 −Y _V1 )/(X _V2 −X _V1 )} (2)

Thus, the correction unit 83 derives the tilt angle θ based on the moving direction of the mark 92 within the imaging field 90 when the XY stage 20 is moved along the X direction.

In step S14, the correcting unit 83 performs rotation correction by rotating each image captured by the first microscope unit 10A in the opposite direction by an angle equal to the tilt angle θ derived in step S13. The correction unit 83 also performs similar rotation correction on the image captured by the second microscope unit 10B.

In this manner, the correction unit 83 corrects the rotation of the images captured by the first and second microscope units 10A and 10B according to the inclination of the field of view of each of the images, thereby joining the images together. It is possible to do it properly. Note that the correction unit 83 derives the inclination angle of the coordinate axis of the imaging field of view with respect to the Y direction based on the movement direction of the mark 92 within the imaging field of view 90 when the XY stage 20 is moved along the Y direction. may

As described above, according to the imaging system 1 according to the embodiment of the disclosed technique, the specifying unit 82 causes the mark 92 arranged at a predetermined position on the XY stage 20 to be positioned within the imaging field of view of the first microscope unit 10A. , based on the amount of movement in the X and Y directions of the XY stage 20 when moved into the imaging field of view of the second microscope unit 10B, the field of view of the first microscope unit 10A and the field of view of the second microscope unit Identify the positional relationship of the imaging field of view of 10B. The positional relationship of the imaging fields of view is specified based on the amount of movement of the XY stage 20 when the mark 92 is moved between the imaging fields of view of the first and second microscope units 10A and 10B. can be specified with high precision (micron order). This makes it possible to appropriately join together the images captured by each of the first and second microscope units 10A and 10B.

Further, according to the imaging system 1 according to the embodiment of the disclosed technology, the correction unit 83 corrects the images acquired by the first and second microscope units 10A and 10B according to the inclination of the imaging field of each microscope unit. Since the rotation correction is performed, it is possible to appropriately join the images even when the field of view is tilted.

Further, according to the imaging system 1 according to the embodiment of the disclosed technology, each of the first and second microscope units 10A and 10B shares the imaging range and performs parallel operations. Thereby, the imaging time can be shortened as compared with the case of using a single microscope unit. Also, the moving distance of the XY stage 20 can be shortened.

Further, according to the imaging system 1 according to the embodiment of the disclosed technique, the arrangement interval of the first and second microscope units 10A and 10B juxtaposed along the X direction is set to It corresponds to the length in the X direction divided by the number of microscope units. As a result, the photographing ranges shared by the first and second microscope units 10A and 10B can be made uniform, so that the moving distance of the XY stage 20 in the X direction can be minimized.

Further, according to the imaging system 1 according to the embodiment of the technology disclosed, the first moving mechanism 31 moves the focal positions of the first and second microscope units 10A and 10B integrally along the Z direction. and a second moving mechanism 35 for individually moving each focal position. Thereby, for example, when roughly adjusting the focus position, the first moving mechanism 31 is operated, and when finely adjusting the focus position, the second moving mechanism 35 is operated. . Further, since the imaging system 1 is provided with the second moving mechanism 35, for example, when the nuclear emulsion plate NP is warped, the focal positions of the first and second microscope units 10A and 10B are adjusted according to the warp. can be different from each other. As a result, in each of the first and second microscope units 10A and 10B, the movement range of the focal position along the Z direction can be minimized. shooting time can be shortened.

[Second embodiment]
The imaging system 1 according to the first embodiment described above includes two microscope units. In contrast, the imaging system according to the second embodiment includes four microscope units.

FIG. 17 is a plan view showing an example of the positional relationship between four microscope units included in an imaging system according to the second embodiment of technology disclosed herein. The imaging system according to the second embodiment of technology disclosed herein includes a first microscope unit 10A, a second microscope unit 10B, a third microscope unit 10C, and a fourth microscope unit 10D.

The second microscope unit 10B is arranged apart from the first microscope unit 10A in the X direction. The third microscope unit 10C is arranged apart from the first microscope unit 10A in the Y direction. The fourth microscope unit 10D is arranged apart from the second microscope unit 10A in the Y direction and away from the third microscope unit 10C in the X direction.

D1 is the arrangement interval of the microscope units juxtaposed along the X direction, D2 is the arrangement interval of the microscope units juxtaposed along the Y direction, W is the length of the nuclear emulsion plate NP in the X direction, and the Y direction of the nuclear emulsion plate NP. , the number of microscope units juxtaposed along the X direction is n1, and the number of microscope units juxtaposed along the Y direction is n2, the following equations (3) and (4) are given by preferably fulfilled.
D1=W/n1 (3)
D2=L/n2 (4)

This makes it possible to equalize the photographing ranges shared by each of the plurality of microscope units, so that the moving distances of the XY stage 20 in the X direction and the Y direction can be minimized. In the configuration shown in FIG. 17, the number of microscope units juxtaposed along the X direction is two (n1=2), and the number of microscope units juxtaposed along the Y direction is two (n2=2). Therefore, it is preferable that D1=W/2 and D2=L/2.

FIG. 18 is a plan view showing a preferred arrangement of marks 92 when the first through fourth microscope units 10A-10D are arranged in the manner shown in FIG. As shown in FIG. 18, the mark 92 is formed by a straight line V1 passing through the midpoint of a straight line H1 connecting the optical axes of the first and second microscope units 10A and 10B and perpendicular to the straight line H1, and the first and third straight line V1. is preferably arranged on the intersection Q of a straight line H2 passing through the midpoint of a straight line V2 connecting the optical axes of the microscope units 10A and 10C and perpendicular to the straight line V2. By arranging the mark 92 on the intersection point Q, it is possible to observe the mark 92 in each of the first to fourth microscope units 10A to 10D by moving the XY stage 20 in the X and Y directions. .

It should be noted that the number of microscope units provided in the imaging system can be three or five or more. It is also possible for the imaging system to have two or more microscope units juxtaposed along the Y direction.

[Third Embodiment]
The imaging system 1 according to the first embodiment described above includes the illumination device 50 provided corresponding to each of the plurality of microscope units included in the imaging system 1 . In contrast, the imaging system according to the third embodiment includes a single illumination device 50 that is commonly used for each of the plurality of microscope units.

FIG. 19 is a diagram showing an example of a configuration of an illumination device 50A included in an imaging system according to the third embodiment of technology disclosed herein. The illumination device 50A includes a strobe light source 51, a condenser lens 52, condenser lenses 55A and 55B, a half mirror 56 and mirrors 57, 58 and 59. Light from the strobe light source 51 of the illumination device 50A is split by a half mirror 56 . One of the split lights is applied as illumination light to the imaging position of the first microscope unit 10A via mirrors 58 and 59 and condenser lens 55A. The other part of the split light is applied as illumination light to the imaging position of the second microscope unit 10B via the mirror 57 and the condenser lens 55B.

In this way, the illumination device 50A according to the present embodiment splits the light from a single light source and the light source, and irradiates illumination light to the imaging positions of the first and second microscope units 10A and 10B. and an optical system configured to According to the illumination device 50A according to the present embodiment, the number of parts can be reduced compared to the case where an illumination device is provided for each microscope unit, so it is possible to reduce the size and cost of the device. Become.

1 imaging system 10A first microscope unit 10B second microscope unit 10C third microscope unit 10D fourth microscope unit 11 microscope 11i imaging lens 11o objective lens 12 imaging device 12A imaging element 20 XY stage 20S main surface 21, 22 Motor 23 Holder 24 Glass 31 First Moving Mechanism 32 Supporting Member 33 Post 35 Second Moving Mechanism 40 Detector 50, 50A Illuminating Device 51 Strobe Light Source 52 Collecting Lenses 55, 55A, 55B Condenser Lens 56 Half Mirror 57, 58, 59 mirror 60 information processing device 61 CPU
62 memory 63 storage unit 64 display unit 65 input unit 66 network I/F
67 External I/F
68 bus 71 imaging control program 72 positional relationship identification program 73 image correction program 81 control unit 82 identification unit 83 correction unit 90 imaging field of view 92 mark 94 film NP nuclear emulsion plate FB support EL trace recording layer

Claims (16)

a stage movable in a first direction and a second direction orthogonal to the first direction;
A microscope for obtaining an image of an object to be photographed, which is arranged along at least one of the first direction and the second direction and each is mounted on the stage, and an image obtained by the microscope is photographed. a plurality of microscope units equipped with imaging devices for
A mark arranged at a predetermined position on the stage is moved from within the imaging field of view of a first microscope unit among the plurality of microscope units to within the imaging field of view of a second microscope unit among the plurality of microscope units. The positional relationship between the field of view of the first microscope unit and the field of view of the second microscope unit is specified based on the amount of movement of the stage in the first direction and the second direction when the a specific part to
For each of the plurality of microscope units, an inclination angle of the coordinate axes in the imaging field of view is derived with respect to the first direction or the second direction, and the inclination angle is calculated for the image photographed by each of the plurality of microscope units. a correction unit that performs rotation correction according to
shooting system, including The imaging system according to claim 1, wherein the identifying unit derives coordinate values in a common coordinate system for images captured by each of the plurality of microscope units, based on the identified positional relationship. The specifying unit provides the position coordinates of the stage when the mark is positioned within the imaging field of view of the first microscope unit, and the position coordinates of the stage when the mark is positioned within the imaging field of view of the second microscope unit. 3. The imaging system according to claim 1, wherein the positional relationship between the field of view of said first microscope unit and the field of view of said second microscope unit is specified based on the position coordinates of the stage. The correction unit is configured to determine the moving direction of the mark placed on the stage within the imaging field of view of the microscope unit when the stage is moved in the first direction or the second direction. to derive the tilt angle for the microscope unit
The imaging system according to any one of claims 1 to 3 . The arrangement interval of the plurality of microscope units juxtaposed along the first direction is the length of the object to be photographed in the first direction, the plurality of microscope units juxtaposed along the first direction. The imaging system according to any one of claims 1 to 4 , which corresponds to a value divided by the number of . 6. The movement mechanism for moving the focus position of each of the plurality of microscope units along a third direction orthogonal to both the first direction and the second direction. The imaging system according to any one of items 1 and 2. 7. The imaging system according to claim 6 , wherein said moving mechanism includes a first moving mechanism for integrally moving the focal positions of said plurality of microscope units along said third direction. 8. The imaging system according to claim 6 , wherein the moving mechanism includes a second moving mechanism that individually moves the focal position of each of the plurality of microscope units along the third direction. 9. The controller according to any one of claims 6 to 8 , further comprising a control unit that controls each microscope unit to instruct the microscope unit to take an image based on the focal position of the microscope unit in the third direction. The imaging system according to paragraph. Further comprising an illumination device that irradiates illumination light from a side opposite to each of the plurality of microscope units with the object to be photographed interposed therebetween, at the photographing position of each of the plurality of microscope units of the object to be photographed. The imaging system according to any one of claims 1 to 9 . The lighting device
a single light source;
an optical system configured to split the light from the light source and irradiate the illumination light onto each imaging position of the plurality of microscope units;
11. The imaging system of claim 10 , comprising: 12. The detector according to any one of claims 1 to 11 , further comprising a detector provided corresponding to each of the plurality of microscope units and detecting the height of the photographed object at the photographing position of the corresponding microscope unit. The imaging system described. the optical performance of the microscope provided in each of the plurality of microscope units is equivalent,
13. The imaging system according to any one of claims 1 to 12 , wherein the imaging devices included in each of the plurality of microscope units have the same imaging performance. The plurality of microscope units are
a first microscope;
a second microscope spaced apart from the first microscope in the first direction;
a third microscope spaced from the first microscope in the second direction;
a fourth microscope spaced from the second microscope in the second direction and spaced from the third microscope in the first direction;
The imaging system according to any one of claims 1 to 13 , comprising: a stage movable in a first direction and a second direction perpendicular to the first direction; By each of the plurality of microscope units of an imaging system including a plurality of microscope units equipped with a mounted microscope for obtaining an image of an object to be photographed and an imaging device for photographing the image obtained by the microscope A method of processing a captured image, comprising:
A mark arranged at a predetermined position on the stage is moved from within the imaging field of view of a first microscope unit among the plurality of microscope units to within the imaging field of view of a second microscope unit among the plurality of microscope units. The positional relationship between the field of view of the first microscope unit and the field of view of the second microscope unit is specified based on the amount of movement of the stage in the first direction and the second direction when the death,
For each of the plurality of microscope units, an inclination angle of the coordinate axis in the photographed image is derived with respect to the first direction or the second direction, and for the image photographed by each of the plurality of microscope units, the Perform rotation correction according to the tilt angle
An image processing method comprising: 16. The image processing method according to claim 15 , wherein coordinate values in a common coordinate system are derived for images captured by each of said plurality of microscope units based on said identified positional relationship.

Images