JP7268992B2 - Magnifying observation device - Google Patents

Description

The present invention relates to a magnifying observation device.

In general, a microscope has been required to have an optical system having high optical performance in order to observe an object to be observed with high accuracy. In addition, microscopes that display observation images in real time on a display have appeared, and microscope manufacturers have been competing to see how they can accurately and precisely display the actual appearance of the observation target.

On the other hand, the performance of image processing chips has improved, and they can now be used in microscopes. In addition, the users of microscopes have expanded not only to biological researchers but also to inspectors who inspect (magnify) products produced in factories. As a result, market needs for microscopes with a different orientation from the conventional pursuit of optical performance have arisen. This is sometimes called a magnifying observation device to distinguish it from a conventional microscope (Patent Document 1).

JP 2018-013734 A Japanese Patent Application Laid-Open No. 2017-531201

A magnifying observation device has a plurality of objective lenses, and an appropriate illumination device may be different for each objective lens. In such a case, it lacks usability if the user manually switches the lighting device. SUMMARY OF THE INVENTION Accordingly, an object of the present invention is to improve the convenience of a user who uses a magnifying observation apparatus.

The present invention, for example,
A magnifying observation device for observing an observation object,
an optical system including a first objective lens, a second objective lens having a shorter working distance and higher magnification than the first objective lens , and an imaging lens;
a lens selector that alternatively arranges the first objective lens and the second objective lens in an observation optical path of the optical system;
an XY stage movable relative to the optical system in at least the X direction and the Y direction;
an illumination device that irradiates an observation object placed on the XY stage with illumination light from different directions;
an imaging unit that receives light from the observation target through the optical system and generates a luminance image of the observation target;
A control unit that controls the lighting device and the imaging unit ,
The lighting device
a first illumination unit that irradiates the observation object with illumination light that passes around the first objective lens without passing through the first objective lens ;
a second illumination unit that irradiates the observed object with illumination light passing through the second objective lens;
has
The first lighting unit is
a first light source that irradiates the observation object with the illumination light from a first illumination direction;
a second light source that irradiates the observation object with the illumination light from a second illumination direction;
has
The second lighting unit is
a third light source that irradiates the observation object with the illumination light from the first illumination direction;
a fourth light source that irradiates the observation object with the illumination light from the second illumination direction;
has
The control unit controls the illumination device to selectively use the first illumination unit and the second illumination unit according to the objective lens arranged in the observation optical path,
When the first objective lens is arranged in the observation optical path, the control unit turns on the first light source to direct illumination light passing around the first objective lens from the first illumination direction. irradiating an observation object, receiving illumination light from the observation object through the first objective lens by the imaging unit to generate a first luminance image of the observation object, and the second light source irradiates the observation object from the second illumination direction with illumination light passing around the first objective lens by lighting the observation object through the first objective lens by the imaging unit controlling the lighting device and the imaging unit to receive illumination light from the observation object and generate a second luminance image of the observation object;
When the second objective lens is arranged in the observation optical path, the control unit turns on the third light source to illuminate the illumination light passing through the second objective lens from the first illumination direction. illuminating an object, receiving illumination light from the observation object through the second objective lens by the imaging unit to generate a third luminance image of the observation object, and using the fourth light source By turning on, illumination light passing through the second objective lens is applied to the observation object from the second illumination direction, and the imaging unit emits illumination from the observation object through the second objective lens. controlling the lighting device and the imaging unit to receive light and generate a fourth luminance image of the observed object;
The magnifying observation device further generates a first observation image based on the first luminance image and the second luminance image when the first objective lens is arranged in the observation optical path, and A magnifying observation apparatus is provided, comprising an image generating section that generates a second observation image based on the third luminance image and the fourth luminance image when a lens is arranged in the observation optical path.

According to the present invention, since the illumination unit is automatically selected according to the objective lens, convenience for the user using the magnifying observation apparatus is improved.

FIG. 2 is a diagram for explaining an overview of the magnifying observation device 100; Diagram explaining the control unit, etc. Diagram explaining the image processor Diagram explaining focus stacking and unevenness enhancement A diagram showing an example of the user interface A diagram showing an example of the user interface Flowchart for explaining magnifying observation processing Diagram explaining the optical system Diagram explaining switching of the imaging lens Diagram explaining switching of the imaging lens Diagram explaining switching of the imaging lens Diagram explaining switching of the imaging lens Diagram explaining the objective lens Diagram explaining the objective lens Diagram explaining the objective lens Diagram explaining the light source of coaxial epi-illumination Diagram explaining ring lighting Diagram explaining ring lighting Flowchart showing how to select lighting

<Magnifying observation device>
FIG. 1 shows a magnifying observation device 100 . The magnifying observation apparatus 100 is an apparatus for magnifying and displaying a workpiece such as a sample such as a minute object, an electronic component, or a workpiece (hereinafter referred to as an observation object). A user can use the magnifying observation apparatus 100 to inspect the appearance of an observation object, perform dimensional measurement, and the like. The magnifying observation device 100 may be called a microscope or a digital microscope. Observation targets are not limited to the examples described above, and various objects can be observation targets.

The observation section 1 has a base section 10 , a stand section 20 , a head section 22 and a mounting table 30 . The base portion 10 is a portion for placing the observation portion 1 on a desk or the like without wobbling, and constitutes a substantially lower half portion of the observation portion 1 . A mounting table 30 is provided on the base portion 10 . The mounting table 30 is supported on the front portion of the base portion 10 from the vicinity of the central portion in the front-rear direction, and protrudes upward from the base portion 10 . The mounting table 30 is a portion for mounting an observation object, and is configured by an electric mounting table in this embodiment. That is, the observation object can be supported movably in both the width direction (X direction) and the depth direction (Y direction) of the electric table, and can be rotated in the vertical direction (Z direction) and around the Z axis. It's like The stand portion 20 is swingable with respect to the base portion 10 . For example, the stand section 20 can swing the observation section 1 clockwise and counterclockwise when viewed from the front. As the stand section 20 swings, the head section 22 also swings. The stand part 20 and the head part 22 are attached so as to be movable in the Z-axis direction. The head unit 22 has an objective lens, an image forming lens, an illumination device, an imaging device, and the like. The head unit 22 irradiates an observation object placed on a mounting table 30 with illumination light, detects the amount of reflected light or transmitted light from the observation object of the illumination light, and produces an image of the observation object. Generate. The details of the configuration and function of the observation unit 1 are disclosed in Japanese Patent Application No. 2018-161347 filed by the same applicant as the present application, and the entire disclosure thereof is incorporated herein by reference. be.

The display unit 2 has a display screen 2a capable of color display, such as a liquid crystal display panel or an organic EL panel, and is supplied with power from the outside. A touch operation panel (an example of a reception unit) may be incorporated in the display screen 2a. Further, in this embodiment, an example in which the control unit 60 is incorporated in the display unit 2 is explained, but the control unit 60 may be incorporated in the observation unit 1 or in the console unit 3 without being limited to this. It may be incorporated, or an external unit separate from the display section 2, the observation section 1 and the console section 3 may be used. The display unit 2 and the observation unit 1 are connected by a cable 5 so that signals can be transmitted and received. Power supply to the observation unit 1 may be performed by the cable 5 or may be performed by a power cable (not shown).

The console unit 3 is connected to the control unit 60, and is different from a general keyboard and mouse. It is a dedicated operation device that can be used to specify, for example. A mouse 4 is connected to the control unit 60 as a pointing device. Since the console unit 3 and the mouse 4 only need to be able to operate the magnifying observation device 100, they may be touch panel input devices, voice input devices, or the like. In the case of a touch panel type input device, it can be integrated with the display unit 2 and configured to be able to detect an arbitrary position on the display screen displayed on the display unit 2 . The console unit 3 and the mouse 4 are reception units that receive input of an arbitrary position designated by the user on the image displayed on the display unit 2 .

In addition to the devices and devices described above, the magnifying observation apparatus 100 can also be connected to a device for operation and control, a printer, a computer for performing various other processes, a storage device, peripheral devices, and the like. In this case, the connection is, for example, IEEE1394, RS-232x, RS-422, USB serial connection, parallel connection, or electrical or magnetic through a network such as 10BASE-T, 100BASE-TX, 1000BASE-T. A method of connecting them physically or optically can be mentioned. In addition to wired connection, IEEE802. Wireless connection using wireless LAN such as x, radio waves such as Bluetooth (registered trademark), infrared rays, optical communication, or the like may be used. Furthermore, various memory cards, magnetic disks, magneto-optical disks, semiconductor memories, hard disks, etc., can be used as storage media used in storage devices for exchanging data and storing various settings. The magnifying observation apparatus 100 can also be said to be a magnifying observation system that combines the various units, devices, and devices described above.

<Control part>
As shown in FIG. 2, the control section 60 has a CPU 61, an image processor 66, a storage section 67, and the like. The CPU 61 implements various functions by executing programs stored in the ROM area of the storage unit 67 . The imaging control section 62 controls the revolver drive section 24 to rotate the revolver 21 provided on the head section 22 . Thereby, the magnification of the objective lens 23 is changed. That is, the objective lens 23 with a certain magnification is switched to the objective lens 23 with another magnification. The imaging control unit 62 may have an identification unit (recognition unit) that identifies the objective lens 23 by communicating with the objective lens 23 . The head section 22 may have multiple imaging lenses 41 . The imaging control unit 62 switches the imaging lens 41 by driving the imaging lens driving unit 42 (eg, motor). Thereby, the magnification of the optical system including the objective lens 23 and the imaging lens 41 is changed. When distinguishing between the plurality of objective lenses 23 and the plurality of imaging lenses 41, a lower-case alphabetic character is attached to the end of the reference numeral. The electric diaphragm 37 is a variable diaphragm provided inside the head portion 22 and inside the objective lens 23 . The electric diaphragm 37 is also controlled by the imaging control section 62 .

The imaging control unit 62 controls the Z-direction driving unit 28 that vertically moves the head unit 22 to change the focus position of the objective lens 23 . The imaging control unit 62 moves the mounting table 30 in the X direction, the Y direction, and the Z direction through the mounting table driving unit 29, and rotates the mounting table 30 by θ. The imaging control unit 62 controls the imaging unit 25 provided in the head unit 22, causes the imaging unit 25 to capture an image of the observation object W, and acquires an image of the observation object W. FIG.

The display control unit 63 causes the display unit 2 to display an image of the observation object W and the like. The illumination control unit 64 controls turning on and off the ring illumination 26 and the coaxial epi-illumination 27 provided in the head unit 22 . The UI unit 65 displays a UI (user interface) on the display unit 2 through the display control unit 63 and receives user instructions input from the console unit 3 or the mouse 4 . The image processor 66 creates various image data from the image signal acquired by the imaging section 25 . Image processor 66 may be implemented by CPU 61 . The storage unit 67 has a ROM area, a RAM area, a memory card, and the like.

The detection unit 68 detects whether the mounting table 30 is stationary or moved, and whether or not the revolver 21 has changed the magnification of the objective lens 23 . The console section 3 may have a joystick for instructing movement of the mounting table 30 in the X direction or the Y direction. In this case, the detection unit 68 detects that the mounting table 30 is moving if the joystick of the console section 3 is tilted in any direction, and that the mounting table 30 is moving if the joystick is stationary at the neutral position. Detect when stationary. Also, when the rotation of the revolver 21, that is, the magnification change of the objective lens 23 is instructed through the console section 3 or the mouse 4, and the revolver driving section 24 rotates the revolver, the detection section 68 detects that the magnification has been changed. If there is no instruction to change the magnification, the detection unit 68 detects that the magnification has not been changed. The detection unit 68 may detect manual adjustment of the focus position by the user.

<Image processor>
As shown in FIG. 3 , in the image processor 66 , the brightness image generation section 31 creates a brightness image from the image signal acquired by the imaging section 25 through the imaging control section 62 . The HDR processing unit 32 controls the imaging unit 25 through the imaging control unit 62 to acquire a plurality of sub luminance images with different exposure times, and generates a luminance image by performing HDR processing on the plurality of sub luminance images. . HDR is an abbreviation for high dynamic range. The focus compositing unit 33 controls the imaging unit 25 through the imaging control unit 62 to obtain a plurality of sub-luminance images with different in-focus positions, and composes the plurality of sub-luminance images to obtain a depth composite image. Generate. Note that the sub luminance image may be a luminance image subjected to HDR processing. The unevenness enhancement unit 34 synthesizes a first brightness image of the observation object W illuminated from the first illumination direction and a second brightness image of the observation object W illuminated from the second illumination direction, thereby performing observation. An unevenness-enhanced image in which unevenness on the surface of an object W is emphasized is created. The first illumination direction and the second illumination direction are symmetrical with respect to the optical axis A1. Note that the first luminance image and the second luminance image may be depth-stacked images that are depth-stacked. For example, the unevenness enhancement unit 34 may generate an unevenness-enhanced image based on the luminance difference between the first focus stacking image and the second focus stacking image. The luminance value i1 of the pixel (coordinates x, y) in the first focus stacking image and the luminance value i2 of the pixel (coordinates x, y) in the second focus stacking image are used to determine the value of the pixel (coordinates x, y) in the unevenness-enhanced image. The pixel value Ixy may be obtained from the following equation.
Ixy=(i1-i2)/(i1+i2) (1)
The coloring unit 36 acquires color information from the color image of the observed object W acquired by the imaging unit 25 and colors the unevenness-enhanced image. The height image generation unit 35 obtains the height of the surface of the observation target W for each pixel by integrating each pixel of the unevenness-enhanced image, and generates a height image having the height as each pixel. The height image generator 35 may generate a color height image by coloring each pixel of the height image according to the height of each pixel.

<Principles of focus stacking and unevenness enhancement>
FIG. 4A shows that illumination light is applied to the observed object W from the first illumination direction by lighting a part of the ring illumination 26 . The ring illumination 26 has four light source areas 140A, 140B, 140C and 140D. That is, the ring illumination 26 can irradiate the observation object W with illumination light from four illumination directions by selectively turning on and off the four light source regions 140A, 140B, 140C, and 140D. The illumination direction of the light source area 140A and the illumination direction of the light source area 140C are symmetrical with respect to the optical axis A1. The illumination direction of the light source region 140B and the illumination direction of the light source region 140D are symmetrical with respect to the optical axis A1. In FIG. 4A, only the light source area 140A is illuminated.

FIG. 4B shows a luminance image I1 obtained by irradiating the observed object W with illumination light from the first illumination direction. SH indicates the shadow of the object W to be observed.

FIG. 4C shows that the observation object W is irradiated with illumination light from the second illumination direction by lighting only the light source region 140C. FIG. 4C shows a luminance image I2 obtained by irradiating the observed object W with illumination light from the second illumination direction.

FIG. 4(E) shows n first luminance images I11 to I1n acquired while illuminating the observed object W from the first illumination direction and changing the focus position. The focus stacking unit 33 creates a first focus stacking image Ia1 by stacking the n first luminance images I11 to I1n. For example, the focus stacking unit 33 analyzes a plurality of pixels having the same pixel position in each of the plurality of first luminance images I11 to I1n, and selects the pixel with the highest degree of focus among the plurality of pixels. A first depth-stacked image I1a composed of in-focus pixels at a plurality of pixel positions is generated by selecting the in-focus pixels at the positions.

FIG. 4F shows n second luminance images I21 to I2n acquired while illuminating the observed object W from the second illumination direction and changing the focal position. The focus stacking unit 33 creates a second focus stacking image I2a by stacking n second luminance images I21 to I2n. For example, the focus stacking unit 33 analyzes a plurality of pixels having the same pixel position in each of the plurality of second luminance images I21 to I2n, and selects the pixel with the highest degree of focus among the plurality of pixels. A second depth-stacked image I2a composed of in-focus pixels at a plurality of pixel positions is generated by selecting the in-focus pixels at the positions.

The unevenness enhancement unit 34 generates an unevenness-enhanced image using the first focus stacking image I1a and the second focus stacking image I2a. The unevenness enhancement unit 34 generates an unevenness-enhanced image based on the luminance difference between the first focus stacking image I1a and the second focus stacking image I2a.

<Moving the table and displaying the image>
The UI unit 65 displays the unevenness-enhanced image generated by the unevenness enhancement unit 34 on the display unit 2 . The user utilizes the unevenness-enhanced image for magnifying and observing a part of the observation object W. FIG. Here, there are cases in which a plurality of locations on the observation target W are targets for magnified observation. In this case, the user operates the console unit 3 to move the mounting table 30 in the X direction or the Y direction, or rotate it in the θ direction. When a movement instruction in the X direction is input from the console section 3, the CPU 61 causes the mounting table drive section 29 to move the mounting table 30 in the X direction, and the CPU 61 receives a movement instruction in the Y direction from the console section 3. Then, the mounting table drive unit 29 moves the mounting table 30 in the Y direction. As long as such a movement instruction is input, the CPU 61 continuously moves the mounting table 30 according to the movement instruction. When the input of the movement instruction by the user stops, the CPU 61 stops the movement of the mounting table 30 by the mounting table driving section 29 .

Here, a certain amount of processing time is required to generate the unevenness-enhanced image. Images cannot be displayed on the display unit 2. Therefore, the UI unit 65 displays the brightness image of the observation object W generated by the brightness image generation unit 31 on the display unit 2 while the mounting table 30 is moving. As a result, the user can confirm whether or not the next observation site is positioned within the visual field range of the imaging unit 25 by confirming the luminance image displayed on the display unit 2 . Here, the UI unit 65 updates the luminance image of the observation object W at regular time intervals, so that the display unit 2 displays the luminance image of the observation object W like a moving image. When the CPU 61 detects that the input of the movement instruction has stopped (stop of the mounting table 30), the CPU 61 instructs the image processor 66 to generate an unevenness-enhanced image. As a result, the acquisition of the luminance image, the focus stacking, and the unevenness-enhanced image described above are executed again, and the unevenness-enhanced image at the new observation position is displayed on the display unit 2 . The unevenness-enhanced image is regenerated in response to the stop of the mounting table 30 even if the user does not instruct the regeneration of the unevenness-enhanced image. Therefore, the user will be able to observe a plurality of observation sites efficiently.

<Coloring of unevenness-enhanced image>
Each pixel forming the unevenness-enhanced image indicates the unevenness of the surface of the observed object W, and does not include surface color information. Therefore, it is difficult for the user to grasp the relationship between the color of the observation site and the position of the unevenness. Therefore, the coloring section 36 turns on all the light source areas 140A to 140D of the ring illumination 26, and causes the imaging section 25 to image the observed object W. FIG. As a result, the luminance image generator 31 generates a color image (luminance image) of the observed object W. FIG. The coloring unit 36 acquires color information from the color image, maps the color information to the unevenness-enhanced image, colors the unevenness-enhanced image, and displays it on the display unit 2 . This makes it easier for the user to grasp the relationship between the color of the observation site and the position of the unevenness.

<HDR processing>
When the surface of the observation object W is metal, pixels with blown-out highlights and pixels with blown-up shadows may occur in the luminance image. In such a case, it becomes difficult to confirm the unevenness in the finally generated unevenness-enhanced image. Therefore, the HDR processing unit 32 may be employed. The HDR processing unit 32 acquires a plurality of sub-brightness images with different exposure times for one in-focus position, and performs HDR processing on the plurality of sub-brightness images to generate a single brightness image. The HDR processing unit 32 acquires a plurality of sub luminance images with different exposure times each time the focus position is changed, and performs HDR processing on the plurality of sub luminance images to generate one luminance image. As a result, the first luminance images I11 to I1n and the second luminance images I21 to I2n are both HDR-processed images. Therefore, the focus stacking unit 33 performs focus stacking on the HDR-processed first luminance images I11 to I1n to generate a first depth image I1a, and performs focus stacking on the HDR-processed second luminance images I21 to I2n. , to generate a second depth image I2a. Further, the unevenness enhancement unit 34 generates an HDR-processed unevenness-enhanced image by synthesizing the HDR-processed first depth image I1a and the HDR-processed second depth image I2a. As a result, blown-out highlights and blocked-up shadows are less likely to occur, so that the user can easily grasp the unevenness of the observation target W more accurately by checking the unevenness-enhanced image.

<Height image>
The unevenness-enhanced image described above does not include height information of the surface of the observation object W. FIG. Therefore, the crater illusion occurs. The crater illusion is a phenomenon in which an observer mistakenly recognizes a concave portion and a convex portion because a concave shape and a convex shape cannot be distinguished in an image. Therefore, the height image generation unit 35 obtains the height of the surface of the observation object W for each pixel by integrating each pixel of the unevenness-enhanced image, and generates a height image having the height as each pixel. and may be displayed on the display unit 2. Furthermore, the height image generating unit 35 converts the height data of each pixel of the height image into color information, and maps the color information to the unevenness emphasized image, thereby coloring the image with different colors depending on the height. An unevenness-emphasized image may be generated and displayed on the display unit 2 . This will make it easier for the user to distinguish between a concave shape and a convex shape on the surface of the observation object W based on the color information.

<Others>
Although light source regions 140A and 140C have been described in FIG. 4, this description is also applicable to light source regions 140B and 140D. That is, the image processor 66 may generate an unevenness-enhanced image using the pair of light source regions 140A and 140C, or may generate an unevenness-enhanced image using the pair of light source regions 140B and 140D. The illumination direction of the light source region 140B may be called a third illumination direction. The illumination direction of the light source region 140D may be called a fourth illumination direction. The third illumination direction and the fourth illumination direction are symmetrical (line symmetrical) across the optical axis A1.

The brightness image generation unit 31 controls the ring illumination 26 to irradiate the observation object W with illumination light from a third illumination direction different from the first illumination direction and the second illumination direction. . The luminance image generation unit 31 controls the Z-direction driving unit 28 and the imaging unit 25, and acquires a plurality of third luminance images by imaging the observation target W at each of a plurality of different in-focus positions. Similarly, the brightness image generator 31 irradiates the observation object W with illumination light from the fourth illumination direction by controlling the ring illumination 26 . The luminance image generation unit 31 controls the Z-direction driving unit 28 and the imaging unit 25, and acquires a plurality of fourth luminance images by imaging the observation target W at each of a plurality of different in-focus positions. The image processor 66 converts the unevenness-enhanced image corresponding to the lighting direction selected by the UI unit 65 out of the first lighting direction, the second lighting direction, the third lighting direction, and the fourth lighting direction into a plurality of first luminance images, It is generated using a plurality of luminance images corresponding to the illumination direction selected from among the plurality of second luminance images, the plurality of third luminance images and the plurality of fourth luminance images. The display unit 2 displays an unevenness-enhanced image corresponding to the selected illumination direction. Although the first lighting direction is selected in the above embodiments, the second lighting direction may be selected. In this case, i1 and i2 in equation (1) are interchanged. When the third illumination direction is selected, the luminance value of each pixel in the third luminance image is i1, and the luminance value of each pixel in the fourth luminance image is i2 in equation (1). When the fourth illumination direction is selected, the luminance value of each pixel in the fourth luminance image is i1, and the luminance value of each pixel in the third luminance image is i2 in equation (1).

<Modification>
In the above-described embodiment, unevenness enhancement is performed after focus stacking is performed. However, unevenness enhancement may be performed first, and then focus stacking may be performed.

The unevenness enhancement unit 34 uses the first brightness image I11 and the second brightness image I21 to generate the sub- unevenness-enhanced image I1x. Similarly, the unevenness enhancement unit 34 uses the first brightness image I12 and the second brightness image I22 to generate the sub- unevenness-enhanced image I2x. Repeating the same as below, finally, the unevenness enhancement unit 34 generates the sub unevenness-enhanced image Ixn using the first luminance image I1n and the second brightness image I2n. As a result, the focus stacking unit 33 performs focus stacking of the n sub unevenness-enhanced images Ix1 to Ixn, generates a final unevenness-enhanced image, and displays it on the display unit 2 .

The ring illumination 26 may be arranged around the objective lens 23 , or may be configured to irradiate the observation object W with illumination light through the objective lens 23 .

<User interface>
FIG. 5 shows a user interface 70 displayed on the display section 2 by the UI section 65 . The image display area 71 displays an image output from the image processor 66 through the display control section 63 . This image includes a live image of the observed object W, a depth-combined unevenness-enhanced image, an unevenness-enhanced image colored according to the color of the surface of the observed object W, and an unevenness-enhanced image colored based on height data ( color height image), etc. When the detection unit 68 detects that the mounting table 30 has moved or that an instruction to switch the objective lens 23 has been issued, the UI unit 65 displays a live image of the observation target W in the image display area 71 . When the detection unit 68 detects that the magnification has been changed and that the mounting table 30 has stopped, the UI unit 65 displays the depth-composited unevenness-enhanced image in the image display area 71 . When it is detected that the mounting table 30 moves while the unevenness emphasized image is being displayed, the UI unit 65 displays the live image in the image display area 71 . The illumination direction selection section 72 has four buttons corresponding to the light source areas 140A to 140D. As described above, since the light source region 140A and the light source region 140C form a pair, when the button corresponding to the light source region 140A is selected by the pointer 74, the illumination direction of the light source region 140A is changed to the first illumination direction. is determined, and the illumination direction of the light source region 140C is determined as the second illumination direction. Similarly, when the button corresponding to the light source region 140D is selected by the pointer 74, the lighting direction of the light source region 140D is determined as the first lighting direction, and the lighting direction of the light source region 140B is determined as the second lighting direction. The magnification selection section 73 is a pull-down list for selecting any one of the multiple objective lenses 23 attached to the revolver 21 . For example, the UI unit 65 acquires the magnification of each of the multiple objective lenses 23 attached to the revolver 21 from the setting information of the objective lenses 23 and creates a pull-down list. When the user operates the pointer 74 to select any magnification, the UI section 65 notifies the revolver driving section 24 of the selected magnification (objective lens 23). The revolver driving unit 24 rotates the revolver 21 so that the objective lens 23 with the notified magnification is positioned on the photographing optical axis A1. When the revolver drive unit 24 finishes rotating, it outputs a rotation end signal to the CPU 61 . Upon receiving the rotation end signal, the detection unit 68 determines that the magnification change has been completed. Normally, once the change in magnification is completed, it is necessary to adjust the position of the observation site within the field of view. The user positions the observation site at a desired position within the visual field range by moving the mounting table 30 . When the mounting table 30 stops, the UI unit 65 instructs the image processor 66 to generate an unevenness-enhanced image. The UI unit 65 displays the depth-stacked unevenness-enhanced image in the image display area 71 instead of the live image.

Note that the user interface 70 may be provided with a switching button for explicitly switching between display of the unevenness-enhanced image and display of the live image. Accordingly, after the unevenness-enhanced image is completed, the user may compare the unevenness-enhanced image and the live image by switching between them.

FIG. 6 shows another example of the user interface 70 displayed on the display unit 2 by the UI unit 65 . In this example, the image display area 71 displays a live image, and the image display area 75 displays a depth-combined unevenness-enhanced image, an unevenness-enhanced image colored according to the color of the surface of the observation object W, and height data. Display colored or unevenness-enhanced images (color height images) based on Although the image display areas 71 and 75 are arranged vertically in FIG. 6, they may be arranged horizontally.

Based on the color information of the surface of the observation object W, the coloring designation unit 76 has a radio button for instructing coloring of the depth-composited unevenness-enhanced image, and the height information of the surface of the observation object W. Based on this, it has a radio button for instructing to color the unevenness-enhanced image that has undergone depth composition, and a radio button for instructing not to color it. The UI unit 65 notifies the image processor 66 of the coloring instruction designated by the coloring designation unit 76 . The coloring designation section 76 may also be provided in the user interface 70 shown in FIG. The focus stacking instruction section 77 is a check box for designating whether to enable or disable focus stacking. When focus stacking is disabled in the focus stacking instruction unit 77, the unevenness enhancement unit 34 generates a unevenness-enhanced image using the first luminance image and the second luminance image according to Equation (1). In this case, the image display area 75 displays an unevenness-enhanced image that is not depth-stacked.

<Flowchart>
FIG. 7 is a flowchart showing magnifying observation processing. When the mouse 4 or the console unit 3 instructs to start, the CPU 61 executes the following processes.

A case where the unevenness-enhanced image is displayed in the image display area 71 will be described.

In S0, the CPU 61 controls the mounting table driving section 29 in accordance with the movement instruction input by the mouse 4 or the console section 3 to move the mounting table 30. FIG.

When the movement start is detected in S1, a live image is displayed in the image display area 71. FIG. That is, the CPU 61 controls the imaging unit 25 and the ring illumination 26 via the imaging control unit 62 and the illumination control unit 64 to image the observation object W, controls the image processor 66, and produces a live image of the observation object W ( A luminance image updated at regular time intervals) is generated, and a live image is displayed on the display unit 2 via the display control unit 63 . A live image is displayed in the image display area 71 . Illumination control unit 64 illuminates all four light source areas 140A-140D for live images. While viewing the live image, the user moves the mounting table 30 in the X, Y, Z, and .theta.

In S2, the CPU 61 uses the detection section 68 to determine whether or not the movement of the mounting table 30 has ended. For example, when the input of the movement instruction by the mouse 4 or the console section 3 is completed and a certain period of time elapses, or when an explicit imaging instruction is input, the CPU 61 (detection section 68) detects the placement table 30. It is determined that the movement of has ended. When the CPU 61 determines that the movement of the mounting table 30 has ended, the process proceeds to S3.

In S3, the CPU 61 displays a live image in the image display area 71 when the movement ends. If the image of the observation target W is not displayed in the display area during the period from S4 to S10, it will be difficult for the user to grasp the observation target W. Therefore, the CPU 61 may display the image of the observation target W immediately before the unevenness-enhanced image is generated in the image display area 71 as necessary while the unevenness-enhanced image is being generated.

In S4, the CPU 61 generates a plurality of first brightness images I11 to I1n with different focus positions for the first illumination direction. The lighting control unit 64 turns on one light source region corresponding to the first lighting direction among the light source regions 140A to 140D. The imaging control unit 62 causes the imaging unit 25 and the luminance image generation unit 31 to acquire the first luminance images I11 to I1n while gradually changing the focus position of the objective lens 23 through the Z-direction driving unit 28 .

In S5, the CPU 61 generates a plurality of second luminance images I21 to I2n with different focus positions for the second illumination direction. The illumination control unit 64 lights one light source area corresponding to the second illumination direction among the light source areas 140A to 140D. The imaging control unit 62 causes the imaging unit 25 and the luminance image generation unit 31 to acquire the second luminance images I21 to I2n while gradually changing the focus position of the objective lens 23 through the Z-direction driving unit 28 .

In S6, the CPU 61 generates a first depth image I1a for the first illumination direction by subjecting the first luminance images I11 to I1n to focus stacking using the focus stacking unit 33. FIG.

In S7, the CPU 61 generates a second depth image I2a for the second illumination direction by subjecting the second brightness images I21 to I2n to focus stacking using the focus stacking unit 33. FIG.

In S8, the CPU 61 uses the unevenness enhancement unit 34 to generate an unevenness-enhanced image based on the first depth image I1a and the second depth image I2a.

In S9, the CPU 61 uses the coloring section 36 or the height image generating section 35 to color the unevenness emphasized image. As a result, the unevenness-enhanced image is given the color of the surface of the observation object W, or the unevenness-enhanced image is colored according to the height data acquired from the height image. Whether or not the coloring process is to be executed and what kind of coloring process is to be executed depend on which radio button is pressed in the coloring specifying section 76 .

In S<b>10 , the CPU 61 displays the unevenness emphasized image on the display section 2 through the UI section 65 and the display control section 63 . In other words, instead of the live image of the observed object W, the unevenness-enhanced image is displayed in the image display area 71 . In the user interface 70 shown in FIG. 6 , the image display area 71 displays a live image of the observation target W, and the image display area 75 displays a depth-composite unevenness-enhanced image.

In S11, the CPU 61 determines whether or not an instruction to end the magnifying observation process has been input. When an end instruction is input from the mouse 4 or the like, the CPU 61 ends the magnifying observation process. If the end instruction has not been input, the CPU 61 proceeds to S12.

In S12, the CPU 61 uses the detection unit 68 to determine whether an instruction to move the mounting table 30 or an instruction to change the magnification is input from the mouse 4 or the like. If neither the movement instruction nor the magnification change instruction has been input, the CPU 61 returns to S11. If a movement instruction or magnification change instruction is input, the CPU 61 returns to S<b>1 and displays a live image of the observed object W on the display section 2 . After that, when the movement of the mounting table 30 stops, the CPU 61 executes again the generation of the unevenness emphasized image.

<Internal Configuration of Head Unit 22>
FIG. 8 shows the internal configuration of the head section 22. As shown in FIG. The revolver 21 has at least three mounts 50a, 50b, 50c. Objective lenses 23a, 23b and 23c are connected to the mounts 50a, 50b and 50c, respectively. The objective lens 23a is, for example, a low-magnification objective lens. An electric diaphragm 37a is provided at the pupil position of the objective lens 23a. The electric aperture 37a is a variable aperture that is driven by a motor or the like to change the amount of aperture. In this example, the objective lens 23a is arranged on the observation optical axis A1. A ring light 26 is provided around the barrel of the objective lens 23a. The objective lens 23b also has a motorized diaphragm 37b but does not have the ring illumination 26. FIG. Since the pupil position of the objective lens 23c exists outside the lens barrel, no diaphragm is provided inside the lens barrel. The electric aperture 37c is an aperture arranged between the objective lens 23 and the half mirror 38. As shown in FIG. The half mirror 38 is arranged on the observation optical axis A1 and guides the illumination light from the coaxial epi-illumination 27 to the objective lens 23 . The half mirror 38 guides the incident light from the objective lens 23 to the imaging section 25 via the imaging lens 41 and the like. A condensing lens 39 is arranged between the coaxial epi-illumination 27 and the half mirror 38 . A motorized diaphragm 37 d is arranged between the coaxial epi-illumination 27 and the condensing lens 39 . The electric diaphragms 37a to 37d are controlled by the imaging control section 62 of the CPU61.

The imaging lens group includes a low-magnification imaging lens 41 a, a high-magnification imaging lens 41 b, and a lens adapter 43 . The low-magnification imaging lens 41a and the high-magnification imaging lens 41b are held by a slide member 49a. The slide member 49a is slidably held with respect to the frame 40a. In this example, the slide member 49a is driven by the imaging lens driving section 42 to slide left and right in FIG. Note that left and right in FIG. 8 correspond to the front and rear of the magnifying observation apparatus 100 . Stop members 44a and 44b are provided at both ends of the frame 40a. As shown in FIG. 9, the slide member 49a stops in contact with the stop member 44b, thereby positioning the low-magnification imaging lens 41a on the observation optical axis A1. As shown in FIG. 10, the slide member 49a stops in contact with the stop member 44a, thereby positioning the high-magnification imaging lens 41b on the observation optical axis A1. The lens adapter 43 is held by the slide member 49b. The slide member 49b is slidably held with respect to the frame 40b. In this example, the slide member 49b is driven by the imaging lens driving section 42 to slide left and right in FIG. The lens adapter 43 is used in combination with the low-magnification imaging lens 41a. The lens adapter 43 is, for example, a lens that changes the magnification to double or the like. As shown in FIGS. 9 and 10, the lens adapter 43 is separated from the observation optical axis A1 by stopping the slide member 49b in contact with the stop member 44c. As shown in FIG. 11, the lens adapter 43 is positioned on the observation optical axis A1 by stopping the slide member 49b in contact with the stop member 44d.

FIG. 12 shows the low-magnification imaging lens 41a, the high-magnification imaging lens 41b, the frame 40a, and the stop members 44a and 44b when the head section 22 is not tilted. Illustration of the slide member 49a is omitted. FIG. 13 shows the low-magnification imaging lens 41a, the high-magnification imaging lens 41b, the frame 40a, and the stop members 44a and 44b when the head portion 22 is tilted. As can be seen from FIGS. 12 and 13, the sliding directions of the low-magnification imaging lens 41a and the high-magnification imaging lens 41b are parallel to the tilt axis. Therefore, even if the head portion 22 tilts, the low-magnification imaging lens 41a and the high-magnification imaging lens 41b are less likely to slide. The same applies to the lens adapter 43 and the slide member 49b.

<Objective lens>
14 and 15 show the relationship between the objective lens 23 and the mount 50. FIG. The objective lens 23 has a connecting portion 59 . The connecting portion 59 is fitted into the mount 50 to fix the objective lens 23 to the revolver 21 . As shown in FIG. 15, the connecting portion 59 has positioning holes 45 and electronic contacts 46 . As shown in FIG. 14 , the objective lens 23 is positioned with respect to the revolver 21 by inserting the positioning pin 51 provided on the revolver 21 into the positioning hole 45 . The electronic contact 46a is electrically connected to the electronic contact 46b on the revolver 21 side. The electronic contacts 46a, 46b include contacts for power supply, contacts for communication, and the like. The control board 47 receives control signals from the CPU 61 via electronic contacts 46a and 46b, and controls the motor 48 that drives the electric diaphragm 37a and the ring illumination 26 according to the control signals. The control board 47 may have a CPU, a communication circuit, a memory, and the like. The memory stores identification information of the objective lens 23 and the like.

<Coaxial epi-illumination>
FIG. 16 shows the arrangement of the light sources in the coaxial epi-illumination 27. As shown in FIG. The light source group 54a has a plurality of LEDs 53a, 53e, 53f, 53i, 53j, 53k, and 53q for irradiating the observed object W with illumination light from the first illumination direction. The light source group 54c has a plurality of LEDs 53c, 53e, 53g, 53h, 53m, 53n, and 53o for irradiating the observed object W with illumination light from the second illumination direction. The light source group 54b has a plurality of LEDs 53b, 53e, 53g, 53l, 53k, and 53m for irradiating the observed object W with illumination light from the third illumination direction. The light source group 54d has a plurality of LEDs 53d, 53e, 53h, 53i, 53o, 53p, and 53q for irradiating the observed object W with illumination light from the fourth illumination direction.

For example, only the LED 53e may be lit during high-magnification observation. When the illumination light is deflected from the first illumination direction using the high-magnification objective lens 23, only the LED 53a may be lit. When the illumination light is deflected from the second illumination direction using the high-magnification objective lens 23, only the LED 53c may be lit. When the illumination light is deflected from the first illumination direction using the objective lens 23 with medium magnification, the LEDs 53a and 53j may be turned on at the same time. When the illumination light is deflected from the second illumination direction using the objective lens 23 with medium magnification, the LEDs 53c and 53n may be turned on at the same time.

By the way, a low-magnification objective lens has a long working distance, and a high-magnification objective lens has a short working distance. Therefore, by adopting the ring illumination 26 for the low-magnification objective lens 23a, it is possible to irradiate the illumination light from an appropriate oblique angle and realize surface observation and unevenness observation of the observation object W. FIG. On the other hand, when the ring illumination 26 is adopted for the high-magnification objective lens 23, dark field observation is performed. Therefore, only the edge portion shines, making it difficult to obtain reflected light from the surface. Therefore, by applying the coaxial epi-illumination 27 to the high-magnification objective lens 23, surface observation of the observation object W can be realized.

As described with reference to FIG. 4, an unevenness-enhanced image can be obtained by polarizing the illumination light from two directions using the ring illumination 26 . Therefore, by polarizing the illumination light from two directions for the coaxial epi-illumination 27 as well, it becomes possible to obtain an unevenness-enhanced image even with the objective lens 23 of high magnification. Deflection using coaxial epi-illumination 27 may be used to produce a dehalated view image. Focus stacking using the coaxial epi-illumination 27 can also be realized.

<Ring lighting>
FIG. 17 shows the structure of the ring illumination 26 employed in the low-magnification objective lens 23. As shown in FIG. A plurality of LEDs 55 are mounted on a ring-shaped substrate. The illumination light output from the LED 55 is condensed by a condensing lens 56 and irradiated onto an observation object W via an illumination lens 57 . A diffusion plate 58 may be employed between the condenser lens 56 and the illumination lens 57 . The LED 55 functions as a primary light source, while the diffusion plate 58 functions as a secondary light source. The illumination lens 57 functions as a relay lens that relays the illumination light from the secondary light source. With these, the ring illumination 26 realizes illumination close to Koehler illumination. Since the illumination lens 57 has a curvature only in the radial direction, loss of light quantity may occur. Therefore, a toroidal lens may be employed as the condensing lens 56 in order to provide curvature in the radial direction. This reduces light loss.

FIG. 18 shows the structure of the ring illumination 26 employed in the objective lens 23 of high magnification. A plurality of LEDs 55 are mounted on a ring-shaped substrate. Illumination light output from the LED 55 is condensed by a condensing lens 56 and irradiated onto an observation object W via an illumination lens 57 and a diffusion plate 58 . In FIG. 18, the illumination lens 57 is composed of a mirror. The high-magnification objective lens 23 must efficiently concentrate the illumination light in a narrow field of view. In order to reduce the loss of the amount of light, it is required that the light from the LED 55 be irradiated to the observation object W as parallel light as much as possible. Therefore, the illumination light must be strongly refracted from the outer periphery of the objective lens 23 toward the viewing area. Therefore, as the illumination lens 57, a material with a high refractive index is adopted, or a reflective optical system is adopted. However, if such an optical component is employed, the light source may be reflected in the viewing area. A diffusion plate 58 is employed to solve this problem.

Thus, the ring illumination 26 is designed according to the NA (numerical aperture) of the objective lens 23 and the field of view. By providing one condensing lens 56 for each LED 55, light guide efficiency is improved.

<Automatic switching of lighting>
In the magnifying observation apparatus 100 having a plurality of types of lighting devices, it is necessary for the user to select the lighting device to be turned on. However, since an appropriate illumination device differs for each magnification of the objective lens 23, it was sometimes difficult for the user to select an appropriate illumination device.

FIG. 19 is a flow chart showing automatic switching of lighting. In S21, the CPU 61 determines whether or not an instruction to rotate the revolver 21 (magnification switching instruction) has been input from the console section 3 or the like. When the rotation instruction is input, the CPU 61 controls the revolver driving section 24 based on the rotation instruction to rotate the revolver 21, and proceeds to S22.

The CPU 61 acquires the identification information of the objective lens 23 in S22. For example, the CPU 61 may acquire the identification information held in the memory of the control board 47 by communicating with the control board 47 via the electronic contacts 46a and 46b. The identification information may be information indicating the model name of the objective lens 23, magnification information indicating the magnification of the objective lens 23, illumination information indicating the presence or absence or type of illumination device, and the like. The CPU 61 may acquire corresponding magnification information and illumination information from the storage unit 67 based on the model name of the objective lens 23 .

In S23, the CPU 61 selects illumination based on the identification information of the objective lens 23. FIG. For example, if the identification information directly or indirectly indicates that the objective lens 23 has the ring light 26, the CPU 61 selects the ring light 26 as the lighting device. If the identification information directly or indirectly indicates that the objective lens 23 does not have the ring illumination 26, the CPU 61 selects the coaxial epi-illumination 27 as the illumination device. Direct means that the identification information acquired from the objective lens 23 includes information indicating the presence or absence of the ring illumination 26 . Indirect means that the identification information acquired from the objective lens 23 does not include the presence/absence information indicating the presence/absence of the ring illumination 26, but the presence/absence information linked to the identification information acquired from the objective lens 23 is stored in the storage unit. 67 or the like.

In S24, the CPU 61 determines the light sources to be turned on based on the observation method selected by the user and the illumination light amount of each light source to be turned on at the same time. Observation methods may include, for example, surface observation and unevenness observation. Surface observation may also include a mode (halation removal) for reducing overexposure due to surface reflection.

When surface observation using the objective lens 23a is specified, the CPU 61 selects the ring illumination 26. FIG. Further, the CPU 61 selects lighting of the light source regions 140A to 140D at the same time, and determines the illumination light intensity of each of the light source regions 140A to 140D so that the total value of the illumination light intensity of the light source regions 140A to 140D becomes a predetermined light intensity. . When the user selects halation removal, the CPU 61 turns on the light source regions 140A to 140D one by one to generate four luminance images. The image processor 66 compares the luminance values of four pixels at the same position in the four luminance images, and uses the remaining three pixels excluding the pixel with the highest luminance value to calculate the luminance value of the pixel at that position ( Example: average value calculation). In this case, since the light source areas 140A to 140D are lit one by one, the CPU 61 determines the illumination light amount so that the illumination light amount of each of the light source areas 140A to 140D is a predetermined light amount. When unevenness observation is selected, the CPU 61 determines the illumination light intensity so that the illumination light intensity of each of the light source regions 140A and 140C (or the light source regions 140B and 140D) used for unevenness observation becomes a predetermined light intensity.

When the objective lenses 23b and 23c are selected, the CPU 61 selects the coaxial epi-illumination 27. FIG. In surface observation, when the objective lens 23 with the highest magnification is selected, the CPU 61 may select and light only the LED 53e. The CPU 61 controls the drive current flowing through the LED 53e so that the amount of illumination light from the LED 53e becomes a predetermined amount. When the high-magnification objective lens 23 is selected for unevenness observation, the CPU 61 selects and lights the pair of LEDs 53a and 53c (or the pair of LEDs 53b and 53d).

When surface observation is selected with the medium-magnification objective lens 23, the CPU 61 selects the LEDs 53a-53i and controls the drive currents of the LEDs 53a-53i so that the total light intensity of these LEDs 53a-53i becomes a predetermined light intensity. When unevenness observation is selected with the medium magnification objective lens 23, the CPU 61 selects and turns on the pair of LEDs 53a and 53c (or the pair of LEDs 53b and 53d). Since the LEDs 53a and 53c are not turned on at the same time, the CPU 61 controls the drive currents of the LEDs 53a and 53c so that the light intensity of each of the LEDs 53a and 53c becomes a predetermined light intensity.

When surface observation is selected with the low-magnification objective lens 23 that does not have the ring illumination 26, the CPU 61 selects the LEDs 53a to 53q and adjusts the drive currents of the LEDs 53a to 53q so that the total light intensity of these LEDs 53a to 53q becomes a predetermined light intensity. to control. When unevenness observation is selected with the objective lens 23 of low magnification, the CPU 61 selects and lights a pair of LEDs 53a, 53j and LEDs 53c, 53n (or a pair of LEDs 53b, 53l and LEDs 53d, 53p). . Since the LEDs 53a and 53j are lit at the same time, the CPU 61 controls the drive currents of the LEDs 53a and 53j so that the total light intensity of the LEDs 53a and 53j is a predetermined light intensity. Similarly, since the LEDs 53c and 53n are lit at the same time, the CPU 61 controls the driving currents of the LEDs 53c and 53n so that the total light intensity of the LEDs 53c and 53n becomes a predetermined light intensity.

Both the ring illumination 26 and the coaxial epi-illumination 27 may be turned on depending on the type of the objective lens 23 and the observation method.

<Summary>
As described with reference to FIG. 1 and the like, the head section 22 functions as an optical system including an objective lens and an imaging lens. The mounting table 30 is an example of an XY stage that can move relative to the optical system in at least the X direction and the Y direction. The detector 68 functions as a detector that detects relative movement of the XY stage with respect to the optical system. The ring illumination 26 functions as an illumination unit that irradiates the observation object W placed in the field of view of the optical system with illumination light from different directions. The imaging unit 25 receives light from the observed object W via an optical system and generates a luminance image of the observed object W. FIG. The Z-direction driving section 28 functions as a changing section that changes the focus position of the optical system along the optical axis A1 of the optical system. The control section 60 functions as a control section that controls the lighting section, the imaging section, and the changing section. The display unit 2 functions as a display unit that displays an observation image, which is an image of the observation object W. FIG. The image processor 66 (i) controls the illumination unit to irradiate the observation object with illumination light from the first illumination direction, and controls the changing unit and the imaging unit to produce a plurality of different images. A plurality of first luminance images are acquired by imaging the observed object at each of the in-focus positions, and (ii) the illumination unit is controlled to obtain a plurality of first luminance images with respect to the observed object in the first illumination direction. By irradiating the observation object with illumination light from a second illumination direction that is symmetrical with respect to the acquiring a plurality of second luminance images, and (iii) performing focus stacking and unevenness enhancement on the plurality of first luminance images and the plurality of second luminance images, thereby emphasizing the unevenness of the surface of the observation object, and imaging functions as an image generation unit that generates an unevenness-enhanced image with a wider depth of field than a single luminance image that can be acquired by the unit. The display unit 2 displays an unevenness-enhanced image as an observation image. As a result, a depth-composited unevenness-enhanced image is provided.

The display unit 2 displays a moving image (live image) of the observation object W acquired by the imaging unit 25 as an observation image while the detection unit 68 is detecting the relative movement of the XY stage. When the detection unit 68 detects that the relative movement of the XY stage has stopped, the display unit 2 displays, as an observation image, the unevenness-enhanced image generated by the image generation unit after the relative movement of the XY stage has stopped. This allows the user to move the XY stage while confirming the relationship between the visual field range of the imaging unit 25 and the observation object W. Further, when the XY stage stops, the user can immediately view the unevenness-enhanced image that has undergone focus stacking, so that the observation target W can be observed efficiently.

The detector 68 may be configured to further detect changes in the magnification of the optical system. The display unit 2 displays the moving image of the observation object W acquired by the imaging unit 25 as an observation image while the detection unit 68 is detecting the relative movement of the XY stage. Furthermore, when the detection unit 68 detects that the relative movement of the XY stage has stopped and the magnification of the optical system has changed, the display unit 2 displays the XY stage as an observation image that has stopped the relative movement and changed the magnification. An unevenness-enhanced image generated by the image generator may be displayed later.

As shown in FIGS. 5 and 6, the display unit 2 may have a display area for moving images and a display area for unevenness-enhanced images.

The detection unit 68 may detect that the manual or autofocus focus adjustment of the objective lens 23 has been completed. When the magnification of the objective lens 23 is changed, the imaging control section 62 controls the Z-direction driving section 28 to perform autofocus. Therefore, the unevenness-enhanced image may be generated using the fact that the magnification is changed and the focus adjustment of the objective lens 23 is completed as a trigger.

The UI unit 65 may further display another window for displaying the navigation image on the display unit 2 separately from the user interface 70 . A navigation image is, for example, an image of an observation object W acquired using an objective lens 23 with a low magnification. When the user designates an observation site in the navigation image with the pointer 74, the UI unit 65 converts the designated position in the navigation image into a position on the coordinates of the mounting table 30, and moves the mounting table 30 to that position. This makes it possible to easily fit the observation site within the field of view.

The focus stacking unit 33 performs focus stacking to generate a first focus stacking image by stacking a plurality of first luminance images, and to generate a second focus stacking image by stacking a plurality of second luminance images. functions as a department. The unevenness enhancement unit 34 functions as an enhanced image generation unit that creates an unevenness enhanced image based on the luminance difference between the first focus stacking image and the second focus stacking image.

The focus stacking unit 33 analyzes a plurality of pixels having the same pixel position in each of the plurality of first luminance images, and selects a pixel with the highest degree of focus among the plurality of pixels as a focused pixel at the pixel position. The selection may generate a first depth-stacked image consisting of in-focus pixels at each of the plurality of pixel locations. Note that the degree of focus may be obtained for each pixel region composed of a plurality of adjacent pixels. The focus stacking unit 33 analyzes a plurality of pixels having the same pixel position in each of the plurality of second luminance images, and selects a pixel with the highest degree of focus among the plurality of pixels as a focused pixel at the pixel position. The selection may generate a second depth-stacked image consisting of in-focus pixels at each of the plurality of pixel positions.

The mounting table 30 is an example of an XY stage on which the observation target W is mounted and which is movable in at least the X direction and the Y direction. The mounting table drive section 29 is an example of a drive section that drives the XY stage. As described in relation to S12, the CPU 61 may function as a detector that detects that the XY stage has stopped after moving. The CPU 61 and the image processor 66 may generate the unevenness-enhanced image again when the detection unit detects that the XY stage has stopped after moving. As a result, the user can save the trouble of explicitly instructing the generation of the unevenness-enhanced image.

The CPU 61 may be configured to display a moving image (live image) acquired by the imaging unit 25 on the display unit 2 when the CPU 61 detects that the XY stage is moving. When the CPU 61 detects that the XY stage has stopped, the CPU 61 may cause the image processor 66 to generate the unevenness-enhanced image again and cause the display unit 2 to display the unevenness-enhanced image.

The coloring unit 36 may function as a color synthesizing unit that acquires color information from the color image of the observed object W acquired by the imaging unit 25 and colorizes the unevenness-enhanced image. This will help the user to understand the relationship between unevenness and surface color.

When generating the first luminance image, the image processor 66 obtains a plurality of sub luminance images with different exposure times, and performs HDR processing on the plurality of sub luminance images to generate the first luminance image. good. Similarly, when generating the second luminance image, the image processor 66 obtains a plurality of sub luminance images with different exposure times, and HDR-processes the plurality of sub luminance images to generate the second luminance image. may By applying HDR processing in this way, it is possible to reduce blown-out highlights and blocked-up shadows, and obtain a depth-composite unevenness-enhanced image.

The height image generation unit 35 obtains the height of the surface of the observation object for each pixel by integrating each pixel of the unevenness-enhanced image, and generates a height image in which each pixel has the height. good. Furthermore, the height image generator 35 may generate a color height image by coloring each pixel of the height image according to the height of each pixel. This would make the crater illusion less likely to occur and would allow the user to more accurately distinguish between concave and convex shapes.

Mouse 4, console section 3 and CPU 61 may function as a selection section for selecting the lighting direction of the lighting section. The image processor 66 controls the illumination unit to irradiate the observation object W with illumination light from a third illumination direction different from the first illumination direction and the second illumination direction. A plurality of third brightness images may be acquired by controlling the imaging unit and capturing an image of the observed object at each of a plurality of different in-focus positions. The image processor 66 controls the illumination unit to irradiate the observation object with illumination light from a fourth illumination direction that is symmetrical with respect to the third illumination direction with respect to the optical axis with respect to the observation object, A plurality of fourth luminance images may be acquired by controlling the changing section and the imaging section to capture an image of the observed object at each of a plurality of different in-focus positions. Further, the image processor 66 converts the unevenness-enhanced image corresponding to the illumination direction selected by the selection unit from among the first illumination direction, the second illumination direction, the third illumination direction, and the fourth illumination direction to the plurality of first luminance images. , the plurality of second luminance images, the plurality of third luminance images, and the plurality of fourth luminance images may be generated using a plurality of luminance images corresponding to the illumination direction selected by the selector. The display unit 2 may display an unevenness-enhanced image corresponding to the illumination direction selected by the selection unit.

The unevenness enhancement unit 34 synthesizes a pair of luminance images having the same focus position from the plurality of first luminance images and the plurality of second luminance images based on the luminance difference, and synthesizes each of the plurality of focus positions. may generate a plurality of sub unevenness-enhanced images in which unevenness is emphasized. The focus stacking unit 33 may function as a focus stacking image generation unit that generates a depth-stacked unevenness-enhanced image by stacking a plurality of sub unevenness-enhanced images. In this way, the order of unevenness enhancement and focus stacking may be either first.

The illumination unit may be a ring illumination 26 arranged around the objective lens 23 . The illumination unit may be an illumination light source provided inside the head unit 22 that irradiates the observation object W with illumination light through the objective lens 23 .

As shown in FIG. 8, the objective lenses 23a to 23c, imaging lenses 41a and 41b, etc. form an optical system of the magnifying observation apparatus 100. As shown in FIG. The revolver 21 functions as a lens selector that alternatively places the first objective lens and the second objective lens on the observation optical path (observation optical axis A1) of the optical system. The ring illumination 26 is an example of a first illumination section that irradiates an observation object with illumination light without passing through the first objective lens. The coaxial epi-illumination 27 functions as a second illuminator that passes through the second objective lens and irradiates the object to be observed with illumination light. The light source region 140A of the ring illumination 26 functions as a first light source that irradiates the observation object with illumination light from the first illumination direction. The light source region 140D functions as a second light source that irradiates the observation object with illumination light from the second illumination direction. The LED 53a of the coaxial epi-illumination 27 functions as a third light source that irradiates the observation object with illumination light from the first illumination direction. The LED 53d functions as a fourth light source that irradiates the observation object with illumination light from the second illumination direction.

The CPU 61 selectively uses the first illumination section and the second illumination section according to the objective lens 23 arranged in the observation optical path. Since the illumination device is automatically selected according to the objective lens 23 arranged in the observation optical path, the convenience for the user using the magnifying observation device 100 is improved.

For example, when the first objective lens is arranged in the observation optical path, the CPU 61 turns on the first light source to irradiate the observation object with illumination light from the first illumination direction, and controls the imaging unit 25. Then, the first brightness image is acquired by capturing an image of the observed object. The CPU 61 turns on the second light source to irradiate the object to be observed with illumination light from the second illumination direction, controls the imaging unit 25 to image the object to be observed, and acquires the second luminance image. . The CPU 61 may use the image processor 66 to generate a first observation image based on the first luminance image and the second luminance image, and display the first observation image on the display section 2 . When the second objective lens is arranged in the observation optical path, the CPU 61 irradiates the observation object with illumination light from the first illumination direction by turning on the third light source, and controls the imaging unit 25. A third luminance image is acquired by imaging the observed object. The CPU 61 turns on the fourth light source to irradiate the object to be observed with illumination light from the second illumination direction, controls the imaging unit 25 to image the object to be observed, and obtains a fourth luminance image. . Image processor 66 generates a second observed image based on the third intensity image and the fourth intensity image.

When the first objective lens is arranged in the observation optical path, the CPU 61 turns on the first light source and the third light source to irradiate the observation object with the illumination light from the first illumination direction. You may acquire a 1st brightness|luminance image by controlling and imaging an observation target object. Further, the CPU 61 turns on the second light source and the fourth light source to irradiate the observation object with illumination light from the second illumination direction, and controls the imaging unit to capture an image of the observation object. A luminance image may be acquired. Image processor 66 may be configured to generate a first observed image based on such first intensity image and second intensity image. In this way, the polarized light from the ring illumination 26 and the polarized light from the coaxial epi-illumination 27 may be combined.

A half mirror 38 is arranged in the observation optical path and functions as a half mirror that guides the illumination light from the third light source and the illumination light from the fourth light source to the second objective lens. By adopting such a half mirror 38, the coaxial epi-illumination 27 may be realized. Here, the light source of the coaxial epi-illumination 27 may be provided inside the head portion 22 or may be provided outside the head portion 22 .

The CPU 61 (illumination control unit 64) controls the amount of light of the simultaneously lit light sources according to the number of light sources that are lit at the same time among the first light source, the second light source, the third light source, and the fourth light source. The amount of illumination light applied to the object may be maintained at a predetermined amount. Also, the third light source and the fourth light source may each have a plurality of lighting means (eg, LEDs, etc.). The CPU 61 may switch the illumination means to be lit in the third light source and the fourth light source according to the objective lens selected by the lens selection section.

Mounts 50a-50c, as shown in FIG. 8, are an example of a first mount to which a first objective lens is attached and a second mount to which a second objective lens is attached. The CPU 61 recognizes the first objective lens by acquiring the lens identification information from the first objective lens via the first mount, and acquires the lens identification information from the second objective lens via the second mount to recognize the first objective lens. Two objectives may be recognized. The CPU 61 functions as a lens recognition section. As shown in FIG. 19, the CPU 61 may control lighting/non-lighting of the first light source, the second light source, the third light source, and the fourth light source based on the lens identification information. The lens identification information of the first objective lens may include information indicating that the first illumination unit is arranged around the barrel of the first objective lens.

The image processor 66 may generate the bump-enhanced image according to the procedure described above. The unevenness-enhanced image may be an unevenness-enhanced image with a wide (deep) depth of field. The image processor 66 emphasizes the unevenness of the surface of the observation target by performing focus stacking and unevenness enhancement on the plurality of first luminance images and the plurality of second luminance images, and a single image that can be acquired by the imaging unit 25. An unevenness-enhanced image having a greater depth of field than the luminance image of 1 may be generated as the first observation image and the second observation image. In this way, the ring illumination 26 and the coaxial epi-illumination 27 may be used to generate an unevenness-enhanced image with a wide depth of field.

The illumination direction selection section 72 is an example of a direction selection section that selects the illumination direction in the observed image. The first illumination unit (eg, ring illumination 26) includes a fifth light source (light source region 140B) that irradiates the observation object with illumination light from the third illumination direction, and a fifth light source (light source region 140B) that irradiates the observation object with illumination light from the fourth illumination direction. A sixth light source (light source region 140D) that emits illumination light may also be provided. The second illumination unit (coaxial epi-illumination 27) includes a seventh light source (e.g., LED 53b) that irradiates the observation object with illumination light from the third illumination direction, and a seventh light source (eg, LED 53b) that illuminates the observation object from the fourth illumination direction. and an eighth light source (LED 53d) for emitting light. When the first objective lens is arranged in the observation optical path, the CPU 61 turns on the first light source to irradiate the observation object with illumination light from the first illumination direction, and controls the imaging unit to perform observation. A first brightness image is obtained by capturing an image of an object, a second light source is turned on to irradiate the observation object with illumination light from a second lighting direction, and an imaging unit is controlled to control the observation object. Acquiring a second luminance image by imaging, irradiating the observation object with illumination light from a third lighting direction by turning on the fifth light source, and controlling the imaging unit to image the observation object. acquires a fifth brightness image, turns on the sixth light source to irradiate the observation target with illumination light from the fourth illumination direction, controls the imaging unit to capture an image of the observation target, and obtains the sixth A luminance image may be acquired. Furthermore, when the second objective lens is arranged in the observation optical path, the CPU 61 turns on the third light source to irradiate the observation object with the illumination light from the first illumination direction, thereby controlling the imaging unit. A third luminance image is obtained by capturing an image of the observation target, the fourth light source is turned on to irradiate the observation target with illumination light from the second illumination direction, and the imaging unit is controlled to obtain the observation target. is captured to obtain a fourth luminance image, the seventh light source is turned on to irradiate the observation target with illumination light from the third illumination direction, and the imaging unit is controlled to capture the observation target. By turning on the eighth light source, the object to be observed is irradiated with illumination light from the fourth illumination direction, and the imaging unit is controlled to capture an image of the object to be observed. Eight intensity images may be acquired. The image processor 66 converts the observation image corresponding to the illumination direction selected by the direction selection unit from among the first illumination direction, the second illumination direction, the third illumination direction, and the fourth illumination direction into a first luminance image, a second luminance image, and a second luminance image. a plurality of luminance images corresponding to the illumination direction selected by the direction selection unit from among the image, the third luminance image, the fourth luminance image, the fifth luminance image, the sixth luminance image, the seventh luminance image, and the eighth luminance image; may be generated using

Claims (9)

A magnifying observation device for observing an observation object,
an optical system including a first objective lens, a second objective lens having a shorter working distance and higher magnification than the first objective lens , and an imaging lens;
a lens selector that alternatively arranges the first objective lens and the second objective lens in an observation optical path of the optical system;
an XY stage movable relative to the optical system in at least the X direction and the Y direction;
an illumination device that irradiates an observation object placed on the XY stage with illumination light from different directions;
an imaging unit that receives light from the observation target through the optical system and generates a luminance image of the observation target;
A control unit that controls the lighting device and the imaging unit ,
The lighting device
a first illumination unit that irradiates the observation object with illumination light that passes around the first objective lens without passing through the first objective lens ;
a second illumination unit that irradiates the observed object with illumination light passing through the second objective lens;
has
The first lighting unit is
a first light source that irradiates the observation object with the illumination light from a first illumination direction;
a second light source that irradiates the observation object with the illumination light from a second illumination direction;
has
The second lighting unit is
a third light source that irradiates the observation object with the illumination light from the first illumination direction;
a fourth light source that irradiates the observation object with the illumination light from the second illumination direction;
has
The control unit controls the illumination device to selectively use the first illumination unit and the second illumination unit according to the objective lens arranged in the observation optical path,
When the first objective lens is arranged in the observation optical path, the control unit turns on the first light source to direct illumination light passing around the first objective lens from the first illumination direction. irradiating an observation object, receiving illumination light from the observation object through the first objective lens by the imaging unit to generate a first luminance image of the observation object, and the second light source irradiates the observation object from the second illumination direction with illumination light passing around the first objective lens by lighting the observation object through the first objective lens by the imaging unit controlling the lighting device and the imaging unit to receive illumination light from the observation object and generate a second luminance image of the observation object;
When the second objective lens is arranged in the observation optical path, the control unit turns on the third light source to illuminate the illumination light passing through the second objective lens from the first illumination direction. illuminating an object, receiving illumination light from the observation object through the second objective lens by the imaging unit to generate a third luminance image of the observation object, and using the fourth light source By turning on, illumination light passing through the second objective lens is applied to the observation object from the second illumination direction, and the imaging unit emits illumination from the observation object through the second objective lens. controlling the lighting device and the imaging unit to receive light and generate a fourth luminance image of the observed object;
The magnifying observation device further generates a first observation image based on the first luminance image and the second luminance image when the first objective lens is arranged in the observation optical path, and A magnifying observation apparatus, comprising: an image generation section that generates a second observation image based on the third luminance image and the fourth luminance image when a lens is arranged in the observation optical path. When the first objective lens is arranged in the observation optical path, the image generation unit turns on the first light source and the third light source to emit the illumination light from the first illumination direction to the observation target. The second illumination is obtained by irradiating an object, controlling the imaging unit to capture an image of the observation target, and acquiring the first luminance image, and turning on the second light source and the fourth light source. The observation object is irradiated with the illumination light from a direction, and the imaging unit is controlled to capture an image of the observation object to obtain the second luminance image, and the first luminance image and the second luminance image are obtained. 2. A magnifying observation apparatus according to claim 1 , wherein said first observation image is generated based on a luminance image. 3. The apparatus according to claim 1, further comprising a half mirror arranged in said observation optical path and guiding illumination light from said third light source and illumination light from said fourth light source to said second objective lens. A magnifying observation device. The control unit controls the amount of light of the simultaneously lit light sources according to the number of light sources that are simultaneously lit among the first light source, the second light source, the third light source, and the fourth light source, thereby performing the observation. 4. The magnifying observation apparatus according to any one of claims 1 to 3 , wherein the amount of illumination light applied to the object is maintained at a predetermined amount. The third light source and the fourth light source each have a plurality of illumination means,
5. The enlargement according to any one of claims 1 to 4 , wherein illumination means to be lit in said third light source and said fourth light source are switched according to the objective lens selected by said lens selection unit. Observation device. The lens selection unit
a first mount to which the first objective lens is attached;
a second mount to which the second objective lens is attached;
has
The control unit
recognizing the first objective lens by acquiring lens identification information from the first objective lens via the first mount;
recognizing the second objective lens by acquiring lens identification information from the second objective lens via the second mount;
6. The method according to any one of claims 1 to 5 , wherein lighting/non-lighting of the first light source, the second light source, the third light source, and the fourth light source is controlled based on the lens identification information. magnifying observation device. 7. The enlargement according to claim 6 , wherein the lens identification information of the first objective lens includes information indicating that the first illumination unit is arranged around the barrel of the first objective lens. Observation device. The image generator is
By controlling the illumination device, the observation object is irradiated with the illumination light from the first illumination direction, and the imaging unit is controlled to set a plurality of different in-focus positions. Acquiring a plurality of first luminance images by respectively imaging the observation object,
By controlling the illumination device, the illumination light is directed toward the observation object from the second illumination direction that is symmetrical with respect to the observation object with respect to the first illumination direction with respect to the optical axis of the optical system. acquire a plurality of second brightness images by irradiating the object to be observed, and controlling the imaging unit to image the observation object at each of a plurality of different in-focus positions;
The unevenness of the surface of the observation object is emphasized by performing focus stacking and unevenness enhancement on the plurality of first brightness images and the plurality of second brightness images, and a single brightness that can be acquired by the imaging unit. 2. The magnifying observation apparatus according to claim 1 , wherein an unevenness-enhanced image having a greater depth of field than the image is generated as the first observation image and the second observation image. further comprising a direction selection unit that selects an illumination direction in the observed image;
The first lighting unit is
a fifth light source that irradiates the observation object with the illumination light from a third illumination direction;
a sixth light source that irradiates the observation object with the illumination light from a fourth illumination direction;
has
The second lighting unit is
a seventh light source that irradiates the observation object with the illumination light from the third illumination direction;
an eighth light source that irradiates the observation object with the illumination light from the fourth illumination direction;
has
The control unit
When the first objective lens is arranged in the observation optical path, the first light source is turned on to irradiate the observation object with the illumination light from the first illumination direction, thereby causing the imaging unit to operate. acquiring the first brightness image by controlling and imaging the observation object, and irradiating the observation object with the illumination light from the second illumination direction by turning on the second light source; obtaining the second brightness image by controlling the imaging unit to capture an image of the observation object, and turning on the fifth light source to illuminate the observation object from the third illumination direction; and control the imaging unit to capture an image of the observation target to acquire a fifth luminance image, and turn on the sixth light source to illuminate the observation target from the fourth illumination direction. acquire a sixth brightness image by irradiating an object and controlling the imaging unit to capture an image of the observation target;
When the second objective lens is arranged in the observation optical path, the third light source is turned on to irradiate the observation object with the illumination light from the first illumination direction, thereby causing the imaging unit to operate. acquiring the third luminance image by controlling and imaging the observation object, and irradiating the observation object with the illumination light from the second illumination direction by turning on the fourth light source; obtaining the fourth brightness image by controlling the imaging unit to capture an image of the observation object, and turning on the seventh light source to illuminate the observation object from the third illumination direction; and controls the imaging unit to capture an image of the observation target to acquire a seventh luminance image, and turns on the eighth light source to illuminate the observation target from the fourth illumination direction. Acquiring an eighth brightness image by irradiating an object and controlling the imaging unit to capture an image of the observation target,
The image generation unit further generates an observation image corresponding to the illumination direction selected by the direction selection unit from among the first illumination direction, the second illumination direction, the third illumination direction, and the fourth illumination direction, of the first luminance image, the second luminance image, the third luminance image, the fourth luminance image, the fifth luminance image, the sixth luminance image, the seventh luminance image, and the eighth luminance image 9. The magnifying observation apparatus according to claim 1, wherein the magnifying observation apparatus is generated using a plurality of luminance images corresponding to the illumination direction selected by the direction selecting section.

Images