JP7052377B2 - Inspection equipment and inspection method - Google Patents

Description

The present invention relates to an inspection device and an inspection method for inspecting an inspection object using a neural network.

Patent Document 1 describes a method of inspecting a semiconductor wafer by using a neural network.

Japanese Unexamined Patent Publication No. 2004-354251

In the inspection process using a neural network, an inspection object such as a semiconductor wafer is visually enlarged using a microscope or the like, and then an input image obtained by dividing the inspection image obtained by imaging the inspection object into a plurality of parts is input to the neural network. do.

However, even when a defect cause such as a foreign substance is present in the input image, it may or may not be determined to be defective depending on the position of the defect cause in the inspection object. In the prior art, the input image input to the neural network does not include information on the position of the cause of the defect in the inspection object, so that there is a possibility that accurate inspection cannot be performed. Such a problem is not limited to the semiconductor wafer, but is a common problem when performing an inspection by a neural network using an input image obtained by cutting out a part of an inspection image showing an inspection target.

The present invention has been made to solve at least a part of the above-mentioned problems, and can be realized as the following aspect.

(1) According to the first aspect of the present invention, there is provided an inspection device for inspecting an inspection object using a neural network. This inspection device has a cutout portion that cuts out a part of an inspection image obtained by capturing an image of the inspection object as an input image, and an inspection mark that affects the inspection result of the input image of the input image in the inspection image. The embedded portion to be embedded in the input image according to the cutting position and the input image in which the inspection mark is embedded are input to the neural network, and the inspection result of the input image is obtained using the output of the neural network. It is equipped with an inspection unit for determining.
According to this inspection device, it is possible to perform an inspection according to the cutout position of the input image in the inspection image.

(2) In the inspection device, the neural network may be a convolutional neural network.
According to this inspection device, it is possible to perform an inspection according to the cutout position of the input image in the inspection image.

(3) In the inspection device, the inspection mark may include a region mark for distinguishing a specific region in the inspection image from other regions.
According to this inspection device, it is possible to perform an inspection according to the cutout position of the input image in the inspection image by using the area mark.

(4) In the inspection device, the inspection mark may include a size mark for distinguishing the size of the cause of defect.
According to this inspection device, it is possible to perform an inspection according to the cutout position of the input image in the inspection image by using the size mark.

(5) In the inspection device, the inspection mark may include a shape mark for distinguishing the shape of the cause of the defect.
According to this inspection device, it is possible to perform an inspection according to the cutout position of the input image in the inspection image by using the shape mark.

The present invention can also be realized in various forms other than the above. For example, it can be realized in the form of an inspection method for inspecting an inspection object, a computer program for realizing the function of the inspection method, a non-transitory storage medium on which the computer program is recorded, or the like. ..

The block diagram which shows the structure of the inspection system in one Embodiment. The figure which shows the flowchart at the time of learning. The figure explaining the relationship between the inspection image and the input image. The figure for demonstrating the embedding of the area mark in the input image. The figure for demonstrating the embedding of the area mark in the input image. A conceptual diagram for explaining CNN. The figure which shows the flowchart at the time of inspection. Diagram to explain the size mark. The figure for demonstrating the shape mark.

A. 1st Embodiment FIG. 1 is a block diagram showing a configuration of an inspection system 10 including an inspection apparatus 300 in one embodiment. The inspection system 10 includes an image pickup apparatus 100, an inspection image database 200, an inspection apparatus 300, and an input image database 400. In the inspection of this embodiment, a semiconductor wafer having a plurality of chips arranged in a grid pattern is used. Specifically, a chip is used as an inspection target of the present embodiment. The plurality of chips used in this embodiment all have the same structure.

The image pickup device 100 is a device that takes an image of an object to be inspected. In the present embodiment, the image pickup apparatus 100 includes a camera that captures an object, a microscope that visually magnifies the object, and a lighting device that irradiates the object with light. In the present embodiment, the image pickup apparatus 100 captures an image of an inspection object magnified by a microscope. The image of the image to be inspected is called an inspection image. The inspection image captured by the image pickup apparatus 100 is stored in the inspection image database 200.

The inspection device 300 is a device that inspects an object to be inspected. The inspection device 300 inspects the inspection target using the neural network. In this embodiment, a convolutional neural network (CNN) is used as the neural network, but other types of neural networks may be used.

The inspection device 300 includes an information input / output unit 310 that inputs / outputs information to / from the user of the inspection device 300, a preprocessing unit 320 that performs image preprocessing, and a learning unit 330 that learns a neural network. It also includes an inspection unit 340 that inspects an inspection object using a trained neural network. The image processed by the preprocessing unit 320 is stored in the input image database 400.

FIG. 2 is a diagram showing a flowchart during learning. This process is executed by receiving information from the user to the effect that learning is started by the information input / output unit 310.

First, the preprocessing unit 320 of the inspection device 300 cuts out a part of the inspection image stored in the inspection image database 200 as an input image (step S110). The step S110 is performed by the cutting section 322 (see FIG. 1) of the pretreatment section 320.

FIG. 3 is a diagram illustrating the relationship between the inspection image D1 and the input image D2. In the present embodiment, each of the plurality of chips Ch arranged in a grid pattern on the semiconductor wafer W is used as an inspection target. In the present embodiment, the image captured by the image pickup apparatus 100 for each chip Ch is the inspection image D1, and the image cut out by the cutout portion 322 as a part of the inspection image is the input image D2. Information on the cutout position in the inspection image D1 is attached to each of the input images D2. In the present embodiment, the adjacent input images D2 are cut out so that there is no overlapping portion with each other, but there may be a portion overlapping with each other. Further, after cutting out the input image D2, data compression, normalization, data expansion, and the like may be performed. Here, data expansion means increasing the number of input images for learning by performing processing such as rotation and inversion on the input images. In the present embodiment, 100 input images D2 are cut out from one inspection image D1.

Further, the input image D2 at the time of learning is attached with an inspection result label for each input image D2. Here, the inspection result label is a label indicating the inspection result of each input image D2. This inspection result is the result of a visual inspection by the inspector in advance. In this embodiment, a non-defective product and a defective foreign substance are used as the inspection result label.

After cutting out the input image D2 (step S110 (see FIG. 2)), the preprocessing unit 320 sets an inspection mark for influencing the inspection result of the input image D2 at the cutout position of the input image D2 in the inspection image D1. Is embedded in the input image D2 according to the above (step S120). In the present embodiment, as the inspection mark, a region mark that distinguishes a specific region in the inspection image D1 from other regions is used. The step S120 is performed by the embedded unit 325 (see FIG. 1) of the pretreatment unit 320.

In the present embodiment, the information input / output unit 310 acquires the position where the area mark is embedded from the user of the inspection device 300. Specifically, the user of the inspection device 300 inputs a specific area of the inspection image D1 to the information input / output unit 310, and the embedding unit 325 responds to the cutout position of the input image D2 in the inspection image D1. The input image is processed so that the specific input area can be distinguished from other areas.

4 and 5 are diagrams for explaining the embedding of the region mark RM in the input image D2. In FIG. 4, in the present embodiment, the specific region in which the region mark RM is embedded is defined as the region R1 to be inspected as "foreign matter defect" or "deficiency" when a defect cause such as a foreign substance is present. That is, the region R1 is distinguished from other regions R2 that are inspected as "good products" even in the presence of foreign matter. In the present embodiment, the embedded unit 325 of the pretreatment unit 320 embeds the area mark RM by performing a luminance conversion in the area R1. Specifically, the embedded portion 325 sets the brightness in the region R1 to be smaller than that in the other regions R2. The method of processing the input image D2 is not limited to the luminance conversion, and for example, the shade of a specific color may be changed, or a pattern image prepared in advance such as a dot pattern may be embedded. ..

FIG. 5 is a diagram illustrating the difference between the input image D2 in which the area mark RM is not embedded and the input image D2 in which the area mark RM is embedded. The input images D2a, D2b, and D2c of FIG. 5 correspond to the input images D2a, D2b, and D2c of FIG. In FIGS. 4 and 5, foreign substances Ca, Cb, and Cc are present at the centers of the input images D2a, D2b, and D2c, respectively. The shapes of the foreign substances Ca, Cb, and Cc are the same.

In such a case, as shown in the upper part of FIG. 5, when the area mark is not embedded in the input image D2, the position of the foreign matter in the input image D2 is different even if the position of the foreign matter in the inspection image D1 is different. The same inspection result is obtained for the input images D2a, D2b, and D2c having the same value.

On the other hand, as shown in the lower part of FIG. 5, when the area mark RM is embedded in the input image D2, even if the position of the foreign matter in the input image D2 is the same, the input images D2a, D2b, are displayed by the area mark RM. D2c is a different image. As a result, in the learning step (step S140) described later, the learning unit 330 can learn the characteristics of the region mark RM and the foreign matter combined, so that the input image D2c in which the foreign matter Cc is present in the region mark RM is defective. You can learn correctly if it is. The area mark RM may be embedded in another area R2 instead of the area R1.

After the area mark RM is embedded in the input image D2 (step S120 (see FIG. 2)), the preprocessing unit 320 stores a learning data set including a plurality of input images in the input image database 400 (step S130). ).

After that, the learning unit 330 learns the CNN using the learning data set (step S140).

FIG. 6 is a conceptual diagram for explaining CNN. The CNN includes an input layer LI, a convolutional layer LC, a pooling layer LP, a fully connected layer LF, and an output layer LO. In the present embodiment, the convolutional layer LC and the pooling layer LP are repeated a total of ten or more times. Then, the CNN outputs the score of each classification of the inspection result from the input image D2. In the present embodiment, good products and defective foreign substances are used as the classification of inspection results, but other classifications such as defective shapes may be used. In the present embodiment, each score of the classification is output from the output layer LO so that the total of the scores of the classification is 1. The learning by the learning unit 330 is performed by matching the score output from the output layer LO with the correct answer obtained by the test performed in advance.

After the learning is completed (process S140), the learning unit 330 stores the learned CNN in the input image database 400 (process S150), and ends the process at the time of learning. The learning unit 330 may store the parameters of the learned CNN in the input image database 400 instead of the learned CNN. Next, the time of inspection will be described.

FIG. 7 is a diagram showing a flowchart at the time of inspection. This process is executed by receiving information from the user to the effect that the inspection is started by the information input / output unit 310.

First, the preprocessing unit 320 cuts out a part of the inspection image D1 stored in the inspection image database 200 as the input image D2 (step S210), as in the case of learning. The step S210 is performed by the cutting section 322 of the pretreatment section 320. The input image at the time of learning has an inspection result label attached to each input image, but the input image D2 at the time of inspection is different in that the inspection result label is not attached, but is the same except for the above. Is.

After the input image D2 is cut out (step S210), the preprocessing unit 320 embeds an inspection mark in the input image according to the cutout position of the input image in the inspection image D1 (step S220). In this embodiment, the area mark RM is used as the inspection mark. In addition, the position where the inspection mark is embedded is the same as the position where the inspection mark is embedded at the time of learning. In the present embodiment, as the method of embedding the area mark RM, the luminance conversion under the same conditions as at the time of learning is used. The step S220 is performed by the embedded unit 325 of the pretreatment unit 320.

After the inspection mark is embedded in the input image D2 (process S220), the inspection unit 340 reads the learned CNN created by the learning unit 330 from the input image database 400 (process S240). If the parameters of the trained CNN are stored in the input image database 400 instead of the trained CNN, the inspection unit 340 reads the parameters from the input image database 400. After that, the inspection unit 340 inputs the input image D2 in which the inspection mark is embedded into the trained neural network, and determines the inspection result of the input image using the output of the neural network (step S250).

In the present embodiment, the inspection unit 340 determines the inspection result of the input image D2 by using the inspection result having the highest score obtained by the CNN. For example, if the score that the inspection result is good is 0.8 and the score that the inspection result is defective foreign matter is 0.2, the inspection unit 340 considers the inspection result of the input image D2 to be good. ..

After the inspection is completed (process S240), the inspection unit 340 outputs the inspection result by the information input / output unit 310 (process S250), and the processing at the time of inspection is completed. In the present embodiment, when an inspection result of a foreign matter defect is obtained in at least one input image D2 of the input image D2 constituting the inspection image D1 as an output of the inspection result, the information input / output unit 310 sets the inspection image D1. The inspection result indicating that the inspection object shown in is defective in foreign matter is output. On the other hand, when an inspection result of a non-defective product is obtained in all the input images D2 constituting the inspection image D1, the information input / output unit 310 outputs an inspection result indicating that the inspection object shown in the inspection image is a non-defective product. Output.

As described above, according to the inspection device 300 of the present embodiment, the inspection mark is embedded in the input image according to the cutout position of the input image D2 in the inspection image D1 according to the position of the input image in the inspection image. Can be inspected. Therefore, according to the inspection device 300 of the present embodiment, even if the input images D2 are indistinguishable from each other in appearance, the inspection can be performed according to the cutout position of the input image in the inspection image D1.

B. 2nd Embodiment The 2nd embodiment is different from the 1st embodiment in the inspection mark, but is the same except for the above. In the first embodiment, the area mark RM is used as the inspection mark, but in the second embodiment, the size mark for distinguishing the size of the cause of the defect is used as the inspection mark.

FIG. 8 is a diagram for explaining the size mark. In the second embodiment, the size mark SM1 or SM2 is embedded in all the input images D2. In the present embodiment, a circle is embedded in the input image D2 as the size marks SM1 and SM2, but other shapes such as a triangle or a quadrangle may be used, and parallel lines separated by a predetermined distance may be used. In the present embodiment, the diameter of the size mark SM2 in the strict region strictly inspected as a foreign matter defect is smaller than the diameter of the size mark SM2 in the region other than the strict region. The strict area is input from the information input / output unit 310 by the user of the inspection device 300. In the present embodiment, the input image D2 in which a foreign matter larger than the circle having the size mark SM1 or SM2 is present is inspected as a foreign matter defect, and the other input image D2 is inspected as a good product. In the present embodiment, the size mark SM1 or SM2 is embedded in the center of the input image D2, but the present invention is not limited to this, and for example, the size mark SM1 or SM2 may be embedded in the lower right of the input image D2. The position where the size mark SM1 or SM2 is embedded may be different for each input image D2.

In FIG. 8, the input images D2d to D2i with the size marks SM1 or SM2 are shown. The foreign matter existing in the input image D2d and the input image D2g has the same shape, and the inspection results are both the same non-defective product. Similarly, the foreign matter existing in the input image D2e and the input image D2h has the same shape, and the inspection results are all the same foreign matter defects. On the other hand, although the foreign matter existing in the input image D2f and the input image D2i has the same shape, the inspection result of the input image D2f is a good product, while the inspection result of the input image D2i is a foreign matter defect. Even in such a case, according to the second embodiment, the learning unit 330 can learn the characteristics of the size mark SM1 or SM2 and the cause of the defect such as a foreign substance. Therefore, according to the second embodiment, by using the size mark SM1 or SM2, it is possible to perform an inspection according to the cutout position of the input image D2 in the inspection image D1.

C. Other Embodiments In the above-described embodiment, the area mark and the size mark are used as the inspection marks, but the present invention is not limited thereto. For example, as the inspection mark, a shape mark for distinguishing the shape of the cause of the defect may be used. In the present embodiment, the shape of the wiring provided on the chip is used as the cause of the defect, but the shape is not limited to this, and for example, the shape of the etched portion, the shape of the insulating film, or the like may be used. In the present embodiment, an inspection mark is embedded in a region where the shape of the wiring is inspected as a shape defect, and this region is input from the information input / output unit 310 by the user of the inspection device 300.

FIG. 9 is a diagram for explaining the shape mark. In the input images D2j and D2k of FIG. 9, the wiring shapes P1 and P2 exist, respectively, and the shape mark KM is embedded. In the present embodiment, when the shape of the wiring is the same as the shape mark, the inspection result is a good product, and when the wiring pattern is different from the shape mark KM, the inspection result is a poor shape. Since the wiring shape P1 in the input image D2j is the same as the shape mark KM, the inspection result of the input image D2j is a good product. On the other hand, since the shape P2 of the wiring in the input image D2k is different from the shape mark KM in the upper right corner of the paper, the inspection result of the input image D2k is a poor shape. By embedding the shape mark KM in the input image D2 in this way, according to this embodiment, the shape mark KM can be used to perform an inspection according to the cutout position of the input image D2 in the inspection image D1.

In the above-described embodiment, any one of the area mark, the size mark, and the shape mark has been used as the inspection mark, but the description is not limited to this. For example, two or more marks may be used in combination.

In the above embodiment, after the input image D2 is cut out from the inspection image D1, the inspection mark is embedded in the input image D2, but the present invention is not limited to this, and after the inspection mark is embedded in the inspection image D1, the inspection mark is embedded. The input image D2 may be cut out from the inspection image D1. By doing so, the inspection mark can be embedded in the input image D2 even if the input image D2 does not have the information of the cutout position in the inspection image D1.

The present invention is not limited to the above-described embodiments and examples, and can be realized with various configurations within a range not deviating from the gist thereof. For example, the embodiment corresponding to the technical feature in each embodiment described in the column of the outline of the invention, the technical feature in the embodiment may be used to solve a part or all of the above-mentioned problems, or the above-mentioned above. It is possible to replace or combine them as appropriate to achieve some or all of the effects. Further, if the technical feature is not described as essential in the present specification, it can be appropriately deleted.

10 ... Inspection system, 100 ... Imaging device, 200 ... Inspection image database, 300 ... Inspection device, 310 ... Information input / output section, 320 ... Preprocessing section, 322 ... Cutting section, 325 ... Embedded section, 330 ... Learning Unit, 340 ... Inspection unit, 400 ... Database for input image, Ca-Cc ... Foreign matter, Ch ... Chip, D1 ... Inspection image, D2, D2a-D2i ... Input image, KM ... Shape mark, LC ... Folding layer, LF ... Fully coupled layer, LI ... Input layer, LO ... Output layer, LP ... Pooling layer, P1, P2 ... Shape, R1, R2 ... Region, RM ... Region mark, SM1, SM2 ... Size mark, W ... Semiconductor wafer

Claims (6)

It is an inspection device that inspects an inspection object using a neural network whose parameters have been learned in advance by matching the score output using the training data set with the correct answer .
A cutout portion that cuts out a part of the inspection image obtained by imaging the inspection object as an input image, and
An embedded portion for embedding an inspection mark that affects the inspection result of the input image in the input image according to the cutout position of the input image in the inspection image.
The input image in which the inspection mark is embedded is input to the neural network, and an inspection unit for determining an inspection result of the input image by using the output of the neural network is provided.
The inspection mark is an inspection device including a size mark for distinguishing the size of the cause of defect . It is an inspection device that inspects an inspection object using a neural network whose parameters have been learned in advance by matching the score output using the training data set with the correct answer.
A cutout portion that cuts out a part of the inspection image obtained by imaging the inspection object as an input image, and
An embedded portion for embedding an inspection mark that affects the inspection result of the input image in the input image according to the cutout position of the input image in the inspection image.
The input image in which the inspection mark is embedded is input to the neural network, and an inspection unit for determining an inspection result of the input image by using the output of the neural network is provided.
The inspection mark is an inspection device including a shape mark for distinguishing the shape of the cause of defect. The inspection device according to claim 1 or 2 .
The neural network is an inspection device that is a convolutional neural network. The inspection device according to any one of claims 1 to 3 .
The inspection mark is an inspection device including an area mark for distinguishing a specific area in the inspection image from other areas. It is an inspection method that inspects an inspection object using a neural network whose parameters have been learned in advance by matching the score output using the training data set with the correct answer .
A process of cutting out a part of an inspection image obtained by capturing an image of the inspection object as an input image, and
A step of embedding an inspection mark that affects the inspection result of the input image in the input image according to a cutout position of the input image in the inspection image.
An inspection method comprising a step of inputting the input image in which the inspection mark is embedded into the neural network and determining an inspection result of the input image using the output of the neural network.
The inspection mark is an inspection method including a size mark for distinguishing the size of the cause of defect. It is an inspection method that inspects an inspection object using a neural network whose parameters have been learned in advance by matching the score output using the training data set with the correct answer.
A process of cutting out a part of an inspection image obtained by capturing an image of the inspection object as an input image, and
A step of embedding an inspection mark that affects the inspection result of the input image in the input image according to a cutout position of the input image in the inspection image.
An inspection method comprising a step of inputting the input image in which the inspection mark is embedded into the neural network and determining an inspection result of the input image using the output of the neural network.
The inspection mark is an inspection method including a shape mark for distinguishing the shape of the cause of defect.

Images