JP7465446B2 - Image inspection device, image inspection method, and trained model generation device - Google Patents

Description

The present invention relates to an image inspection device, an image inspection method, and a trained model generation device.

Conventionally, image inspection devices are known that inspect an object based on an image of the object.

For example, Patent Document 1 describes an anomaly determination device that performs anomaly determination based on input image data to be determined, which has a process execution means that generates reconstructed image data from the features of the image data to be determined using reconstruction parameters for reconstructing normal image data from features extracted from a group of normal image data, and executes anomaly determination processing to perform anomaly determination based on difference information between the generated reconstructed image data and the image data to be determined. When the image data to be determined includes image data of multiple channels, the anomaly determination device of Patent Document 1 generates reconstructed image data for each channel from the features of the image data of each channel using reconstruction parameters, and performs anomaly determination based on difference information between each generated reconstructed image data and the image data of each channel of the image data to be determined.

JP 2018-5773 A

In Patent Document 1, a trained autoencoder, which is a trained model, is used to generate a reconstructed image from an image to be judged. Here, for example, when a special pattern exists locally in an image of a good inspection object, if the trained model has low expressive power, it may not be possible to reconstruct the special pattern in the image generated by the trained model. In this case, there is a risk that the image of the good inspection object may be erroneously judged to be defective.

In addition, if an image of a good inspection object shows a pattern that is good in one location or part but defective in another location or part, the image generated by the trained model may show a defective pattern, causing the defective inspection object to be overlooked.

Therefore, one of the objectives of the present invention is to provide an image inspection device, an image inspection method, and a trained model generation device that can restore special patterns and suppress the generation of defective patterns.

An image inspection device according to one aspect of the present disclosure includes an image generation unit that inputs inspection split images, which are split images of the inspection object, to multiple trained models that have been trained to output restored split images using one or more good-quality split images, which are split images of a good-quality inspection object, as input, and generates multiple restored split images, and an inspection unit that inspects the inspection object based on the multiple restored split images.

According to this aspect, the inspection split image is input to a plurality of trained models, each of which has been trained to output a restored split image using one or more good-quality split images as input, to generate a plurality of restored split images. This allows each trained model to learn a pattern specific to each good-quality split image, so that a local special pattern contained in the good-quality split image can be restored in the generated restored split image. Also, even if a pattern that is good at one position or part is defective at another position or part in an image of another good-quality inspection object, each trained model can learn a judgment criterion that differs depending on the position in the good-quality image of the inspection object, so that the generation of a defective pattern can be suppressed in the generated restored split image. Therefore, by inspecting the inspection object TA based on a plurality of restored split images in which special patterns are restored and the generation of defective patterns is suppressed, the inspection accuracy of the inspection object can be improved.

In the above-mentioned aspect, the inspection unit may generate a restored image based on a plurality of restored divided images, and inspect the inspection object based on a comparison between the inspection image of the inspection object and the restored image.

According to this aspect, a restored image is generated based on a plurality of restored divided images, and the inspection object is inspected based on a comparison between the restored image and an inspection image of the inspection object. This makes it easy to realize inspection with high inspection accuracy.

In the above-mentioned aspect, the inspection unit may inspect the inspection object based on the difference between the restored image and the inspection image.

According to this aspect, the inspection object is inspected based on the difference between the restored image and the inspection image. This makes it easier to achieve inspection with high inspection accuracy.

The above-mentioned aspect may further include a learning unit that trains a learning model for each of one or more non-defective segmented images and generates multiple trained models.

According to this aspect, a learning unit is further provided that trains a learning model for each of one or more non-defective segmented images and generates multiple trained models. This makes it possible to obtain multiple trained models without a trained model generation device.

In the above-mentioned aspect, the learning unit may train a learning model using one or more good-product segmentation images that have been subjected to data augmentation as input, and generate multiple trained models.

According to this aspect, one or more good-product segmentation images that have been subjected to data augmentation are used as input to train a learning model, and multiple trained models are generated. This makes it possible to increase the limited number of good-product segmentation images, and each trained model can be designed to be robust against variations in good-product segmentation images.

In the above-described aspects, data augmentation may include applying at least one of translation, rotation, enlargement, reduction, left-right flipping, up-down flipping, and filtering to one or more good product segmentation images.

According to this aspect, data augmentation involves applying at least one of translation, rotation, enlargement, reduction, left-right flipping, up-down flipping, and filtering to one or more non-defective segmented images. This makes it easy to obtain variations of non-defective segmented images.

In the above-mentioned aspect, a division unit may be further provided that divides the inspection image of the object to be inspected into a plurality of inspection divided images.

According to this aspect, a division unit that divides the inspection image of the inspection object into a plurality of inspection divided images is further provided. This makes it possible to easily obtain the inspection divided images.

In the above-mentioned aspect, an imaging unit may be further provided for acquiring an inspection image of the object to be inspected.

According to this aspect, an imaging unit is further provided for acquiring an inspection image of the object to be inspected. This makes it possible to easily obtain the inspection image.

In the above-mentioned aspect, a memory unit may be further provided that stores multiple trained models.

According to this aspect, the system further includes a storage unit that stores multiple trained models. This makes it possible to easily read out each trained model.

An image inspection method according to another aspect of the present disclosure includes the steps of inputting inspection split images, which are split images of the object to be inspected, to multiple trained models, each of which has been trained to output a restored split image using one or more good-quality split images, which are split images of a good-quality inspection object, as input, to generate multiple restored split images, and inspecting the object to be inspected based on the multiple restored split images.

According to this aspect, the inspection split image is input to a plurality of trained models, each of which has been trained to output a restored split image using one or more good-quality split images as input, to generate a plurality of restored split images. This allows each trained model to learn a pattern specific to each good-quality split image, so that a local special pattern contained in the good-quality split image can be restored in the generated restored split image. Also, even if a pattern that is good at one position or part is defective at another position or part in an image of another good-quality inspection object, each trained model can learn a judgment criterion that differs depending on the position in the good-quality image of the inspection object, so that the generation of a defective pattern can be suppressed in the generated restored split image. Therefore, by inspecting the inspection object TA based on a plurality of restored split images in which special patterns are restored and the generation of defective patterns is suppressed, the inspection accuracy of the inspection object can be improved.

A trained model generation device according to another aspect of the present disclosure includes a model generation unit that generates multiple trained models, each trained to output a restored segmented image using one or more good-quality segmented images, which are segmented images of a good-quality inspection target, as input.

According to this aspect, a plurality of trained models are generated, each trained to input one or more good-quality divided images and output a restored divided image. As a result, each trained model can learn a pattern specific to each good-quality divided image, and therefore, in the output restored divided image, a localized special pattern contained in the good-quality divided image can be restored. In addition, even if a pattern that is good in one position or part in a good-quality image is defective in another position or part, each trained model can learn a judgment criterion that differs depending on the position in the good-quality image, and therefore, in the output restored divided image, generation of a defective pattern can be suppressed. Therefore, for example, by inputting an inspection divided image to each trained model to generate a restored divided image, it is possible to restore a localized special pattern in the restored divided image and suppress generation of a defective pattern.

The present invention makes it possible to restore special patterns and suppress the generation of defective patterns.

FIG. 1 is a configuration diagram illustrating a schematic configuration of an image inspection system according to an embodiment. FIG. 2 is a configuration diagram showing the physical configuration of a trained model generation device and an image inspection device in one embodiment. FIG. 3 is a configuration diagram showing a functional block configuration of a trained model generation device in one embodiment. FIG. 4 is a conceptual diagram for explaining generation of non-defective divided images by the learning image generating unit shown in FIG. FIG. 5 is a conceptual diagram for explaining a learning model used by the model generating unit shown in FIG. FIG. 6 is a conceptual diagram for explaining an example of generation of a plurality of trained models by the model generation unit shown in FIG. 3. FIG. 7 is a conceptual diagram for explaining another example of generation of multiple trained models by the model generation unit shown in FIG. 3. FIG. 8 is a configuration diagram showing a functional block configuration of an image inspection device according to an embodiment. FIG. 9 is a conceptual diagram for explaining an example of processing by the division unit, image generation unit, and inspection unit shown in FIG. 8. FIG. 9 is a conceptual diagram for explaining another example of the processing performed by the division unit, the image generation unit, and the inspection unit shown in FIG. 8. In FIG. FIG. 11 is a diagram showing an example of a non-defective product image. FIG. 12 is a conceptual diagram for explaining generation of a difference image by the inspection unit shown in FIG. FIG. 13 is a diagram showing an example of an inspection image. FIG. 14 is a diagram showing an example of a restored image. FIG. 15 is a diagram showing an example of a difference image. FIG. 16 is a flowchart illustrating an example of a trained model generation process performed by a trained model generation device in one embodiment. FIG. 17 is a flowchart for explaining an example of image inspection processing performed by an image inspection device in one embodiment.

The following describes an embodiment of the present invention. In the following description of the drawings, identical or similar parts are denoted by identical or similar reference symbols. However, the drawings are schematic. Therefore, specific dimensions, etc. should be determined in light of the following description. Furthermore, the drawings include parts with different dimensional relationships and ratios. Furthermore, the technical scope of the present invention should not be interpreted as being limited to the embodiment.

First, the configuration of an image inspection system according to one embodiment will be described with reference to FIG. 1. FIG. 1 is a configuration diagram illustrating the schematic configuration of an image inspection system 1 according to one embodiment.

As shown in FIG. 1, the image inspection system 1 includes an image inspection device 20 and illumination IL. The image inspection device 20 is connected to a trained model generation device 10 via a communication network NW. The illumination IL irradiates light L onto an inspection object TA. The image inspection device 20 captures reflected light R and inspects the inspection object TA based on an image of the inspection object TA (hereinafter also referred to as an "inspection image"). The trained model generation device 10 generates a trained model that the image inspection device 20 uses to perform the inspection.

Next, the physical configuration of a trained model generation device and an image inspection device according to one embodiment will be described with reference to FIG. 2. FIG. 2 is a configuration diagram showing the physical configuration of a trained model generation device 10 and an image inspection device 20 in one embodiment.

As shown in FIG. 2, the trained model generating device 10 and the image inspection device 20 each include a processor 31, a memory 32, a storage device 33, a communication device 34, an input device 35, and an output device 36. These components are connected to each other via a bus so that they can transmit and receive data.

In the example shown in FIG. 2, the trained model generating device 10 and the image inspection device 20 are each configured with a single computer, but this is not limited to the above. The trained model generating device 10 and the image inspection device 20 may each be realized by combining multiple computers. Furthermore, the configuration shown in FIG. 2 is an example, and the trained model generating device 10 and the image inspection device 20 may each have other configurations or may not have some of these configurations.

The processor 31 is configured to control the operation of each part of the trained model generating device 10 and the image inspection device 20. The processor 31 is configured to include integrated circuits such as a CPU (Central Processing Unit), a DSP (Digital Signal Processor), an ASIC (Application Specific Integrated Circuit), a PLD (Programmable Logic Device), an FPGA (Field Programmable Gate Array), and a SoC (System-on-a-Chip).

The memory 32 and the storage device 33 are each configured to store programs, data, etc. The memory 32 is composed of, for example, a ROM (Read Only Memory), an EPROM (Erasable Programmable ROM), an EEPROM (Electrically Erasable Programmable ROM), and/or a RAM (Random Access Memory), etc. The storage device 33 is composed of, for example, a storage device such as a HDD (Hard Disk Drive), an SSD (Solid State Drive), and/or an eMMC (embedded Multi Media Card).

The communication device 34 is configured to communicate via at least one of a wired and wireless network. The communication device 34 includes, for example, a network card, a communication module, an interface for connecting to other devices, etc.

The input device 35 is configured to allow information to be input by a user's operation. The input device 35 includes, for example, a keyboard, a touch panel, a mouse, a pointing device, and/or a microphone.

The output device 36 is configured to output information. The output device 36 includes a display device such as a liquid crystal display, an EL (Electro Luminescence) display, a plasma display, or an LCD (Liquid Crystal Display), and/or a speaker, etc.

Next, functional blocks of a trained model generating device according to one embodiment will be described with reference to FIG. 3 to FIG. 7. FIG. 3 is a block diagram showing the configuration of functional blocks of a trained model generating device 10 in one embodiment. FIG. 4 is a conceptual diagram for explaining the generation of good-product segmentation images by the training image generating unit 131 shown in FIG. 3. FIG. 5 is a conceptual diagram for explaining a training model 50 used by the model generating unit 135 shown in FIG. 3. FIG. 6 is a conceptual diagram for explaining an example of the generation of multiple trained models 55 by the model generating unit 135 shown in FIG. 3. FIG. 7 is a conceptual diagram for explaining another example of the generation of multiple trained models 55 by the model generating unit 135 shown in FIG. 3.

As shown in FIG. 3, the trained model generation device 10 includes a communication unit 110, a memory unit 120, and a learning unit 130.

The communication unit 110 is configured to be able to transmit and receive various types of information. The communication unit 110 transmits a trained model 55 (described later) to the image inspection device 20, for example, via the communication network NW. The communication unit 110 also receives good-quality images from other devices, for example, via the communication network NW. That is, the communication unit 110 acquires good-quality images that are images of good-quality products. The multiple good-quality images acquired by the communication unit 110 (hereinafter, when there is no need to distinguish between them, one of them is also referred to as a "good-quality image 40") are written and stored in the memory unit 120.

The storage unit 120 is configured to store various types of information. For example, the storage unit 120 stores a non-defective product image 40, a plurality of learning images (hereinafter, when there is no need to distinguish between them, each of them is also referred to as a "learning image 124"), and a plurality of trained models (hereinafter, when there is no need to distinguish between them, each of them is also referred to as a "trained model 55").

The learning unit 130 is for learning a learning model and generating a trained model. The learning unit 130 includes, for example, a learning image generation unit 131 and a model generation unit 135.

The learning image generation unit 131 is configured to generate learning images 124. The learning images 124 are images used to generate the trained model 55. The learning images 124 are, for example, images obtained by dividing one good-quality image into multiple images (hereinafter referred to as "good-quality divided images").

More specifically, the learning image generation unit 131 reads out one of the multiple good product images 40 stored in the storage unit 120, and divides the good product image 40 into multiple good product divided images to generate a learning image.

As shown in FIG. 4, the learning image generation unit 131 generates 16 good-quality divided images by dividing the good-quality image 40, for example, vertically and horizontally into four parts. Note that it is sufficient for the good-quality image 40 to be divided into two or more good-quality divided images, and it may be divided into 2 to 15 good-quality divided images, or into 17 or more good-quality divided images.

The learning image generation unit 131 may also perform data augmentation on one or more non-defective segmented images and generate the data-augmented images as the learning images 124.

Data augmentation involves applying at least one of translation, rotation, enlargement, reduction, left-right flipping, up-down flipping, and filtering to one or more non-defective segmented images. The filter used for filtering may be, for example, a filter that converts the brightness of the non-defective segmented images, a smoothing filter that removes noise, or a filter that extracts edges, and any filter can be used.

Note that data augmentation is not limited to the above-mentioned techniques. For example, data augmentation may involve performing projective transformation or adding random noise to one or more non-defective segmented images.

The model generation unit 135 is configured to train a learning model for each of one or more good-quality segmented images, and generate multiple trained models 55. Each trained model 55 is, for example, a model that receives one or more good-quality segmented images as input, and outputs a restored segmented image. Alternatively, each trained model 55 may be a model that receives one or more good-quality segmented images that have been subjected to data augmentation as input, and outputs a restored segmented image. In the following, unless otherwise specified, each trained model 55 will be described as having been generated using one or more good-quality segmented images as input. In this case, the restored segmented image is an image that restores the good-quality segmented image.

As shown in FIG. 5, the learning model 50 used by the model generation unit 135 to generate the trained model 55 is, for example, an autoencoder, which is a type of neural network and also a method of unsupervised machine learning. The learning model 50 includes an input layer 501, an output layer 505, and an intermediate layer 503 disposed between the input layer 501 and the output layer 505.

The learning model 50 is not limited to an autoencoder. The learning model 50 may be, for example, a model using PCA (Principal Component Analysis). In addition, the intermediate layer 503 of the learning model 50 is not limited to one layer, and may be two or more layers.

When an input image is input to the input layer 501, the dimensions of the input image are reduced in the intermediate layer 503 and a feature vector is extracted. Then, the dimensions are restored in the output layer 505 and an output image is output.

This learning model 50 learns weights (also called coefficients) so that the difference between the input image and the output image is minimized. More specifically, each node represented by a circle assigns a unique weight to each edge represented by a line, and the weighted value is input to the node in the next layer. By learning the weights in this weighting, the difference between each pixel in the input image and the corresponding pixel in the output image is minimized.

In this embodiment, the input image is, for example, a good-quality divided image, and the output image is a restored divided image. As shown in FIG. 6, for example, when generating a trained model 55 for a good-quality divided image 400 located at the upper left in a good-quality image 40, the upper left good-quality divided image 400 in each of the multiple good-quality images 40 is input as an input image to the corresponding training model 50A. Then, this training model 50A learns each weight so as to output a restored divided image that restores the upper left good-quality divided image 400 as an output image. In this way, a trained model 55A that restores the good-quality divided image 400 is generated. In this way, the model generation unit 135 generates multiple trained models 55A, 55B, 55C, ... by training the corresponding training models 50A, 50B, 50C, ... for each of the good-quality divided images 400, 402, 404, ... in the good-quality image 400.

The learning model 50 is not limited to the case where a good-quality divided image at one position is used as an input image. In other words, the number of divisions of the good-quality image 40, for example, the division number m (m is an integer of 2 or more), the number of learning models 50 used for learning, and the number of trained models 55 to be generated, for example, the generation number n (n is an integer of 2 or more), are not limited to the case where they are the same number. The learning model 50 may be trained using multiple good-quality divided images, for example, the good-quality divided image 400 and the good-quality divided image 402, as input images. In this case, a trained model 55 that restores the divided images of two areas of the good-quality divided image 400 and the good-quality divided image 402 is generated.

As shown in FIG. 7, among the multiple good-quality divided images 400', 402', 404', ... obtained by dividing the good-quality image 40, for example, if the good-quality divided image 402' and the good-quality divided image 404' are similar, the learning models 50 used for each may be integrated. That is, the good-quality divided image 402' and the good-quality divided image 404' that are similar to each other are input to the same learning model 50B, and as a result of learning, one trained model 55B' is generated. In this case, the number n of trained models 50 generated is smaller than the number m of divisions of the good-quality image 40 (number n generated < number m of divisions). Note that the determination of whether two good-quality divided images are similar may be made based on, for example, whether the degree of agreement is equal to or greater than a predetermined threshold value.

In this way, by generating multiple trained models 55 each trained to output a restored segmented image using one or more good segmented images as input, as described below, each trained model 55 can learn patterns specific to each good segmented image, and therefore can restore localized special patterns contained in the good segmented image in the output segmented restored image. Also, even if a pattern that is good in one position or part in the good image is defective in another position or part in the good image, as described below, each trained model 55 can learn different judgment criteria depending on the position in the good image, and therefore can suppress the generation of defective patterns in the output segmented restored image.

At least one of the learning image generation unit 131 and the model generation unit 135 may be realized by the processor 31 executing a program stored in the storage device 33. When executing a program, the program may be stored in a storage medium. The storage medium storing the program may be a non-transitory computer readable medium. The non-transitory storage medium is not particularly limited, and may be, for example, a storage medium such as a USB memory or a CD-ROM (Compact Disc ROM).

Next, functional blocks of an image inspection device according to an embodiment will be described with reference to FIG. 8 to FIG. 12. FIG. 8 is a configuration diagram showing the configuration of functional blocks of an image inspection device 20 in an embodiment. FIG. 9 is a conceptual diagram for explaining an example of processing by the division unit 250, image generation unit 260, and inspection unit 270 shown in FIG. 8. FIG. 10 is a conceptual diagram for explaining another example of processing by the division unit 250, image generation unit 260, and inspection unit 270 shown in FIG. 8. FIG. 11 is a diagram showing an example of a non-defective product image. FIG. 12 is a conceptual diagram for explaining generation of a difference image by the inspection unit 270 shown in FIG. 8. FIG. 13 is a diagram showing an example of an inspection image. FIG. 14 is a diagram showing an example of a restored image. FIG. 15 is a diagram showing an example of a difference image.

As shown in FIG. 8, the image inspection device 20 includes a communication unit 210, a memory unit 220, a learning unit 230, an imaging unit 240, a division unit 250, an image generation unit 260, and an inspection unit 270.

The communication unit 210 is configured to be able to transmit and receive various information. The communication unit 210 receives a plurality of trained models 55 from the trained model generation device 10, for example, via the communication network NW. The communication unit 210 also receives a plurality of training images 124 from the trained model generation device 10 or another device, for example, via the communication network NW. The received plurality of training images 124 and plurality of trained models 55 are written and stored in the storage unit 220. Note that the communication unit 210 may receive only one of the plurality of training images 124 and the plurality of trained models 55. When the communication unit 210 receives only the plurality of training images 124, the learning unit 230, which will be described later, uses the plurality of training images 124 to generate a plurality of trained models 55.

The storage unit 220 is configured to store various types of information. For example, the storage unit 220 stores a plurality of learning images 124 and a plurality of trained models 55. In this way, by providing the storage unit 220 that stores a plurality of trained models 55, each trained model can be easily read out.

The learning unit 230 is configured to train a learning model for each of one or more non-defective segmented images, and generate multiple trained models 55. Each trained model 55 is a model that receives one or more non-defective segmented images as input, and outputs a restored segmented image. The multiple trained models 55 thus generated are written and stored in the storage unit 220. Note that the method of training the learning model is similar to the description of the model generation unit 135 in the trained model generation device 10, and therefore will not be described here.

In this way, by providing a learning unit 230 that trains a learning model for one or more non-defective segmented images and generates multiple trained models 55, multiple trained models can be obtained even without a trained model generation device 10.

In addition, when each of the multiple learning images stored in the memory unit 220 is a non-defective segmented image, the learning unit 230 may perform data augmentation on one or more of the non-defective segmented images, train a learning model using the data-augmented images as input, and generate multiple trained models 55. This allows a limited number of non-defective segmented images to be increased, and each trained model 55 can be designed to be robust with increased robustness against variations in the non-defective segmented images.

As described above, data augmentation includes applying at least one of translation, rotation, enlargement, reduction, left-right flip, up-down flip, and filtering to one or more good-quality segmented images. The filter used for filtering may be, for example, a filter that converts the brightness of the good-quality segmented image, a smoothing filter that removes noise, or a filter that extracts edges, and any filter can be used. In this way, data augmentation includes applying at least one of translation, rotation, enlargement, reduction, left-right flip, up-down flip, and filtering to one or more good-quality segmented images, and thus variations of good-quality segmented images can be easily obtained.

Note that data augmentation is not limited to the above-mentioned techniques. For example, data augmentation may involve performing projective transformation or adding random noise to one or more non-defective segmented images.

The imaging section 240 is for acquiring an inspection image of the inspection object TA. The imaging section 240 is configured to include an imaging device such as a camera. In this embodiment, the imaging section 240 receives reflected light R from the inspection object TA and acquires an inspection image. The imaging section 240 then outputs the acquired inspection image to the division section 250 and the inspection section 270. In this manner, by acquiring an inspection image of the inspection object TA, the inspection image can be easily obtained.

The division unit 250 is configured to divide the inspection image of the inspection object TA into a plurality of inspection split images. More specifically, the division unit 250 divides the inspection image using a method similar to that used to divide a non-defective product image in the trained model generation device 10. Specifically, the division unit 250 divides the inspection image of the inspection object TA vertically and horizontally into four, thereby generating 16 inspection split images. The division unit 250 then outputs the generated inspection split images to the image generation unit 260. In this manner, by dividing the inspection image of the inspection object TA into a plurality of inspection split images, the inspection split images can be easily obtained.

The image generation unit 260 is configured to input the inspection split image to each of the multiple trained models 55 and generate multiple restored split images.

As shown in FIG. 9, the inspection image of the inspection object TA acquired by the imaging unit 240 is divided into a plurality of inspection split images 420, 422, 424, ... by the division unit 250. The image generation unit 260 inputs each of the inspection split images 420, 422, 424, ... to a corresponding trained model 55A, 55B, 55C, ... among the plurality of trained models 55. Each trained model 55A, 55B, 55C, ... outputs a restored split image 440, 442, 444, ... corresponding to each of the inspection split images 420, 422, 424, ..., and the image generation unit 260 generates a plurality of restored split images 440, 442, 444, ....

The inspection unit 270, which will be described later, generates a restored image 44 having the same vertical and horizontal size (number of pixels) as the inspection image 42 by integrating the multiple restored split images 440, 442, 444, ... that have been generated.

As described above, when a plurality of mutually similar non-defective divided images are integrated into one learning model 50 to generate a learned model 55, as shown in FIG. 10, the image generating unit 260 inputs the inspection divided images at positions corresponding to the mutually similar non-defective divided images in the inspection image 42, for example, the inspection divided image 422 and the inspection divided image 424, into the corresponding learned model 55B'. The learned model 55B' outputs the restored divided image 442 corresponding to the inspection divided image 422, and outputs the restored divided image 444 corresponding to the inspection divided image 424. The image generating unit 260 generates the restored divided images 440, 442, 444, ... by each of the learned models 55A, 55B', ... outputting the restored divided images 440, 442, 444, .... Note that the restored split image 442 output by the trained model 55B' is an image corresponding to the inspection split image 422, and the restored split image 444 output by the trained model 55B' is an image corresponding to the inspection split image 424. In other words, if the inspection split image 422 and the inspection split image 424 are the same, the restored split image 442 and the restored split image 444 will be the same image, but if there is a difference between the inspection split image 422 and the inspection split image 424, the restored split image 442 and the restored split image 444 may be different images.

Here, as shown in FIG. 11, a non-defective product image 60 of a non-defective inspection object TA is divided, for example, into six non-defective product divided images 600, 602, 604, 606, 608, and 610. Of these six non-defective product divided images, the non-defective product divided images 602, 606, and 608 contain patterns that are similar to each other. In addition, the non-defective product divided image 604 contains a special pattern that is different from the other non-defective product divided images.

Unlike the multiple trained models 55A, 55B, 55C, ... of this embodiment, if one trained model is generated using these six good-product segmented images, the patterns of the good-product segmented images 602, 604, or 608 can be reproduced in the restored segmented image output by the trained model, but if the expressive ability of the trained model is low, there is a possibility that the special pattern contained in the good-product segmented image 604 will not be restored.

In contrast, the image inspection device 20 of this embodiment inputs the good-quality divided images 600, 602, 604, 606, 608, and 610 to multiple trained models 55, each of which has been trained to take one or more good-quality divided images as input and output a restored divided image, to generate multiple restored divided images. This allows each trained model 55 to learn patterns specific to each of the good-quality divided images 600, 602, 604, 606, 608, and 610, so that the localized special pattern contained in the good-quality divided image 604 can be restored in the generated restored divided image.

In addition, in a good-quality image of another good-quality inspection target TA, a pattern that is good in one position or part may be defective in another position or part. Even in such cases, in the image inspection device 20 of this embodiment, each trained model 55 can learn different judgment criteria depending on the position in the good-quality image of the inspection target TA, so that the generation of defective patterns can be suppressed in the generated split-restored image.

Returning to the explanation of FIG. 8, the inspection unit 270 is configured to inspect the inspection object TA based on the multiple restored divided images generated by the image generation unit 260. As described above, in each restored divided image generated by the image generation unit 260, special patterns are restored and the generation of defective patterns is suppressed. Therefore, by inspecting the inspection object TA based on these multiple restored divided images, the inspection accuracy of the inspection object TA can be improved.

More specifically, the inspection unit 270 is configured to generate a restored image based on a plurality of restored divided images, and to inspect the inspection object TA based on a comparison between the restored image and an inspection image of the inspection object TA. This makes it easy to realize inspection with high inspection accuracy.

Specifically, the inspection unit 270 inspects the inspection object TA based on the difference between the restored image and the inspection image. This makes it easier to achieve inspection with high inspection accuracy.

As shown in FIG. 12, the inspection unit 270, for example, calculates the difference between the inspection image 42 and the restored image 44 to generate a difference image 46. Then, the inspection unit 270 determines whether the inspection target TA is a non-defective product based on the difference image 46.

As shown in FIG. 13, the inspection image 72 includes, for example, a linear defect image 700. The defect image is an image of a defect in the object being inspected.

On the other hand, as shown in FIG. 12, in the restored image 74, a non-defective image is restored, and therefore the defective image has been removed.

For this reason, as shown in FIG. 15, the difference image 76 showing the difference between the inspection image 72 and the restored image 74 of the inspection object TA mainly contains a defect image 760. In this embodiment, the inspection unit 270 judges whether the inspection object TA is a good product or not based on the difference image 76 including this defect image 760. For example, when the size of the defect image 760 exceeds a predetermined threshold, the inspection unit 270 judges that the inspection object TA is not a good product, that is, is a defective product.

Alternatively, the inspection unit 270 may detect defects in the inspection object TA based on the presence or absence of a defect image 760 contained in the difference image 76.

At least one of the learning unit 230, the division unit 250, the image generation unit 260, and the inspection unit 270 may be realized by the processor 31 executing a program stored in the storage device 33. When executing a program, the program may be stored in a storage medium. The storage medium storing the program may be a computer-readable non-transient storage medium. The non-transient storage medium is not particularly limited, and may be, for example, a storage medium such as a USB memory or a CD-ROM.

Next, a processing procedure performed by a trained model generation device according to one embodiment will be described with reference to FIG. 16. FIG. 16 is a flowchart for explaining an example of a trained model generation process S100 performed by a trained model generation device 10 in one embodiment.

As shown in FIG. 16, first, the communication unit 110 acquires a plurality of non-defective product images 40 via the communication network NW (step S101). The acquired non-defective product images 40 are stored in the storage unit 120.

Next, the learning image generation unit 131 reads out a plurality of good-quality images 40 from the storage unit 120, and generates a plurality of learning images 124 based on the plurality of good-quality images 40 (step S102). As described above, the learning images 124 may be good-quality divided images, or may be images obtained by performing data augmentation on one or more good-quality divided images.

Next, the model generation unit 135 receives the multiple training images 124 generated in step S102 as input, and generates multiple trained models 55, each trained to output a restored split image (step S103). The multiple trained models 55 thus generated are stored in the storage unit 120.

Next, the communication unit 110 transmits the multiple trained models 55 generated in step S103 to the image inspection device 20 via the communication network NW (step S104). This allows the image inspection device 20 to use the multiple trained models generated by the trained model generation device 10.

After step S104, the trained model generation device 10 terminates the trained model generation process S100.

Next, the processing procedure performed by the image inspection device according to one embodiment will be described with reference to FIG. 17. FIG. 17 is a flowchart for explaining an example of the image inspection process S200 performed by the image inspection device 20 in one embodiment.

In the following example, it is assumed that the communication unit 210 receives multiple trained models 55 from the trained model generation device 10, and the multiple trained models 55 are stored in the storage unit 220.

As shown in FIG. 17, first, the imaging unit 240 acquires an inspection image of the inspection object TA (step S201). The acquired inspection image is output to the division unit 250 and the inspection unit 270.

Next, the division unit 250 divides the inspection image acquired in step S201 to generate a plurality of inspection divided images (step S202). The generated plurality of inspection divided images are output to the image generation unit 260.

Next, the image generating unit 260 reads out the multiple trained models 55 previously stored in the storage unit 220, and inputs the multiple inspection split images generated in step S207 into the multiple trained models 55, respectively, to generate multiple restored split images (step S203). The multiple restored split images thus generated are output to the inspection unit 270.

Next, the inspection unit 270 integrates the multiple restored divided images generated in step S203 to generate a restored image (step S204). Then, the inspection unit 270 calculates the difference between the inspection image acquired in step S201 and the restored image generated in step S204 to generate a difference image (step S205).

Next, the inspection unit 270 inspects the inspection object TA based on the difference image generated in step S205 (step S206).

After step S206, the image inspection device 20 terminates the image inspection process S200.

The sequence and flow chart described in this embodiment may be rearranged as long as no inconsistencies occur in the processing.

The above describes an exemplary embodiment of the present invention. According to the image inspection device 20 and image inspection method of this embodiment, the good-quality divided images 600, 602, 604, 606, 608, and 610 are input to the plurality of trained models 55, each of which has been trained to output a restored divided image using one or more good-quality divided images as input, to generate a plurality of restored divided images. As a result, each trained model 55 can learn a pattern specific to each of the good-quality divided images 600, 602, 604, 606, 608, and 610, so that the local special pattern contained in the good-quality divided image 604 can be restored in the generated restored divided image. Also, even if a pattern that is good at a certain position or part is defective at another position or part in an image of another good-quality inspection object TA, each trained model 55 can learn a judgment criterion that differs depending on the position in the inspection image of the inspection object TA, so that the generation of a defective pattern can be suppressed in the generated restored divided image. Therefore, by inspecting the inspection object TA based on multiple divided restored images in which special patterns are restored and the generation of defective patterns is suppressed, the inspection accuracy of the inspection object TA can be improved.

In addition, according to the trained model generating device 10 of this embodiment, a plurality of trained models 55 are generated, each trained to input one or more good-quality divided images and output a restored divided image. As a result, each trained model 55 can learn a pattern specific to each good-quality divided image, so that a local special pattern contained in the good-quality divided image can be restored in the output restored divided image. Also, even if a pattern that is good in one position or part in a good-quality image is defective in another position or part, each trained model 55 can learn a judgment criterion that differs depending on the position in the good-quality image, so that the generation of a defective pattern can be suppressed in the output restored divided image. Therefore, for example, by inputting an inspection divided image to each trained model 55 to generate a restored divided image, a local special pattern can be restored in the restored divided image, and the generation of a defective pattern can be suppressed.

The above-described embodiments are intended to facilitate understanding of the present invention, and are not intended to limit the present invention. The elements of the embodiments, as well as their arrangements, materials, conditions, shapes, sizes, etc., are not limited to those exemplified, and may be modified as appropriate. Furthermore, configurations shown in different embodiments may be partially substituted or combined.

[Appendix 1]
an image generating unit (260) that inputs an inspection split image, which is a split image of an inspection object (TA), to a plurality of trained models (55) each trained to output a restored split image using one or more non-defective split images, which are split images of a non-defective inspection object (TA), and generates a plurality of restored split images;
An inspection unit (270) that inspects an inspection object (TA) based on a plurality of restored divided images.
An image inspection device (20).
[Appendix 10]
A step of inputting an inspection split image, which is a split image of an inspection object (TA), into a plurality of trained models (55), each of which has been trained to output a restored split image using one or more good split images, which are split images of a good inspection object (TA), as input, and generating a plurality of restored split images;
Inspecting an object to be inspected (TA) based on the plurality of restored split images.
Imaging methods.
[Appendix 11]
A model generation unit (135) is provided which generates a plurality of trained models (55) each trained to output a restored divided image using one or more non-defective divided images, which are divided images of a non-defective inspection object (TA), as input.
A trained model generating device (10).

1...Image inspection system, 10...Trained model generation device, 20...Image inspection device, 31...Processor, 32...Memory, 33...Storage device, 34...Communication device, 35...Input device, 36...Output device, 40...Good product image, 42...Inspection image, 44...Restored image, 46...Difference image, 50...Learning model, 55, 55A, 55B, 55C...Trained model, 60...Good product image, 72...Inspection image, 74...Restored image, 76...Difference image, 110...Communication unit, 120...Storage unit, 124...Learning image, 130...Learning unit, 131...Learning image generation unit, 135...Model generation unit, 210...Communication unit, 220...storage unit, 230...learning unit, 240...imaging unit, 250...segmentation unit, 260...image generation unit, 270...inspection unit, 400, 402, 404...good product segmented images, 420, 422, 424...inspection segmented images, 440, 442, 444...restored segmented images, 501...input layer, 503...intermediate layer, 505...output layer, 600, 602, 604, 606, 608, 610...good product segmented images, 700...defective image, 760...defective image, IL...illumination, L...light, NW...communication network, R...reflected light, S100...trained model generation process, S200...image inspection process, TA...inspection object.

Claims (11)

an image generating unit that inputs an inspection split image, which is a split image of an inspection object, to a plurality of trained models each trained to output a restored split image using one or more of the non-defective split images, which are split images of a non-defective inspection object, as input, and generates a plurality of the restored split images;
an inspection unit that inspects the inspection object based on the plurality of restored divided images ,
The good-quality divided image is a good-quality image divided into a plurality of parts, and the plurality of trained models include trained models trained to input the good-quality divided images at a plurality of positions in the good-quality image and output the restored divided images.
Image inspection equipment. the inspection unit generates a restored image based on the plurality of restored divided images, and inspects the inspection object based on a comparison between an inspection image of the inspection object and the restored image.
The image inspection device according to claim 1 . the inspection unit determines whether the inspection object is a non-defective product based on a difference between the inspection image and the restored image.
The image inspection device according to claim 2. A learning unit that trains a learning model for each of the one or more non-defective divided images and generates the plurality of trained models,
4. An image inspection device according to claim 1. The learning unit trains the learning model using one or more of the non-defective segmented images that have been subjected to data augmentation as input, and generates the multiple trained models.
5. The image inspection device according to claim 4. The data augmentation includes applying at least one of translation, rotation, enlargement, reduction, left-right flipping, up-down flipping, and filtering to the non-defective divided image.
6. An image inspection device according to claim 5. A division unit that divides an inspection image of an inspection object into a plurality of the inspection divided images.
7. An image inspection device according to claim 1. Further comprising an imaging unit for acquiring an inspection image of the inspection object.
8. An image inspection device according to claim 1. Further comprising a storage unit that stores the plurality of trained models.
9. An image inspection device according to any one of claims 1 to 8. A step of inputting an inspection split image, which is a split image of an inspection object, to a plurality of trained models each trained to output a restored split image using one or more good-item split images, which are split images of a good-item inspection object, as input, and generating a plurality of the restored split images;
inspecting the inspection object based on the plurality of restored divided images ;
The good-quality divided image is a good-quality image divided into a plurality of parts, and the plurality of trained models include trained models trained to input the good-quality divided images at a plurality of positions in the good-quality image and output the restored divided images.
Imaging methods. A model generation unit that generates a plurality of trained models each trained to output a restored divided image using one or more non-defective divided images, which are divided images of a non-defective inspection object, as input ;
The good-quality divided image is a good-quality image divided into a plurality of parts, and the plurality of trained models include trained models trained to input the good-quality divided images at a plurality of positions in the good-quality image and output the restored divided images.
A trained model generator.