JP7600909B2 - Anomaly detection system and anomaly detection method - Google Patents

Description

The present invention relates to an anomaly detection system and an anomaly detection method.

Conventionally, a method for detecting abnormalities in an inspection object using image analysis involves training an object with images of non-defective products without abnormalities and images of defective products containing abnormalities, generating a model, and then using the model to detect abnormalities in the inspection object. However, depending on the inspection object, defective products may occur only infrequently, making it difficult to collect sufficient images of the defective products.

For example, Patent Document 1 discloses a technology in which a neural network trained using images of non-defective products as learning images generates a restored image (pseudo image) from an image of the inspection target, and detects the presence of an abnormality in the inspection target based on the difference between the image of the inspection target and the restored image.

JP 2020-160997 A

The inventors have discovered that this method of detecting anomalies using the difference between an image of an object to be inspected and a restored image is prone to overdetection, where an object to be inspected that is actually a good product is detected as an anomaly.

For example, when inspecting images of test objects that have variation in shape, even among good products, such as weld marks formed by welding, or when inspecting images of test objects that have deviations or variations in the position or orientation of the test object in the test object image due to problems with the imaging environment of the test object, these variations can easily cause test objects that are actually good to be detected as abnormal, resulting in overdetection.

The present invention has been developed in consideration of these circumstances, and provides an anomaly detection system and an anomaly detection method that suppresses the occurrence of overdetection and improves the accuracy of anomaly detection.

An anomaly detection system according to one aspect of the present invention includes:
an acquisition unit for acquiring an inspection object image of an inspection object;
a restored image generating unit that inputs the image of the test object to a learning model that has been trained using images of the test object that are normal as training data, and generates a restored image;
a detection unit that uses the restored image and a normal restored image that is a restored image generated by inputting an image of the inspection target that is normal to the learning model to determine whether or not the inspection target in the inspection target image is abnormal;
It is equipped with the following.

With this configuration, even in cases where overdetection occurs due to variations in brightness or other factors in the conventional method of detecting abnormalities using the difference between the image of the test subject and the restored image, the restored image generated from the image of the test subject and the normal restored image generated from the image of the normal test subject are similar images, making it possible to suppress overdetection and improve the accuracy of detecting abnormalities.

In the anomaly detection system of the above aspect, the detection unit may determine whether the inspection object in the inspection object image is abnormal by using the similarity between the restored image and a normal restored image, which is a restored image generated by inputting an image of the inspection object that is normal to the learning model.

This configuration allows for more accurate determination of whether an abnormality exists, reducing overdetection and improving the accuracy of abnormality detection.

The anomaly detection system of the above aspect may further include a preliminary detection unit that uses the difference between the inspection target image and the restored image to determine whether or not the inspection target in the inspection target image is abnormal, and the detection unit may re-determine whether or not the inspection target determined to be abnormal by the preliminary detection unit is abnormal.

With this configuration, the preliminary detection unit performs a judgment using the difference, which requires less calculation processing, and the detection unit re-evaluates only the test objects that are judged to be abnormal, thereby shortening the time required for abnormality processing.

In the anomaly detection system of the above aspect, the learning model may be a learning model using a generative adversarial network.

This configuration can improve the reproducibility of the restored image generated by the restored image generation unit, thereby improving the accuracy of detecting abnormalities.

In the anomaly detection system of the above aspect, the inspection object may be a weld mark formed by welding on a member.

This configuration makes it possible to reduce overdetection and improve the accuracy of abnormality detection, particularly in detecting abnormalities in weld marks, which are prone to overdetection due to variations in shape, etc.

In the anomaly detection system of the above aspect, the inspection target image may be an image captured on a production line.

This configuration makes it possible to reduce overdetection and improve the accuracy of anomaly detection, particularly in anomaly detection using images captured on a production line where overdetection is likely to occur due to misalignment or variation in the position or orientation of the object being inspected.

An anomaly detection method according to one aspect of the present invention includes:
An acquisition step of acquiring an inspection object image of an inspection object;
a restored image generating step of inputting the inspection target image into a learning model trained using a normal inspection target image as training data, and generating a restored image;
a detection step of determining whether or not the inspection target in the inspection target image is abnormal using the restored image and a normal restored image, which is a restored image generated by inputting an image of the inspection target that is normal to the learning model;
It is equipped with the following.

The present invention provides an anomaly detection system and an anomaly detection method that suppresses the occurrence of overdetection and improves the accuracy of anomaly detection.

FIG. 1 is a block diagram showing an outline of a hardware configuration of an anomaly detection system according to a first embodiment. FIG. 1 is a block diagram showing a functional configuration of an anomaly detection system according to a first embodiment. FIG. 4 is a flow diagram showing an anomaly detection process of the anomaly detection system according to the first embodiment. 1A to 1C are explanatory diagrams showing a captured image, a restored image, and a difference image of a normal inspection target without variation in brightness, etc. 1A to 1C are explanatory diagrams showing a schematic diagram of a captured image, a restored image, and a difference image of a normal inspection target, although there is variation in brightness, etc. 1A to 1C are explanatory diagrams showing a captured image, a restored image, and a difference image of an inspection object that includes a defective portion and is abnormal. FIG. 2 is an explanatory diagram illustrating a restored image of an object to be inspected.

Specific embodiments to which the present invention is applied will be described in detail below with reference to the drawings. However, the present invention is not limited to the following embodiments. In addition, the following descriptions and drawings have been simplified as appropriate for clarity of explanation.

First Embodiment
First, a description will be given of an anomaly detection system according to a first embodiment of the present invention. Fig. 1 is a block diagram showing an outline of a hardware configuration of the anomaly detection system according to the present embodiment.

As shown in FIG. 1, the anomaly detection system 100 has computer resources that are possessed by a general information processing device such as a personal computer. Specifically, the system includes a CPU (Central Processing Unit) 101, a ROM (Read Only Memory) 102, a RAM (Random Access Memory) 103, a HDD (Hard Disk Drive) 104, a communication interface (I/F) 105, and an input/output interface (I/F) 106. These components are connected to each other so as to be able to communicate with each other via a data bus 107. Note that, although an example in which each component of the anomaly detection system 100 is realized by a general information processing device will be described here, some or all of these functions may be realized by an external device such as an edge or a cloud.

The CPU 101 is a microprocessor that performs overall control of the anomaly detection system 100. Specifically, it reads out various control programs executed in this embodiment that are stored in the ROM 102 and the HDD 104, and executes these programs that are deployed on the RAM 103. Here, an SSD (Solid State Drive) may be provided as a storage device instead of or in addition to the HDD 104.

The communication interface 105 communicates between the anomaly detection system 100 and an external device. In this embodiment, the communication interface 105 receives an image of the inspection target from an external device, such as an imaging device that captures an image of the inspection target. Here, communication is realized by various communication technologies, whether wireless or wired.

The input/output interface 106 performs input/output between the anomaly detection system 100 and the outside. For example, it may be equipped with an output device such as a display that displays information about anomaly detection, and an input device for use by an operator.

Next, the functional configuration of the anomaly detection system according to this embodiment will be described. Figure 2 is a block diagram showing the functional configuration of the anomaly detection system according to this embodiment.

As shown in FIG. 2, the anomaly detection system 100 has, as its functional configuration, an acquisition unit 201, a restored image generation unit 202, a preliminary detection unit 203, and a detection unit 204. These function when the CPU 101 executes various control programs stored in the ROM 102 or the like. In addition, some or all of the functions of the anomaly detection system 100 may be realized by a hardware circuit.

The acquisition unit 201 acquires captured images from an external device. Specifically, when detecting an abnormality in an inspection target, an inspection target image to be inspected and a normal image of the inspection target that is normal are acquired. Here, a normal image is an image of an inspection target that has been confirmed to have no abnormality. The acquired inspection target image is output to the restored image generation unit 202 and the preliminary detection unit 203, and the acquired normal image is output to the restored image generation unit 202. Note that, although an example in which the captured images are acquired from an imaging device, which is an external device, is described here, the system may be realized in a configuration in which the imaging device is included in the anomaly detection system.

The restored image generating unit 202 generates a restored image by restoring the inspection target image and the normal image output from the acquiring unit 201 to a learning model that has been trained in advance using the normal image as training data. The learning method of this learning model will be described in detail later. The restored image by restoring the inspection target image is output to the preliminary detection unit, and the normally restored image by restoring the normal image is output to the detection unit.

The preliminary detection unit 203 judges whether the inspection object is abnormal by using the difference between the inspection object image output from the acquisition unit 201 and the restored image output from the restored image generation unit 202. Specifically, first, a difference image is generated by subtracting pixel values of the restored image from pixel values of the inspection object image. Next, it is judged whether the sum of pixel values contained in the generated difference image is greater than a predetermined threshold value, and if it is greater, it is judged to be abnormal. The restored image of the inspection object judged to be abnormal is then output to the detection unit. Here, a difference image is generated by subtracting pixels of the restored image at corresponding positions for each pixel of the inspection object image, but the difference image may be generated using the difference for each group including multiple pixels.

The detection unit 204 re-determines whether the inspection target is abnormal or not by using the restored image of the inspection target determined to be abnormal by the preliminary detection unit 203 and outputted, and the normal restored image, which is a restored image obtained by restoring the normal image outputted from the restored image generation unit 202. Specifically, the similarity between these images is calculated first. For example, the similarity can be calculated by various similarity calculation methods such as normalized correlation method (NCC) and shape pattern matching that calculates similarity based on edge features. It is determined whether the calculated similarity is lower than a predetermined threshold, and if it is lower, it is determined to be abnormal. In addition, here, the similarity between the restored image of the inspection target and the normal restored image, which is a restored image obtained by restoring a normal image, is calculated and whether it is abnormal or not is determined, but similar to the method by the preliminary detection unit, a difference image may be generated from these images and whether it is abnormal or not may be determined from the pixel values included in the difference image. In this way, the detection unit re-determines whether it is abnormal or not, and information on the abnormality detection is outputted to the input/output interface 106.

Comparing the anomaly determination methods of the preliminary detection unit 203 and the detection unit 204 in this embodiment, the method of determining anomaly by the preliminary detection unit 203 can determine anomaly simply by taking the difference between the images to be compared. Therefore, although the accuracy of the determination is lower than the method of calculating the similarity of the detection unit 204, less calculation processing is required and the processing time is shorter. Therefore, in this embodiment, the preliminary detection unit 203 determines in advance whether the inspection object is abnormal or not, and the detection unit 204 re-determines only those inspection objects determined to be abnormal, thereby making it possible to reduce the overall time required for inspection while suppressing overdetection.

Although an example in which the functional configuration of the anomaly detection system 100 includes the preliminary detection unit 203 has been described here, the functional configuration may also include no preliminary detection unit 203. In that case, the restored image of the inspection object generated by the restored image generation unit 202 and the normally restored image obtained by restoring the normal image are output directly to the detection unit 204. Then, the detection unit 204 uses the restored image of the inspection object and the normally restored image obtained by restoring the normal image to determine whether or not the inspection object is abnormal.

Next, a learning method for the learning model according to this embodiment will be described. In this embodiment, as a preliminary step before executing the anomaly detection process, learning is performed on the neural network, which is the learning model possessed by the restored image generating unit 202.

When training a neural network, the acquisition unit 201 acquires normal images, which are images of a test subject that is normal, as training data. Here, the acquisition unit 201 may acquire images of the test subject that contain abnormalities, as long as the number of normal images acquired as training data is a small number equal to or less than a preset ratio.

The acquired training data is used to train a generative adversarial network (GAN), which is a learning model of the restored image generation unit 202. Specifically, the restored image generation unit 202 includes a generator having a neural network that generates pseudo data from the acquired training data, and a discriminator having a neural network that determines whether the pseudo data is true or false, and the generator and discriminator are trained using the training data and the pseudo data generated by the generator. In this way, the use of a generative adversarial network can improve the reproducibility of the restored image. In this embodiment, a restored image is generated by inputting the image to be inspected into the learning model trained in this way.

Next, the anomaly detection process flow of the anomaly detection system 100 according to this embodiment will be described. FIG. 3 is a flow diagram of the anomaly detection process according to this embodiment. This flow starts when the learning of the learning model described above is completed. In addition, it is preferable that the normally restored image obtained by restoring the normal image is stored in the anomaly detection system 100 in advance.

In this explanation, we will use diagrams (hereinafter referred to as images) that show the images in Figures 4 to 7 as appropriate, and use as an example a weld mark formed by welding on various members such as parts as an inspection target. Specifically, Figure 4(a) is an image of a normal weld mark with no variation in brightness, etc., Figure 5(a) is an image of a weld mark that is normal but includes a portion 5A that is locally different in color due to variation in brightness, etc., and Figure 6(a) is an image of an abnormal weld mark that includes a defective portion 6A.

First, in S11, the acquisition unit 201 acquires an image of the inspection object captured by an external device, as shown in Figures 4(a), 5(a), and 6(a).

Next, in S12, the restored image generating unit 202 generates restored images by inputting the inspection target image output from the acquiring unit 201 into a learning model that has been previously trained, thereby restoring these captured images. Specifically, Fig. 4(b), Fig. 5(b), and Fig. 6(b) are restored images generated by inputting the images of Fig. 4(a), Fig. 5(a), and Fig. 6(a) into the learning model, respectively. Here, as described above, since this learning model is a learning model that has been trained using normal images as teacher data, the input captured image is restored as an image close to a normal image. Therefore, the locally colored area 5A and defective area 6A confirmed in the images of Fig. 5(a) and Fig. 6(a) are not restored in the restored images of Fig. 5(b) and Fig. 6(b), respectively. On the other hand, the image in FIG. 5(a) is an image of a normal weld mark just like the image in FIG. 4(a) except that it contains a locally different colored area 5A, so the restored image in FIG. 5(b) is relatively close to the restored image in FIG. 4(b) obtained by restoring the image in FIG. 4(a), whereas the image in FIG. 6(a) contains a defective area 6A, so the image features change and the restored image in FIG. 6(b) is blurry.

Next, in S13, the preliminary detection unit 203 uses the difference between the captured image and the restored image to determine whether the inspection target is abnormal. Specifically, first, the difference images shown in Figs. 4(c), 5(c), and 6(c) are generated by the difference between the captured images shown in Figs. 4(a), 5(a), and 6(a) and the restored images shown in Figs. 4(b), 5(b), and 6(b). Here, Fig. 4(c), which is a normal weld mark difference image without variation in brightness, has a small difference because Figs. 4(a) and 4(b) are similar images. On the other hand, the difference images shown in Figs. 5(c) and 6(c) have a large difference because the part 5A with a locally different color and the defective part 6A are not restored in the restored images of Figs. 5(b) and 6(b), as described above. Next, it is determined whether the difference is greater than a predetermined threshold value. If the difference is smaller than the predetermined threshold, as in the difference image shown in FIG. 4(c), the preliminary detection unit 203 determines that the inspection object is normal, and this flow ends. On the other hand, if the difference is larger than the predetermined threshold, as in the difference images shown in FIG. 5(c) and FIG. 6(c), the process proceeds to S14.

Here, even though the captured image shown in FIG. 5(a) is an image of a normal weld mark, the inspection object is not judged to be normal in S13, i.e., in the judgment using the difference between the inspection object image and the restored image. In other words, if there is variation in brightness or shape of the inspection object image, the normal images used to train the learning model cannot cover all normal images, so even normal images cannot be properly restored, and as a result, the inspection object is detected to be abnormal based on the difference, resulting in a problem of so-called overdetection. On the other hand, in the captured image shown in FIG. 4(a), the inspection object is judged to be normal in S13, making it unnecessary for the detection unit 204 to make a re-judgment, and the processing time required for the abnormality detection process can be shortened.

Next, in S14, the detection unit 204 re-determines whether the inspection target is abnormal or not by using the restored image of the inspection target in which an abnormality was detected by the preliminary detection unit 203 and the normal restored image generated by the restored image generation unit 202 or previously stored in the abnormality detection system 100, which is a restored normal image. Specifically, the similarity of these images is first calculated. FIG. 7(a) shows a normal restored image restored from an image of a normal welding mark, FIG. 7(b) shows a restored image of the inspection target shown in FIG. 5(b), and FIG. 7(c) shows a restored image of the inspection target shown in FIG. 6(b). Here, FIG. 7(b), which is a restored image of an image of a welding mark including a portion 5A that is locally different in color due to variations in brightness, etc., is a restored image of an image of a normal welding mark except for the portion 5A that is locally different in color, as described above, and therefore shows a high similarity to FIG. 7(a), which is a normal restored image. On the other hand, FIG. 7(c), which is a restored image of an image of an abnormal weld mark including a defective portion 6A, is a blurred image, and therefore shows a low similarity to FIG. 7(a), which is a normal restored image. Next, it is determined whether the calculated similarity is lower than a predetermined threshold value. When the similarity is high, as in the restored image of the inspection object shown in FIG. 7(b), the detection unit 204 determines that the inspection object is normal, and this flow ends. On the other hand, when the similarity is low, as in the restored image of the inspection object shown in FIG. 7(c), the detection unit 204 determines that the inspection object is abnormal, and this flow ends. Although omitted in this flow, a process of outputting information on the abnormality detection to the input/output interface 106 may be added.

In addition, here, an example including S13 has been described as an anomaly detection processing flow of the anomaly detection system 100, but an anomaly detection processing flow without S13 may be used. In that case, after a restored image is generated in S12, the process proceeds directly to S14. Then, in S14, the detection unit 204 uses the restored image of the inspection object and the normal restored image obtained by restoring the normal image to determine whether the inspection object is abnormal or not. In this anomaly detection processing flow, when anomaly detection processing is performed on the inspection object in the captured images shown in Figures 4(a), 5(a), and 6(a) in the same manner, since S13 is not included, in S14, the inspection object in the captured images shown in Figures 4(a) and 5(a) is determined to be normal, and the inspection object in the captured image shown in Figure 6(a) is determined to be abnormal.

As described above, the anomaly detection process flow of the anomaly detection system 100 according to this embodiment judges that there is no abnormality in the inspection target of the welding mark, which is normal with no variation in brightness, etc., as shown in FIG. 4(a), and that is normal but includes a portion with a different color locally due to variation in brightness, etc., as shown in FIG. 5(a), while judging that there is no abnormality in the inspection target of the welding mark, which includes a defective portion as shown in FIG. 6(a), and is abnormal. In other words, even if the above-mentioned problem of overdetection occurs due to variation in brightness or shape in the abnormality judgment (S13) by the preliminary detection unit 203, the occurrence of overdetection can be suppressed in the abnormality judgment (S14) by the detection unit 204, and the accuracy of the detection of the abnormality can be improved. Here, the anomaly detection process of the anomaly detection system 100 according to this embodiment is not limited to a particular inspection target, but is particularly suitable for inspection of weld marks that are prone to variation in shape and brightness, or for images of the inspection target captured on a production line where the position and orientation of the inspection target in the inspection target image is prone to shifts and variations due to issues with the imaging environment.

<Other embodiments>
In the first embodiment, a generative adversarial network, specifically, a GAN, has been described as an example of a learning model possessed by the restored image generating unit 202, but various generative adversarial networks other than GAN, such as CGAN, DCGAN, SRGAN, CycleGAN, VAEGAN, etc., can be used. Furthermore, various autoencoders, such as AE, VAE, and CVAE, can be used other than GAN.

The present invention is not limited to the above-described embodiment, and can be modified as appropriate without departing from the spirit and scope of the invention. For example, in addition to the anomaly detection system, the present invention can also be realized in various forms, such as an anomaly detection device, an anomaly detection method, and a computer program that realizes the method.

100 Anomaly detection system 101 CPU
102 ROM
103 RAM
104 HDD
105 Communication I/F
106 Input/Output I/F
107 Data bus 201 Acquisition unit 202 Reconstructed image generation unit 203 Preliminary detection unit 204 Detection unit

Claims (5)

an acquisition unit for acquiring an inspection object image of an inspection object;
a restored image generating unit that inputs the image of the test object to a learning model that has been trained using images of the test object that are normal as training data, and generates a restored image;
a preliminary detection unit that uses a difference between the inspection object image and the restored image to determine whether or not an inspection object in the inspection object image is abnormal;
a detection unit that, when the preliminary detection unit determines that the inspection target in the inspection target image is abnormal, determines whether or not the inspection target in the inspection target image is abnormal by using a similarity between the restored image and a normal restored image that is a restored image generated by inputting an image of the inspection target that is normal to the learning model;
An anomaly detection system comprising: The learning model is a learning model using an adversarial generative network.
The anomaly detection system according to claim 1 . The inspection object is a weld mark formed on a member by welding.
The anomaly detection system according to claim 1 or 2 . The inspection object image is an image captured on a production line.
The anomaly detection system according to any one of claims 1 to 3 . An acquisition step of acquiring an inspection object image of an inspection object;
a restored image generating step of inputting the inspection target image into a learning model trained using a normal inspection target image as training data, and generating a restored image;
a preliminary detection step of determining whether or not an object to be inspected in the image to be inspected is abnormal by using a difference between the image to be inspected and the restored image;
a detection step of judging whether or not the inspection object in the inspection object image is abnormal by using a similarity between the restored image and a normal restored image, which is a restored image generated by inputting an image of the inspection object that is normal to the learning model, when the inspection object in the inspection object image is judged to be abnormal in the preliminary detection step;
The anomaly detection method includes:

Images