JP7519232B2 - Anomaly detection method, anomaly detection device, and anomaly detection program - Google Patents

Description

The disclosed embodiments relate to an anomaly detection method, an anomaly detection device, and an anomaly detection program.

Conventionally, techniques for performing image analysis using machine learning are known. For example, the technique disclosed in Patent Document 1 performs machine learning using feature vectors calculated for multiple images whose classification results are known, and classifies images whose classification results are unknown based on the results obtained by the machine learning.

By using this technology, for example, it is possible to classify from an image of a manufactured product whether the product is normal or not, and if not, what is abnormal about it.

JP 2015-011552 A

However, the above-mentioned conventional techniques leave room for further improvement in terms of easily and accurately detecting anomalies using machine learning.

Specifically, for example, if the product abnormalities mentioned above include countless patterns such as scratches and uneven coloring, generating a classification model that can classify all of these patterns requires the collection of a large amount of training data and a long period of training, which is cumbersome.

In addition, during normal operation, it is easy to obtain normal images when things are normal, but it is difficult to obtain abnormal images when an abnormality occurs. Therefore, in the early stages of operation, it is difficult to sufficiently learn classification patterns, and the accuracy of the classification model cannot be guaranteed.

One aspect of the embodiment has been made in consideration of the above, and aims to provide an anomaly detection method, an anomaly detection device, and an anomaly detection program that can easily and accurately perform anomaly detection using machine learning.

The anomaly detection method according to one aspect of the embodiment includes a normal learning step, an evaluation step, a judgment step , and an abnormality learning step . The normal learning step is composed of a generative network and a discriminative network that compete with each other, and a skip connection model is arranged in the generative network, and a normal model is generated by modeling a normal image that is a training image that is a positive example by performing normal learning, which is machine learning, using a training image that is a positive example. The evaluation step calculates an abnormality degree of the judgment target image based on an output value of the normal model obtained by inputting a judgment target image to the normal model. The judgment step judges whether or not the judgment target image is abnormal based on the abnormality degree. The abnormality learning step generates an abnormality classification model that models a pattern of an abnormal image that is a training image that is a negative example by performing abnormality learning, which is machine learning, using a training image that is a negative example. Furthermore, the judgment step judges the pattern based on an output value of the abnormality classification model obtained by inputting the judgment target image to the abnormality classification model when the probability that the judgment target image is abnormal based on the output value of the normal model is smaller than a predetermined first threshold value.

According to one aspect of the embodiment, anomaly detection using machine learning can be performed easily and accurately.

FIG. 1 is a schematic explanatory diagram (part 1) of an anomaly detection method according to an embodiment. FIG. 2 is an explanatory diagram (part 1) of a case where the normal model is an autoencoder. FIG. 3 is an explanatory diagram (part 2) of the case where the normal model is an autoencoder. FIG. 4 is an explanatory diagram (part 1) of a case where the normal model is a skip connection model. FIG. 5 is an explanatory diagram (part 2) of the case where the normal model is a skip connection model. FIG. 6 is a schematic explanatory diagram (part 2) of the anomaly detection method according to the embodiment. FIG. 7 is a schematic explanatory diagram (part 3) of the anomaly detection method according to the embodiment. FIG. 8 is a schematic explanatory diagram (part 4) of the anomaly detection method according to the embodiment. FIG. 9 is a schematic explanatory diagram (part 5) of the anomaly detection method according to the embodiment. FIG. 10 is a block diagram illustrating an example of the configuration of an anomaly detection device according to an embodiment. FIG. 11 is a diagram (part 1) showing a specific example of the detection result. FIG. 12 is a diagram (part 2) showing a specific example of the detection result. FIG. 13 is a diagram (part 3) showing a specific example of the detection result. FIG. 14 is a diagram (part 4) showing a specific example of the detection result. FIG. 15 is a diagram (part 5) showing a specific example of the detection result. FIG. 16 is a diagram (part 6) showing a specific example of the detection result. FIG. 17 is a flowchart (part 1) illustrating a processing procedure executed by the anomaly detection device according to the embodiment. FIG. 18 is a flowchart (part 2) illustrating the processing procedure executed by the anomaly detection device according to the embodiment. FIG. 19 is a hardware configuration diagram illustrating an example of a computer that realizes the functions of the anomaly detection device.

Below, the embodiments of the anomaly detection method, anomaly detection device, and anomaly detection program disclosed in the present application will be described in detail with reference to the attached drawings. Note that the present invention is not limited to the embodiments described below.

First, an overview of the anomaly detection method according to the embodiment will be described with reference to Figs. 1 to 9. Fig. 1 is an overview diagram (part 1) of the anomaly detection method according to the embodiment. Figs. 2 and 3 are explanatory diagrams (part 1) and (part 2) of the case where the normal model is an autoencoder. Figs. 4 and 5 are explanatory diagrams (part 1) and (part 2) of the case where the normal model is a skip connection model. Figs. 6 to 9 are explanatory diagrams (part 2) to (part 5) of the anomaly detection method according to the embodiment.

In the anomaly detection method according to the embodiment, basically, only normal data is learned, and anomalies are detected using a normal model generated as a result of the learning. Because only normal data is learned, there is no need to collect abnormal data or perform so-called annotation work to teach the type and location of anomalies. Furthermore, compared to methods that do not learn abnormal data, there is a high possibility that unexpected and unknown anomalies can be detected.

Specifically, as shown in FIG. 1, in the anomaly detection method according to the embodiment, first, machine learning is performed using only "normal images," which are images of normal products, as a learning dataset (step S1), and a normal model 124 is generated.

Note that even in a "normal image," there may be variation in illuminance, etc., so as shown in FIG. 1, it is preferable to prepare multiple normal patterns for the "normal image." This makes it possible to generate a normal model 124 that is robust to variations in illuminance, etc.

In the anomaly detection method according to the embodiment, the "image to be judged" is input to the generated normal model 124, and the error from normality (hereinafter referred to as the "degree of abnormality") and the accuracy of the degree of abnormality are calculated based on the output value output by the normal model 124 (step S2). If the degree of abnormality is large, it is judged to be abnormal (step S3).

In other words, in the anomaly detection method according to the embodiment, normality or abnormality is determined using a normal model 124 based only on "normal images." In other words, in the anomaly detection method according to the embodiment, anomalies are detected based on the degree of anomaly, which is the error from normality, so it is possible to detect unknown anomalies as anomalies even without learning a wide variety of abnormality patterns. This makes it possible to determine anomalies using the normal model 124 even in the early stages of operation when there is little learning data.

As will be described later using FIG. 8 etc., in the anomaly detection method according to the embodiment, when it is difficult to make a judgment using the normal model 124, for example when the aforementioned accuracy is low, an abnormal target area, which is a difference part between the judgment target image and the normal image, is extracted. Based on such an abnormal target area, an abnormality classification process can be performed using the anomaly classification model 125 described later. This point will be described in the explanation of the anomaly classification process shown in FIG. 8.

In addition, it may be possible to determine that there is a possibility of overdetection based on the extracted anomaly determination area, the results of the anomaly classification process, and the knowledge of the person visually observing the area. In such cases, the anomaly detection method according to the embodiment can perform a re-determination process using an overdetection model 126 that distinguishes between an anomaly and an overdetection. This point will be described later with reference to FIGS. 8 and 9.

Incidentally, it is widely known to use an autoencoder as the normal model 124. An autoencoder that has learned normal images reconstructs the input image to be judged, and anomalies can be detected by calculating the difference between the image to be judged and the reconstructed data (corresponding to the degree of anomaly mentioned above).

As shown in Figure 2, an autoencoder is one of the neural network mechanisms in deep learning, and is a neural network consisting of an encoder and a decoder.

The encoder performs dimensional compression (downsampling) on the input data through one or more intermediate layers that perform convolution, extracting a feature map (a compressed feature representation, also called a latent variable). The decoder reconstructs (upsampling) the input data from the feature map through one or more intermediate layers, and outputs it as output data.

As shown in Figure 2, in an encoder, local features are extracted from intermediate layers closer to the input, while global features are extracted from intermediate layers farther from the input. Local features relate to color, edges, etc. Global features, on the other hand, relate to shape, etc.

However, with autoencoders, the data reconstructed after dimensional compression becomes average, and in the case of images, it has the drawback of being blurred, as shown in Figure 3. For this reason, as shown in Figure 3, even when a normal image is input as the image to be judged, a certain degree of abnormality will occur. This creates problems in terms of detection accuracy.

In addition to such autoencoders, the skip connection model (hereinafter sometimes referred to as "U-Net") is also known. As shown in Figure 4, the skip connection model is a model that introduces skip connections that connect the feature maps output in each hidden layer of the encoder to the feature maps of the corresponding hidden layers in the decoder. Such skip connections enable clear reconstruction of data without blurring.

However, as shown in Figure 5, in the skip connection model, it is easy to reconstruct input data as an identity map. Therefore, if an abnormal image is also reconstructed using an identity map, the degree of abnormality will be almost 0. This makes it difficult to detect anomalies.

Therefore, in the anomaly detection method according to the embodiment, a normal model 124 is generated using a generative adversarial network (GAN) consisting of a competing generative network and discriminative network, with a skip connection model placed in the generative network.

Specifically, as shown in FIG. 6, in the anomaly detection method according to the embodiment, the normal learning unit 112 of the anomaly detection device 10 (see FIG. 10) is composed of a generation network 112a and a discrimination network 112b, and generates a normal model 124 by a GAN in which a skip connection model 112aa is arranged in the generation network 112a.

Basic GANs do not have an encoder and do not have the concept of skip connection. Although there are examples of applying GANs to anomaly detection (AnoGAN, Efficient GAN, etc.), they are not skip connection models and reconstruction errors for local features are thought to be large.

In contrast, in the anomaly detection method according to the embodiment, the identity mapping during reconstruction is suppressed by incorporating a GAN learning algorithm into the skip connection model 112aa having an encoder. Specifically, the generative network 112a generates a false image based on a normal image.

The discrimination network 112b also inputs normal and fake images alternately, for example, and discriminates between the authenticity and falsity of the input images. The discrimination results are fed back to the generative network 112a and the discrimination network 112b, so that the generative network 112a learns to deceive the discrimination network 112b, and the discrimination network 112b learns to discriminate more accurately.

Then, learning is repeated until the discrimination network 112b has a 50/50 probability of making a judgment on the false images generated by the generation network 112a. Then, the discrimination network 112b at the stage where learning is completed becomes the normal model 124.

As shown in FIG. 6, the normal learning unit 112 further includes an Attention mechanism 112ab. The Attention mechanism 112ab is a mechanism that weights the attention area of an image. The Attention mechanism 112ab weights the attention area, for example, by multiplying the feature map by the Attention map.

The attention mechanism 112ab weights the attention area so as to enhance the effect of combining global features during reconstruction. The attention mechanism 112ab may be operated by a user's operation or may be operated automatically. When operating automatically, a known algorithm such as Self-Attention GAN can be used.

As shown in FIG. 6, the normal learning unit 112 further includes a noise addition unit 112ac. The noise addition unit 112ac adds noise to the normal image as self-supervised learning, and learns to restore the noise. For example, the normal learning unit 112 causes the noise addition unit 112ac to add noise so as to hide a part of the normal image, and learns to reproduce the hidden part. This suppresses the identity mapping, and improves the generalizability of the model through the effect of data expansion.

By using the normal model 124 generated by such normal learning processing executed by the normal learning unit 112, as shown in FIG. 7, when a normal image is input, the normal image is clearly reconstructed and the degree of abnormality is approximately 0. On the other hand, when an abnormal image is input, the identity mapping is suppressed, the normal image is clearly reconstructed, and the degree of abnormality is also clearly derived. This makes it possible to perform anomaly detection with high accuracy.

As already mentioned, in the anomaly detection method according to the embodiment, when it is difficult to make a judgment using the normal model 124, for example when the aforementioned accuracy is low, an abnormal target region, which is the difference between the target image and the normal image, is extracted. Based on such an abnormal target region, an anomaly classification process can be performed using the anomaly classification model 125.

Specifically, as shown in FIG. 8, in the anomaly detection method according to the embodiment, machine learning is performed using anomaly target regions R1, R2, R3, etc. of images of previously detected defective products (i.e., "anomaly images") as a learning dataset (step S11), and an anomaly classification model 125 is generated.

In other words, in the anomaly detection method according to the embodiment, detection accuracy is improved by using not only the normal model 124 that models a normal image, but also the anomaly classification model 125 that models an abnormal pattern based on an abnormal image from a past anomaly detection.

Here, as shown in the figure, the "abnormal images" are clustered according to abnormality patterns, such as "character errors," "missing parts," and "protrusions." The abnormality classification model 125 generated based on such abnormal images can model the abnormality patterns of defective products, so to speak.

In the anomaly detection method according to the embodiment, the anomaly classification model 125 receives, for example, an abnormal target region RX of the "image to be determined" that has been determined to be abnormal based on the normal model 124. As a result, the anomaly classification model 125 calculates the classification result (for example, the "class ID" in the figure) and the accuracy of the classification result (step S12).

Here, if the accuracy of the classification result is high, it is highly likely to be a known abnormality pattern, so in the anomaly detection method according to the embodiment, the classification result obtained from the anomaly classification model 125 is notified (step S13).

On the other hand, if the accuracy of the classification result is low, it may be an unknown abnormal pattern. In such a case, the anomaly detection method according to the embodiment can add a classification class for a new abnormal pattern based on human knowledge, etc., perform additional learning using the corresponding abnormality target area, and update the anomaly classification model 125. This makes it possible to improve the accuracy of the anomaly classification model 125 as needed even during operation.

Furthermore, for example, if the accuracy of the classification result is too low, there is a possibility of overdetection, in which normality is detected as an abnormality. In such a case, the anomaly detection method according to the embodiment can perform a re-evaluation using the overdetection model 126 (step S14).

As shown in FIG. 9, in the anomaly detection method according to the embodiment, for example, machine learning is performed using previously detected anomaly images and overdetection images as a learning dataset (step S21), and an overdetection model 126 is generated. As already mentioned, the overdetection model 126 is an identification model that distinguishes between an anomaly and an overdetection.

Then, the overdetection model 126 is input with a judgment target image that has been judged to be abnormal based on the anomaly classification model 125 but with too low a degree of accuracy. As a result, the overdetection model 126 outputs whether the image is abnormal or an overdetection (step S22).

In this way, in the anomaly detection method according to the embodiment, the normal learning unit 112 is composed of a generation network 112a and a discrimination network 112b, and a normal model 124 is generated by a GAN in which a skip connection model 112aa is placed in the generation network 112a.

Therefore, the anomaly detection method according to the embodiment can easily and accurately detect anomalies using machine learning.

The configuration of the anomaly detection device 10 according to the embodiment to which the above-mentioned anomaly detection method is applied will be described in more detail below.

Figure 10 is a block diagram showing an example of the configuration of an anomaly detection device 10 according to an embodiment. Note that in Figure 10, components necessary for explaining the features of this embodiment are shown as functional blocks, and descriptions of general components are omitted.

In other words, each component shown in FIG. 10 is a functional concept, and does not necessarily have to be physically configured as shown. For example, the specific form of distribution and integration of each functional block is not limited to that shown, and all or part of it can be functionally or physically distributed and integrated in any unit depending on various loads, usage conditions, etc.

Note that in the explanation using Figure 10, the explanation of components that have already been described may be simplified or omitted.

As shown in FIG. 10, the anomaly detection device 10 includes a control unit 11 and a memory unit 12. The control unit 11 includes an acquisition unit 111, a normal learning unit 112, an abnormality learning unit 113, an overdetection learning unit 114, an evaluation unit 115, and a judgment unit 116.

The storage unit 12 is a storage device such as a hard disk drive, non-volatile memory, or a register, and stores a normal learning dataset 121, an abnormal learning dataset 122, an overdetection learning dataset 123, a normal model 124, an abnormality classification model 125, an overdetection model 126, and evaluation information 127. The evaluation information 127 includes anomaly degree information 127a, a classification result 127b, and accuracy information 127c.

The control unit 11 performs overall control of the anomaly detection device 10. The acquisition unit 111 acquires clustered learning images. The learning images include normal images for normal learning to generate a normal model 124, abnormal images for abnormal learning to generate an anomaly classification model 125, and abnormal images and overdetection images for overdetection learning to generate an overdetection model 126.

The acquisition unit 111 also acquires anomaly target regions in anomaly learning. Such regions can be specified, for example, from an operation unit 20 such as a keyboard, mouse, or touch panel connected to the anomaly detection device 10. Note that information related to identifying anomaly target regions may be stored in advance in the storage unit 12, etc., and the acquisition unit 111 may extract anomaly target regions from such information. Each anomaly target region is associated with a classification class.

The acquisition unit 111 also stores normal images from the acquired learning images in a normal learning dataset 121. The acquisition unit 111 also stores abnormal target regions from the abnormal images in an abnormal learning dataset 122. The acquisition unit 111 also stores abnormal images and overdetection images from the acquired learning images in an overdetection learning dataset 123.

When the acquisition unit 111 receives a judgment result indicating that additional learning is required from the judgment unit 116 described below, the acquisition unit 111 acquires the abnormal target region RX of the corresponding abnormal image for additional learning and stores it in the abnormality learning dataset 122.

The normal learning unit 112 acquires the normal learning dataset 121 and executes a normal learning process to generate a normal model 124 by machine learning using the normal learning dataset. The normal learning process according to this embodiment has already been described using FIG. 6, so a description thereof will be omitted here.

The anomaly learning unit 113 acquires an anomaly learning dataset 122 and executes an anomaly learning process that uses the dataset to generate an anomaly classification model 125 through machine learning. The anomaly learning process according to this embodiment has already been described using FIG. 8, so a description thereof will be omitted here.

The overdetection learning unit 114 acquires an overdetection learning dataset 123 and executes an overdetection learning process to generate an overdetection model 126 by machine learning using the dataset. The overdetection learning process according to this embodiment has already been described using FIG. 9, so a description thereof will be omitted here.

Note that algorithms other than deep learning may be used for the anomaly learning process and the overdetection learning process. For example, machine learning may be performed by a regression analysis method such as support vector regression using a pattern classifier such as an SVM (Support Vector Machine) to generate the anomaly classification model 125 and the overdetection model 126. Here, the pattern classifier is not limited to an SVM, and may be, for example, AdaBoost. Random forests, etc. may also be used.

In addition, the normal learning unit 112, the abnormal learning unit 113, and the overdetection learning unit 114 can each receive input of various parameters from the operation unit 20 before or during the execution of each learning process, and execute each learning process that reflects this.

The evaluation unit 115 acquires the image to be judged, inputs it to the normal model 124, and receives the output result from the normal model 124. The evaluation unit 115 then calculates the degree of abnormality and accuracy of the image to be judged based on the received output result, and stores them in the abnormality degree information 127a and accuracy information 127c. The degree of abnormality is, for example, the sum of the deviation from the normal state for each pixel of the image to be judged.

When the evaluation unit 115 receives a judgment result from the judgment unit 116 (described later) indicating that the object is abnormal according to the normal model 124 but the accuracy is lower than a predetermined first threshold, the evaluation unit 115 inputs the abnormal target region RX included in the judgment result to the abnormality classification model 125 and receives the output result from the abnormality classification model 125. The evaluation unit 115 then stores the received output result in the classification result 127b and accuracy information 127c.

When the evaluation unit 115 receives a judgment result from the judgment unit 116 indicating that the image is abnormal according to the normal model 124 or the anomaly classification model 125 but has a lower probability of being abnormal than a predetermined second threshold that is even lower than the first threshold, the evaluation unit 115 inputs the image to be judged to the overdetection model 126 and receives the output result from the overdetection model 126. The evaluation unit 115 then notifies the judgment unit 116 of the received output result.

The judgment unit 116 refers to the abnormality degree information 127a and the accuracy information 127c, and if the degree of abnormality based on the normal model 124 is greater than a predetermined judgment threshold and the accuracy is greater than a predetermined first threshold, notifies the abnormality based on the normal model 124 via a display unit 30 or the like connected to the anomaly detection device 10.

In addition, when the degree of abnormality according to the normal model 124 is greater than a predetermined judgment threshold but the accuracy is less than a predetermined first threshold, i.e., when judgment is difficult using the normal model 124, the judgment unit 116 extracts an abnormal target area RX from the judgment target image, and notifies the acquisition unit 111 of the judgment result that the image is abnormal according to the normal model 124, but the accuracy is low, including the abnormal target area RX.

The determination unit 116 also refers to the classification result 127b and the accuracy information 127c, and when the accuracy based on the anomaly classification model 125 is greater than a predetermined first threshold, notifies the user of the abnormality based on the anomaly classification model 125 and the classification result 127b via the display unit 30, etc.

In addition, when the accuracy according to the abnormality classification model 125 is smaller than a predetermined first threshold and is also smaller than a predetermined second threshold that is smaller than the first threshold, the determination unit 116 notifies the acquisition unit 111 of the determination result that the abnormality is determined to be abnormal according to the normal model 124 or the abnormality classification model 125 but the accuracy is smaller than the second threshold.

When the determination unit 116 receives the output result of the overdetection model 126 from the evaluation unit 115, it notifies the user via the display unit 30 or the like whether the output result is an abnormality or an overdetection.

In addition, when the accuracy based on the anomaly classification model 125 is smaller than a predetermined first threshold but is greater than a predetermined second threshold that is smaller than the first threshold, the determination unit 116 notifies the acquisition unit 111 of the determination result that additional learning is required, and causes the acquisition unit 111 to acquire the abnormal target region RX of the corresponding abnormal image for additional learning of the anomaly classification model 125.

Based on this notification, the acquisition unit 111 accepts additional settings for the classification class of the new abnormality pattern via the operation unit 20, etc. The abnormality learning unit 113 executes abnormality learning based on the abnormality target area RX for such additional learning and the contents of the additional settings, and updates the abnormality classification model 125.

Next, to make the above explanation easier to understand, specific examples of detection results using the anomaly detection method according to the embodiment are shown in Figs. 11 to 16. Figs. 11 to 16 are diagrams (part 1) to (part 6) showing specific examples of detection results.

As described above, to generate the normal model 124, normal images of "normal products" are used as shown in FIG. 11. That is, the normal learning dataset 121 stores normal images as shown in FIG. 11, and the normal learning unit 112 uses these to generate the normal model 124.

As already mentioned, it is preferable to prepare multiple normal patterns, such as normal images with different illuminances. This makes it possible to generate a normal model 124 that is robust to variations in illuminance.

The difference is not limited to illuminance, but may be brightness, saturation, position, angle, etc. Also, while FIG. 11 shows a gear as an example, the object of anomaly detection is not limited, and it may be sweets, or as will be shown later, the arrangement of bento boxes, various ingredients, printed content, etc.

Here, assume that there is a defective product with a "chipped" part, as shown as part M1 in FIG. 12. When an image of such a defective product is input as the image to be judged, the evaluation unit 115 inputs the image to the normal model 124, and based on the output value, the area corresponding to the position of the chip can be extracted as the abnormal target area RX, as shown in the "detection result" in the same figure.

Also, Figure 13 shows a defective product with a "scratch" (see part M2 in the figure). When an image of such a defective product is input as the image to be judged, the area corresponding to the position of the scratch can be extracted as the abnormal target area RX, as shown in the "detection result" in the figure.

Figure 14 shows a case where abnormalities in the presentation of a bento box are to be detected. In this case, when an image of a defective product, for example a pickled plum that is "missing" (i.e., left behind) is input as the image to be judged, the area corresponding to the position where the pickled plum should be placed can be extracted as the abnormal target area RX, as shown in the figure.

Figure 15 shows a case where the detection target is an abnormality in various materials, in this case, the same material placed in a box in multiple locations. In such a case, if an image containing a "discolored" material is input as the image to be judged, the area corresponding to the position of the discolored material can be extracted as the abnormal target area RX, as shown in the figure.

Figure 16 shows a case where abnormalities in the printed content are the object of detection. In this case, if an image of a defective product, for example, with "rubbed" print, is input as the image to be judged, compared to a normal product, the area corresponding to the position where the rubbed print is located can be extracted as the abnormal target area RX, as shown in the figure.

Next, the processing procedure executed by the anomaly detection device 10 according to the embodiment will be described with reference to FIG. 17 and FIG. 18. FIG. 17 and FIG. 18 are flowcharts (part 1) and (part 2) showing the processing procedure executed by the anomaly detection device 10 according to the embodiment.

As shown in FIG. 17, first, it is determined whether or not it is the learning period (step S101). The learning period corresponds to, for example, the first operation of the anomaly detection device 10, or a predetermined period when the normal model 124, the anomaly classification model 125, and the overdetection model 126 are created in advance before operation.

If it is the learning period (step S101, Yes), the normal learning unit 112 generates a normal model 124 from the learning dataset of normal images (i.e., the normal learning dataset 121) using a GAN in which the skip connection model 112aa is placed in the generation network 112a (step S102).

The anomaly learning unit 113 also generates an anomaly classification model 125 by machine learning from the learning dataset for the anomaly target area (i.e., the anomaly learning dataset 122) (step S103).

The overdetection learning unit 114 also generates an overdetection model 126 by machine learning from a learning dataset of abnormal images and overdetection images (i.e., the overdetection learning dataset 123) (step S104).

Next, or if it is not the learning period in step S101 (step S101, No), proceed to step S105 (see Figure 18).

In step S105, as shown in FIG. 18, the evaluation unit 115 inputs the image to be judged into the normal model 124 and calculates the degree of abnormality and the accuracy (step S105). Then, the judgment unit 116 judges whether the degree of abnormality is greater than a predetermined judgment threshold (step S106).

If the degree of abnormality is greater than the predetermined judgment threshold (step S106, Yes), the judgment unit 116 judges whether the certainty is greater than a predetermined first threshold (step S107). If the degree of abnormality is less than the predetermined judgment threshold (step S106, No), the process ends.

Next, if the accuracy is greater than a predetermined first threshold (step S107, Yes), the judgment unit 116 notifies the normal model 124 of an abnormality (step S108) and terminates the process.

If the accuracy is less than the first threshold (step S107, No), the determination unit 116 extracts an abnormal target region RX from the image to be determined (step S109). The evaluation unit 115 then inputs the abnormal target region RX to the abnormality classification model 125 and calculates the classification result and accuracy (step S110).

Then, the determination unit 116 determines whether the accuracy is greater than the first threshold value (step S111). If the accuracy is greater than the first threshold value (step S111, Yes), the determination unit 116 notifies the anomaly and classification result according to the anomaly classification model 125 (step S112), and ends the process.

On the other hand, if the certainty is smaller than the first threshold (step S111, No), the determination unit 116 then determines whether the certainty is smaller than a predetermined second threshold that is even smaller than the first threshold (step S113).

Here, if the accuracy is less than the second threshold (step S113, Yes), the evaluation unit 115 inputs the image to be judged to the overdetection model 126, and the judgment unit 116 performs a re-judgment based on the output result of the overdetection model 126 (step S114). Then, the judgment unit 116 notifies the re-judgment result (step S115), and ends the process.

On the other hand, if the accuracy is greater than the second threshold (step S113, No), the acquisition unit 111 accepts additional setting of the classification class of the corresponding abnormal target area RX via the operation unit 20, etc. (step S116).

Then, the anomaly learning unit 113 updates the anomaly classification model 125 through machine learning using the corresponding anomaly target area RX as additional learning (step S117), and ends the process.

The anomaly detection device 10 according to the embodiment described above is realized by a computer 60 having a configuration as shown in FIG. 19, for example. FIG. 19 is a hardware configuration diagram showing an example of a computer that realizes the functions of the anomaly detection device 10. The computer 60 includes a CPU (Central Processing Unit) 61, a RAM (Random Access Memory) 62, a ROM (Read Only Memory) 63, a HDD (Hard Disk Drive) 64, a communication interface (I/F) 65, an input/output interface (I/F) 66, and a media interface (I/F) 67.

The CPU 61 operates based on a program stored in the ROM 63 or the HDD 64, and controls each part. The ROM 63 stores a boot program executed by the CPU 61 when the computer 60 starts up, programs that depend on the hardware of the computer 60, etc.

The HDD 64 stores the programs executed by the CPU 61 and the data used by the programs. The communication interface 65 receives data from other devices via the communication network and sends it to the CPU 61, and transmits data generated by the CPU 61 to other devices via the communication network.

The CPU 61 controls output devices such as a display and a printer, and input devices such as a keyboard and a mouse, via the input/output interface 66. The CPU 61 acquires data from the input devices via the input/output interface 66. The CPU 61 also outputs the data it generates to the output devices via the input/output interface 66.

The media interface 67 reads the program or data stored in the recording medium 68 and provides it to the CPU 61 via the RAM 62. The CPU 61 loads the program from the recording medium 68 onto the RAM 62 via the media interface 67 and executes the loaded program. The recording medium 68 is, for example, an optical recording medium such as a DVD (Digital Versatile Disc) or a PD (Phase change rewritable Disc), a magneto-optical recording medium such as an MO (Magneto-Optical disk), a tape medium, a magnetic recording medium, or a semiconductor memory.

For example, when the computer 60 functions as the anomaly detection device 10, the CPU 61 of the computer 60 executes an anomaly detection program loaded onto the RAM 62, thereby implementing each function of the control unit 11. In addition, the data in the storage unit 12 is stored in the HDD 64. The CPU 61 of the computer 60 reads and executes these programs from the recording medium 68, but as another example, the CPU 61 may obtain these programs from another device via a communication network.

As described above, the anomaly detection device 10 according to the embodiment includes a normal learning unit 112, an evaluation unit 115, and a judgment unit 116. The normal learning unit 112 is composed of a generative network 112a and a discriminative network 112b that compete with each other, and generates a normal model 124 that models a normal image that is a training image that is a positive example by executing normal learning, which is machine learning using a training image that is a positive example, by an adversarial generative network in which a skip connection model 112aa is arranged in the generative network 112a. The evaluation unit 115 calculates the degree of abnormality of the judgment target image based on the output value of the normal model 124 obtained by inputting the judgment target image to the normal model 124. The judgment unit 116 judges whether the judgment target image is abnormal or not based on the degree of abnormality.

Therefore, the anomaly detection device 10 according to this embodiment can easily and accurately detect anomalies using machine learning.

In the above embodiment, an example of combining the normal model 124, the anomaly classification model 125, and the overdetection model 126 has been described, but it is not necessary to combine the three at the same time. For example, anomaly detection may be performed basically by combining the normal model 124 and the anomaly classification model 125, with the overdetection model 126 being selectable as an option. Also, for example, the normal model 124 and the overdetection model 126 may be basically combined, with the anomaly classification model 125 being selectable as an option.

Further advantages and modifications may readily occur to those skilled in the art. Therefore, the invention in its broader aspects is not limited to the specific details and representative embodiments shown and described above. Accordingly, various modifications may be made without departing from the spirit or scope of the general inventive concept as defined by the appended claims and equivalents thereof.

REFERENCE SIGNS LIST 10 Anomaly detection device 11 Control unit 111 Acquisition unit 112 Normal learning unit 113 Anomaly learning unit 114 Overdetection learning unit 115 Evaluation unit 116 Judgment unit 12 Storage unit 121 Normal learning data set 122 Anomaly learning data set 123 Overdetection learning data set 124 Normal model 125 Anomaly classification model 126 Overdetection model 127 Evaluation information 127a Anomaly degree information 127b Classification result 127c Accuracy information

Claims (8)

A normal learning process for generating a normal model by performing normal learning, which is machine learning using training images that are positive examples, using a generative adversarial network that is composed of a generative network and a discriminative network that compete with each other and in which a skip connection model is arranged in the generative network, and generating a normal model that models the normal images that are training images that are positive examples;
an evaluation step of calculating an abnormality degree of the target image based on an output value of the normal model obtained by inputting the target image to the normal model;
a determination step of determining whether the target image is abnormal or not based on the degree of abnormality ;
an anomaly learning process for generating an anomaly classification model that models a pattern of an abnormal image that is a learning image serving as a negative example by executing anomaly learning, which is a machine learning process using learning images serving as negative examples;
Including,
The determination step includes:
When the probability that the target image is abnormal based on the output value of the normal model is smaller than a predetermined first threshold, the pattern is determined based on the output value of the abnormality classification model obtained by inputting the target image to the abnormality classification model.
The anomaly detection method according to the present invention is characterized in that A normal learning process for generating a normal model by performing normal learning, which is machine learning using training images that are positive examples, using a generative adversarial network that is composed of a generative network and a discriminative network that compete with each other and in which a skip connection model is arranged in the generative network, and generating a normal model that models the normal images that are training images that are positive examples;
an evaluation step of calculating an abnormality degree of the target image based on an output value of the normal model obtained by inputting the target image to the normal model;
a determination step of determining whether the target image is abnormal or not based on the degree of abnormality;
An overdetection learning process for generating an overdetection model for distinguishing between an anomaly and an overdetection by executing overdetection learning, which is machine learning using learning images that are negative examples and overdetection examples;
Including,
The determination step includes:
When the probability that the image to be determined is abnormal based on the output value of the normal model is smaller than a predetermined second threshold, the image to be determined is determined to be abnormal or overdetected based on the output value of the overdetection model obtained by inputting the image to be determined into the overdetection model.
The anomaly detection method according to the present invention is characterized in that The generating network includes an Attention mechanism,
The normal learning step includes:
The method according to claim 1 or 2, further comprising weighting the attention region of the normal image using the attention mechanism such that at least global features of the normal image are combined in the skip connection model. The generator network includes a noise adder;
The normal learning step includes:
The anomaly detection method described in claim 1 , 2 or 3, characterized in that noise is added to the normal image using the noise addition unit as self-supervised learning, and normal learning is performed so that the noise is restored in the skip connection model. a normal learning unit that generates a normal model by modeling a normal image, which is a training image serving as a positive example, by executing normal learning, which is machine learning using a training image serving as a positive example, using an adversarial generative network that is composed of a generative network and a discriminative network that compete with each other and has a skip connection model arranged in the generative network;
an evaluation unit that calculates an abnormality degree of the target image based on an output value of the normal model obtained by inputting the target image to the normal model;
a determination unit that determines whether the determination target image is abnormal or not based on the degree of abnormality ;
an anomaly learning unit that generates an anomaly classification model that models a pattern of an abnormal image that is a learning image serving as a negative example by executing anomaly learning, which is a machine learning method using the learning image serving as a negative example;
Equipped with
The determination unit is
When the probability that the target image is abnormal based on the output value of the normal model is smaller than a predetermined first threshold, the pattern is determined based on the output value of the abnormality classification model obtained by inputting the target image to the abnormality classification model.
An anomaly detection device comprising : a normal learning unit that generates a normal model by modeling a normal image, which is a training image serving as a positive example, by executing normal learning, which is machine learning using a training image serving as a positive example, using an adversarial generative network that is composed of a generative network and a discriminative network that compete with each other and has a skip connection model arranged in the generative network;
an evaluation unit that calculates an abnormality degree of the target image based on an output value of the normal model obtained by inputting the target image to the normal model;
a determination unit that determines whether the determination target image is abnormal or not based on the degree of abnormality;
An overdetection learning unit that generates an overdetection model that distinguishes between an anomaly and an overdetection by executing overdetection learning, which is a machine learning method using learning images that are negative examples and overdetection examples;
Equipped with
The determination unit is
When the probability that the image to be determined is abnormal based on the output value of the normal model is smaller than a predetermined second threshold, the image to be determined is determined to be abnormal or overdetected based on the output value of the overdetection model obtained by inputting the image to be determined into the overdetection model.
An anomaly detection device comprising: A normal learning procedure in which a machine learning normal learning is performed using training images that are positive examples using a generative network and a discriminative network that compete with each other, the generative network being configured with a skip connection model, to generate a normal model that models the normal images that are training images that are positive examples;
An evaluation procedure for calculating an abnormality degree of the target image based on an output value of the normal model obtained by inputting the target image to the normal model;
a determination step of determining whether the target image is abnormal or not based on the degree of abnormality ;
an anomaly learning procedure for generating an anomaly classification model that models a pattern of an abnormal image that is a negative example learning image by executing anomaly learning, which is a machine learning procedure using a negative example learning image;
Run the following on your computer :
The determination procedure includes:
When the probability that the target image is abnormal based on the output value of the normal model is smaller than a predetermined first threshold, the pattern is determined based on the output value of the abnormality classification model obtained by inputting the target image to the abnormality classification model.
An anomaly detection program comprising : A normal learning procedure in which a normal model is generated by modeling normal images, which are training images serving as positive examples, by executing normal learning, which is machine learning using a generative adversarial network consisting of a generative network and a discriminative network that compete with each other and in which a skip connection model is arranged in the generative network;
An evaluation procedure for calculating an abnormality degree of the target image based on an output value of the normal model obtained by inputting the target image to the normal model;
a determination step of determining whether the target image is abnormal or not based on the degree of abnormality;
An overdetection learning procedure that generates an overdetection model that can distinguish between an anomaly and an overdetection by performing overdetection learning, which is a machine learning method using learning images that are negative examples and overdetection examples;
Run the following on your computer:
The determination procedure includes:
When the probability that the image to be determined is abnormal based on the output value of the normal model is smaller than a predetermined second threshold, the image to be determined is determined to be abnormal or overdetected based on the output value of the overdetection model obtained by inputting the image to be determined into the overdetection model.
An anomaly detection program comprising:

Images