JP7112924B2 - Information processing device, information processing method, and program - Google Patents

Description

Application of Article 30, Paragraph 2 of the Patent Act Business Communication Co., Ltd. Business Communication August vol. 1st of month

The present invention relates to an information processing device, an information processing method, and a program.

For example, there is a technique of recognizing an object included in an image by performing image recognition on the image. In this image recognition technology, in recent years, a deep learning technology in which a neural network is multi-layered is used to recognize an object with high accuracy. This deep learning technology generally recognizes an object with high accuracy by stacking multiple layers in an intermediate layer between an input layer and an output layer of a neural network.

Detection patterns, classification patterns, and level determination patterns are examples of patterns for determining a target object using the above-described deep learning technology. A detection type determiner detects the type, size, and certainty of a target object. A classifier determines whether a target object is abnormal or normal. A level determination type determiner expresses, for example, the degree of inappropriateness of an image by a number.

In order to perform recognition processing, any of the decision devices collects normal data obtained by imaging the target object and abnormal data obtained by imaging the abnormal state of the target object as teacher data, and inputs the collected teacher data to the learning model. There is a need. Then, for example, when normal data is input, it is determined to be normal (the degree of normality is high), and when abnormal data is input, it is determined to be abnormal (the degree of abnormality is high) in the learning model. Processing parameters had to be learned.

JP 2012-073761 A

However, there are cases where it is difficult to collect an abnormal target object, and in this case, the number of collected abnormal data is smaller than the number of collected normal data. may not be accurate enough.

The present invention has been made in view of the above problems, and provides an information processing device, an information processing method, and a program that can improve the accuracy of determination processing using a feature amount of data representing a normal state. intended to provide.

(1) One aspect of the present invention is an acquisition unit that acquires a plurality of feature amounts related to a normal determination target; A feature quantity in the feature space converted by the conversion unit by receiving information representing an anomaly detection rate and/or a normal detection rate based on a user's operation, and a conversion unit for converting a feature space of a plurality of feature quantities a setting unit for setting a range for judging that the is normal ; a determination target acquiring unit that converts the feature space into the feature space transformed by the transforming unit ; When determining that the feature amount acquired by the target acquisition unit is normal and the feature amount acquired by the determination target acquisition unit is not within the range set by the setting unit, the determination target acquisition unit acquires a determination unit that determines that the extracted feature amount is abnormal , wherein the feature amount extraction unit determines that the determination target is normal when determination target data related to the normal determination target is input. The information processing apparatus is a feature amount extractor that extracts a feature amount to be given to a classifier that determines whether or not the determination target is normal from the recognition model whose processing parameters have been learned as described above .

( 2 ) An aspect of the present invention is the information processing apparatus described above, wherein the determination target data relating to the normal determination target is divided into a plurality of divided data according to a predetermined rule, and each of the divided data is divided into a learning processing unit that learns processing parameters of a recognition model using a learning processing unit, wherein the feature amount extraction unit divides the determination target data according to the predetermined rule, and uses each of the divided data to obtain the determination target feature The determination unit performs determination using the feature amount extracted using each of the divided data based on the recognition model.

( 3 ) One aspect of the present invention is a step in which an information processing apparatus acquires a plurality of feature amounts relating to a normal determination target, and the information processing apparatus selects a plurality of feature amounts from the acquired plurality of feature amounts. Then, the step of transforming the feature space of the plurality of selected feature amounts , and the information processing device receiving information representing the anomaly detection rate and/or the normal detection rate based on the user's operation. a step of setting a range for determining that the feature quantity is normal in the feature space; a step of transforming a feature space of a feature amount to be determined into the feature space; and determining that the feature amount to be determined is abnormal when the feature amount to be determined is not within the range, wherein the feature amount extraction unit determines that the feature amount to be determined is abnormal. Among the recognition models whose processing parameters have been learned so as to determine that the determination target is normal when the determination target data for is input, the classifier determines whether the determination target is normal An information processing method, which is a feature amount extractor for extracting a feature amount .

( 4 ) According to one aspect of the present invention, a computer acquires a plurality of feature amounts related to a normal determination target, selects a plurality of feature amounts from the acquired plurality of feature amounts, and selects the plurality of feature amounts. By transforming the feature space of and receiving information representing the anomaly detection rate and / or the normal detection rate based on the user's operation, the range for determining that the feature amount is normal in the transformed feature space a feature amount extraction unit extracting a feature amount of the determination target based on determination target data relating to the determination target; converting a feature space of the extracted feature amount of the determination target into the feature space; is within the range, it is determined that the feature amount to be determined is normal, and if the feature amount to be determined is not within the range, the determination target is determined to be abnormal, and the feature amount extraction unit learns processing parameters so as to determine that the determination target is normal when determination target data relating to the normal determination target is input. The program is a feature quantity extractor for extracting a feature quantity given to a classifier for judging whether or not the object to be judged is normal from among the recognition models .

According to one aspect of the present invention, it is possible to improve the accuracy of determination processing using a feature amount of data representing a normal state.

1 is a block diagram showing a functional configuration of an image evaluation device 1 of an embodiment to which the invention is applied; FIG. FIG. 4 is a diagram showing learning processing and recognition processing of the image determination device 1 to which the present invention is applied; 3 is a diagram showing an example of a convolutional neural network 300; FIG. FIG. 2 is a diagram showing an example of transforming a feature space, where (a) is a diagram representing a feature space specified by dimensions A, B, and C, and (b) is a diagram representing a feature space specified by dimensions D, E, and F; FIG. 4C is a diagram representing a feature space, (c) is a diagram representing a feature space specified by dimensions G and H, and (d) is a diagram representing a feature space specified by dimensions I, J, K, and L. There is a diagram. FIG. 10 is a diagram for explaining variations in a plurality of feature amounts due to feature space transformation; FIG. FIG. 4 is a diagram showing the distribution of feature quantities in feature space (D, E); FIG. 10 is a diagram showing processing for setting a determination region in a feature space; 6 is a flowchart showing an example of learning processing; 6 is a flowchart showing an example of normality determination processing; FIG. 4 is a schematic diagram for explaining dividing an input image; 3 is a diagram showing an example of a convolutional neural network 300A; FIG. 9 is a flowchart showing an example of determination processing in the second embodiment; FIG. 11 is a flowchart showing another example of determination processing in the second embodiment; FIG.

Hereinafter, an information processing apparatus, an information processing method, and a program to which the present invention is applied will be described with reference to the drawings.

An information processing apparatus according to an embodiment to which the present invention is applied is provided, for example, in an image determination apparatus 1 as shown in FIG. FIG. 1 is a block diagram showing a functional configuration of an image determination device 1 of an embodiment to which the invention is applied. The image determination device 1 learns processing parameters for recognition processing and constructs a recognition model. The image determination device 1 determines an image by processing using a recognition model. By performing image determination, the image determination apparatus 1 can inspect whether or not the target object is normal based on the determination result. In addition, although the description of this embodiment describes the recognition of the image by the image determination device 1, the information processing device according to the present invention is not limited to this. The information processing device may, for example, determine whether or not sounds such as inspection sounds and work sounds of the target object are normal. Any device can be used as long as it can determine whether or not.

The image determination device 1 includes a learning image storage unit 10 , a learning processing unit 20 , a recognition processing unit 30 , a determination result management unit 40 , an interface unit 50 and a learning control unit 70 . The image determination device 1 is a computer having a control device and a storage unit that execute software. The image determination apparatus 1 is a software functional unit that functions by executing a program stored in a storage unit (not shown) by a processor such as a CPU (Central Processing Unit). Further, the image determination device 1 may be a hardware functional unit such as LSI (Large Scale Integration) or ASIC (Application Specific Integrated Circuit). Storage units (14, 32, 42, 72, 74) for storing information in the image determination apparatus 1 include, for example, HDD (Hard Disc Drive), flash memory, EEPROM (Electrically Erasable Programmable Read Only Memory), ROM (Read Only Memory). Memory) or RAM (Random Access Memory) or the like. In addition, the storage unit stores various programs such as firmware and application programs, processing results by various functional units, information acquired from the outside, and the like.

The learning image storage unit 10 has an image conversion unit 12 and a learning image database 14 . The image conversion unit 12 receives input of positive example images used for learning processing. A positive image is, for example, an image of a target object in a normal state. Attribute information (tag, class) of the positive example image is not added to the positive example image. As preprocessing, for example, the image conversion unit 12 divides the positive example image into images of a predetermined size. The image conversion unit 12 accumulates the preprocessed images in the learning image database 14 as learning images. A plurality of positive example images divided by the image conversion unit 12 are accumulated in the learning image database 14 as learning images. Note that the image determination apparatus 1 may be supplied with positive images at any time, and the learning process, which will be described later, may be performed in response to the supply of the positive image.

The learning image output from the learning image storage unit 10 is supplied to the learning processing unit 20 . The learning processing unit 20 performs recognition processing using the learning image and obtains a determination result. The learning processing unit 20 learns the processing parameters of the recognition processing unit 30 (recognition model 32A) so that the determination result is normal, and generates learning result data. The processing parameter of the recognition processing unit 30 (recognition model 32A) is, for example, a filter (also called weight or bias) included in the neural network. The learning result data is stored in the learning result data storage section 32 in the recognition processing section 30 .

The recognition processing unit 30 has, for example, a learning result data storage unit 32 and a determination unit 34 . The learning result data storage unit 32 accumulates processing parameters of the recognition processing unit 30 as learning results, and the processing parameters of the recognition processing unit 30 are updated by the learning processing unit 20 . When the input image of the target object is acquired, the recognition processing unit 30 extracts feature amounts using the recognition model 32A based on the processing parameters of the recognition processing unit 30 . The determination unit 34 determines whether the target object is normal based on the extracted feature amount. The recognition processing unit 30 outputs the determination result by the determination unit 34 to the determination result management unit 40 .

The determination result management unit 40 has a determination result data storage unit 42 . The determination result data storage unit 42 accumulates determination results output by the recognition processing unit 30 . The determination result management unit 40 outputs the determination information to the interface unit 50 and causes the interface unit 50 to output the determination result.

A management terminal 60 may be connected to the determination result data storage unit 42 in the determination result management unit 40 . The management terminal 60 is implemented by a computer having a user interface section 62 operated by an administrator. The user interface unit 62 has an operation unit that receives user operations and a display unit that presents various information to the user. The management terminal 60 identifies the positive example image determined to be abnormal based on the operation accepted by the user interface unit 62 . For example, the management terminal 60 outputs information such as the file name and ID of the positive example image determined to be abnormal to the learning control unit 70 as the correction result. The correction results output from the management terminal 60 to the learning control section 70 are accumulated in the relearning data storage section 72 .

When recognizing an arbitrary image, the recognition processing unit 30 is supplied with an image to be recognized from the outside. The result of recognition processing performed on the unknown image by the recognition processing unit 30 is stored in the determination result data storage unit 42 and read into the management terminal 60 together with the unknown image. Then, when the determination result is corrected by the management terminal 60 , the unknown image is supplied to the learning control section 70 with the correction result added.

The learning control unit 70 has a relearning data storage unit 72 and a repeated learning data storage unit 74 . The determination result output from the determination result management unit 40 and the correction result output from the management terminal 60 are supplied to the learning control unit 70 . The learning control unit 70 stores the positive example images determined to be abnormal in the relearning data storage unit 72 . That is, the relearning data should be determined to be normal, but is a positive example image determined to be abnormal.

The learning control unit 70 creates repetitive learning data. The learning control unit 70 combines the relearning data stored in the relearning data storage unit 72 and the learning images stored in the learning image database 14 to repeatedly create learning data (learning target data). This repeated learning data is stored in the repeated learning data storage unit 74 .

The interface unit 50 is implemented by a communication interface that communicates with an external device of the image determination device 1 . The interface unit 50 receives determination results from the determination result management unit 40 and outputs the determination results to the management terminal 60 . The interface unit 50 outputs the correction result to the learning control unit 70 in response to the correction result being supplied from the management terminal 60 .

FIG. 2 is a diagram showing learning processing and recognition processing of the image determination device 1 to which the present invention is applied. A plurality of positive example images 100 are supplied to the learning processing unit 20 . The recognition processing unit 30 is realized by a convolutional neural network (CNN) 300, for example. The convolutional neural network 300 has a plurality of feature extraction units (also called convolution layers or filter units) 310 as intermediate layers and a discrimination unit (also called an output layer) 320 .

The learning method by the learning processing unit 20 may be the setting of processing parameters for a convolutional neural network, or may be another machine learning method. remembered. The recognition processing section 30 obtains the determination result based on the learning result stored in the learning result data storage section 32 .

The input image 120 is input to the feature amount extraction unit 310 as a determination target. It is assumed that the input image 120 contains black points representing anomalies of the target object. Each of the feature quantity extraction units 310 performs filtering (multiplication processing) on input data based on filter coefficients, and adds a bias value to the filtered image (feature), thereby extracting the feature quantity. The processing parameters of the recognition processing unit 30 described above are, for example, filter coefficients in the feature quantity extraction unit 310 . The feature amount extraction unit 310 extracts the feature amount included in the input image 120 and supplies it to the determination unit 34 .

The determination unit 34 determines that the feature amount supplied from the feature amount extraction unit 310 is normal when the feature amount supplied from the feature amount extraction unit 310 is included in a predetermined area (normal area 500), and the feature amount supplied from the feature amount extraction unit 310 is determined to be normal. If it is not included in the area of , it is determined to be abnormal. The determination unit 34 generates a map 200 as a determination result. The map 200 is image data obtained by mapping an image representing whether or not the part is normal to the part of the target object. The map 200 includes a partial image 200a that identifies a location determined to be abnormal in the input image 120. FIG.

FIG. 3 is a diagram showing an example of a convolutional neural network 300. As shown in FIG. Convolutional neural network 300 corresponds to recognition model 32A. The convolutional neural network 300 includes, for example, a plurality of feature extraction units 310-1 and 310-2 and a determination unit 320. FIG. There is one feature amount extraction unit 310-2 between the feature amount extraction unit 310-1 and the determination unit 320, but this is not a limitation, and a plurality of feature amount extraction units 310 may exist. .

The feature quantity extraction unit 310-1 includes a plurality of elements 310-1a. Each element 310-1a extracts the feature amount of the image by performing filtering as described above. An input image as a determination target is input to each element 310-1a of the feature quantity extraction unit 310-1. The image input to element 310-1a is image 122 obtained by dividing the input image into a predetermined first size. The element 310-1a extracts the feature amount of the input image, and supplies the extracted feature amount to a predetermined element 310-2a in the feature amount extraction section 310-2 as an upper layer. Note that the feature amount extraction section 310-2 to which the feature amount is supplied from the feature amount extraction section 310-1 is set in advance.

The feature amount extraction unit 310-2 has a plurality of elements 310-2a. The element 310-2a is supplied with feature quantities from each of the plurality of elements 310-1a. The element 310-2a extracts the feature amount of the second size image 124 by performing filtering as described above based on the plurality of feature amounts. A feature amount of the second size is extracted. The second size is larger than the first size. The element 310 - 2 a supplies the extracted feature amount to the determination unit 34 instead of the determination unit 320 .

The discrimination unit 320 performs discrimination processing based on the feature amount extracted by the feature amount extraction unit 310-2. The determination unit 320 may determine whether or not it is normal, and may output the degree of normality as a numerical value (level).

4, 5 and 6 are diagrams for explaining the processing of the determination unit 34. FIG.
FIG. 4 is a diagram showing an example of transforming the feature space. 4(a) is a diagram representing a feature space specified by dimensions A, B, and C, and FIG. 4(b) is a diagram representing a feature space specified by dimensions D, E, F, 4(c) is a diagram representing the feature space specified by the dimensions G and H, and FIG. 4(d) is a diagram representing the feature space specified by the dimensions I, J, K and L. FIG.

The feature quantity is expressed as data representing, for example, a multidimensional feature vector (a, b, c, . . . ). In the feature vector (a, b, c...), for example, a is the A-dimensional value, b is the B-dimensional value, and c is the C-dimensional value. The number of dimensions of the feature vector corresponds to the number of features of the target object. For example, if the target object is a human face, there are four (four-dimensional) features: eyes (A), nose (B), mouth (C), and ears (D). , C, D) have values (a, b, c, d).

The determination unit 34 transforms the feature space by selecting the dimension and the number of dimensions of the feature vector representing the feature amount. The determination unit 34 performs a learning process of selecting a dimension in which a plurality of feature quantities are appropriately grouped, from among the dimensions included in the feature vector. When the determining unit 34 selects the feature space (A, B, C) in FIG. It is distributed based on the values a, b, and c of the B and C dimensions.

The determination unit 34 may transform the feature space (A, B, C) shown in FIG. 4A into the feature space (D, E, F) shown in FIG. 4B. The D, E, and F features are different features from the A, B, and C features. As a result, the plurality of feature quantities are distributed in the feature space (D, E, F) based on the values d, e, and f of the dimensions D, E, and F among the dimensions of the feature vector. As a result, the determination unit 34 can change the distribution of feature amounts by transforming the feature space.

The determination unit 34 may transform the feature space (A, B, C) shown in FIG. 4A into the feature space (G, H) shown in FIG. 4C. That is, the determination unit 34 may transform the feature space by reducing features. The G and H features are different features from the A, B and C features. As a result, the plurality of feature quantities are distributed in the feature space (G, H) based on the values g and h of the dimensions of G and H among the dimensions of the feature vector. As a result, the determination unit 34 can change the distribution of feature amounts by transforming the feature space. Note that the determination unit 34 may reduce the feature space (A, B, C) to the feature space (A, B) of the same feature.

The determination unit 34 may convert the feature space (A, B, C) shown in FIG. 4(a) into the feature space (I, J, K, L) shown in FIG. 4(d). The features I, J, K, and L are features different from the features A, B, and C. That is, the determination unit 34 may transform the feature space by increasing the features. As a result, the plurality of feature quantities are obtained in the feature space (I, J, K, L) based on the values i, j, k, and l of the dimensions I, J, K, and L among the dimensions of the feature vector. distributed. As a result, the determination unit 34 can change the distribution of feature amounts by transforming the feature space.

FIG. 5 is a diagram for explaining variations in a plurality of feature amounts due to conversion of the feature space. FIG. 5(a) is a diagram showing the distribution of feature amounts in the feature space (A, B, C), and FIG. 5(b) is a diagram showing the distribution of the feature amount in the feature space (D, E). The determination unit 34 distributes the feature amount according to the values a, b, and c of the feature vector indicating the feature amount supplied from the feature amount extraction unit 310 . In this case, the plurality of feature quantities are distributed over a wide range in the feature space (A, B, C) as shown in FIG. 5(a). Therefore, the determination unit 34 converts the feature space from the feature space (A, B, C) to the feature space (D, E). Then, as shown in FIG. 5(b), the plurality of feature quantities are distributed according to the values d and e corresponding to the feature space (D, E). Thereby, the determination unit 34 can learn that the feature space (D, E) can distribute a plurality of feature amounts in a narrower range than the feature space (A, B, C). In this way, the determination unit 34 transforms the feature space and observes the distribution of the feature quantity, thereby learning the feature space suitable for determining the target object.

FIG. 6 is a diagram showing processing for setting a determination region in the feature space. After determining the feature space, the determination unit 34 sets a normal region 500 in the feature space. The determination unit 34 receives, for example, information representing an anomaly detection rate and/or a normal detection rate. The determination unit 34 receives, for example, information representing an anomaly detection rate and/or a normal detection rate based on a user's operation. The information representing the anomaly detection rate and/or the normal detection rate may be a value (parameter) representing the anomaly detection rate and/or the normal detection rate.

When the abnormality detection rate is increased and the normal detection rate is decreased, the determination unit 34 sets a narrow normal region 500 as shown in FIG. 6(a). As a result, the determination unit 34 can increase the probability of occurrence of feature amounts that are not included in the normal region 500 . When the abnormality detection rate is decreased and the normal detection rate is increased, the determination unit 34 sets a wide normal region 500 as shown in FIG. 6(b). As a result, the determination unit 34 can reduce the probability that the feature quantity included in the normal region 500 will occur.

Since the determination unit 34 receives information representing the abnormality detection rate and/or the normal detection rate based on the user's operation, it is possible to set an appropriate normal region 500 according to the user's task (task). That is, the normal region 500 can be narrowly set when it is desired to strictly extract an abnormal target object among a large number of target objects. The normal region 500 can be set wide when it is desired to mainly extract a target object that largely deviates from a normal target object among a large number of target objects.

FIG. 7 is a flowchart showing an example of learning processing.
The image determination apparatus 1 first inputs a positive image (step S100). The positive example images are accumulated in the learning image database 14 . The learning processing unit 20 performs learning processing using the positive example images (step S102). When dividing the input image in the normality determination process, it is desirable to divide the input image in the learning process in step S102 and perform learning using a plurality of divided images. Thereby, the learning processing unit 20 learns the processing parameters in the recognition model 32A. The learning processing unit 20 determines whether or not the learning of the recognition model 32A using the positive example images accumulated in the learning image database 14 has ended (step S104). For example, when all the positive example images accumulated in the learning image database 14 are used, the learning processing unit 20 determines that the learning process of the recognition model 32A is completed (step S104: YES). If the learning process for the recognition model 32A has not ended (step S104: NO), the learning processing unit 20 repeats the process from step S100.

Next, the feature amount extraction unit 310 in the recognition model 32A extracts a plurality of feature amounts (step S106). For example, the feature amount extraction unit 310 extracts a plurality of feature amounts from one positive example image used in the learning process. The feature quantity extraction unit 310 may extract a plurality of feature quantities from each of the plurality of positive example images used in the learning process. The feature amount extraction unit 310 may extract a plurality of feature amounts from images that are not used for learning processing. Each feature amount may be, for example, the feature amount output from the element 310-2a as described above, or may be the feature amount output from the element 310-1a.

Next, the determination unit 34 sets a feature space (step S108). For example, when the feature vector representing the feature amount has N dimensions (N is a natural number), the determination unit 34 selects a plurality of arbitrary dimensions to set the feature space. The determination unit 34 determines whether or not variations in the plurality of feature amounts are appropriate (step S110). If the variations in the plurality of feature amounts are appropriate (step S110: YES), the determination unit 34 completes learning of the feature space and advances the process to step S112. If the variations in the feature amounts are not appropriate (step S110: NO), the determination unit 34 resets the feature space (step S108).

Next, the determination unit 34 sets a normal region 500 in the learned feature space (step S112). Thereby, the image determination device 1 can learn the processing parameters of the recognition model 32A, learn the feature space for determining whether or not the feature amount is normal, and set the normal region 500 in the feature space.

FIG. 8 is a flowchart illustrating an example of normality determination processing.
First, the image determination device 1 acquires an input image (step S200). The image determination device 1 divides the obtained input image (step S202). FIG. 9 is a schematic diagram for explaining dividing an input image. The image determination device 1 divides the input image 120 into predetermined sizes to obtain a plurality of divided images 120A. The image determination device 1 inputs the divided image 120A to the feature quantity extraction unit 310 of the recognition model 32A (step S204). Note that if the size of the input image does not require division, step S202 is skipped.

Next, the determination unit 34 acquires feature amounts from the feature amount extraction unit 310 (step S206). The determination unit 34 converts the feature space of the acquired feature amount into a learned feature space in the learning process (step S208). For example, if the acquired feature quantity is represented by feature vectors (a, b, c, d, e, f, . . . ) and the learned feature space is feature space (D, E), the feature vector Among them, (d, e) is acquired as a feature quantity and mapped to the feature space (D, E). Thereby, the determination unit 34 transforms the feature space of the acquired feature amount.

Next, the determination unit 34 performs normality determination processing (step S210). In the normality determination process, it is determined whether or not the acquired feature amount (d, e) is within the range of the normal region 500 set in the learned feature space (D, E). The determination unit 34 determines whether or not the normality determination process has ended (step S212). If the determination unit 34 has not performed the normal determination process on all of the divided images 120A (step S212: NO), the determining unit 34 performs the processes from step S204 on the unprocessed divided images 120A. If the determining unit 34 has performed the normal determination process on all divided images 120A (step S212: YES), the process proceeds to step S214.

The determination unit 34 creates a map based on the determination result of the normality determination process (step S214). The determination unit 34 creates a map so that the input image positions where the feature amount outside the normal region 500 is obtained are more conspicuous than the input image positions where the feature amount inside the normal region 500 is obtained. The determination unit 34 outputs the created map to, for example, the management terminal 60 (step S216).

In the present embodiment, the feature amount acquired by the determination unit 34 is the feature amount output from the feature amount extraction unit 310. However, the present invention is not limited to this, and the target feature amount can be extracted. If possible, it does not have to be the feature amount extraction unit 310 of the recognition model 32A.

According to the embodiments described above, the feature space of a plurality of feature values regarding a normal determination target is transformed based on variations in the feature space of the plurality of feature values, and the feature value is determined to be normal in the transformed feature space. Set the range for judgment. That is, according to the image determination device 1, it is possible to learn the feature space so that the feature amount of only positive examples has moderate variations. Specifically, when the feature amount of the input image is within the learned normal region 500, the image determination device 1 determines that the feature value is normal, and when the feature value is not within the normal region 500, It can be determined that the feature amount is abnormal. Thus, according to the image determination device 1, it is possible to improve the accuracy of determination processing using only positive example feature amounts.

Further, according to the image determination apparatus 1, the feature space of the feature amount of the input image is transformed into a learned feature space, and determination is made using the feature amount of the input image whose feature space has been transformed. The input image can be determined in the feature space that has been learned to be

Furthermore, according to the image determination device 1, the area in the feature space that the normal area 500 includes can be changed based on the user's operation. Thereby, according to the image determination device 1, the normal region 500 can be changed according to the task. For example, the normal region 500 can be set narrower when the anomaly detection rate is to be increased to investigate anomalies more precisely, and the normal region 500 can be set wider when the anomaly detection rate is lower to extract obvious anomalies. can be done. As a result, the image determination device 1 can improve user convenience.

Furthermore, according to the image determination device 1, the positive example image is divided into a plurality of divided data according to a predetermined rule, and the processing parameters of the recognition model 32A are learned using each of the divided divided data. Further, according to the image determination device 1, an input image can be divided into a plurality of images according to a predetermined rule, and determination processing can be performed on each image. As a result, according to the image determination device 1, it is possible to suppress erroneous determination due to loss of features of the target object due to processing such as reduction of the input image.

<Second embodiment>
A second embodiment will be described below. The image determination apparatus 1 of the second embodiment extracts a feature amount from the feature amount extraction unit 310 of an arbitrary layer among the plurality of feature amount extraction units 310 in the convolutional neural network 300A, and determines whether the determination target is normal. It is different from the image determination apparatus 1 of the first embodiment in that it is determined whether or not. In the following, the difference will be mainly described.

FIG. 10 is a diagram showing an example of a convolutional neural network 300A. The convolutional neural network 300A includes a plurality of feature amount extraction units 310. FIG. The convolutional neural network 300</b>A outputs a plurality of feature amounts from at least one feature amount extraction unit 310 of the plurality of convolutional neural networks 300 to the determination unit 34 .

Of the feature quantity extraction units 310-1 and 310-2 in the convolutional neural network 300A, the feature quantity extraction unit 310-1 in the lower layer extracts the feature quantity 300a-1 of the first size and extracts the plurality of features The amount is output to the determination unit 34. The first size feature quantity 300a-1 is a smaller size feature quantity than the second size feature quantity 300a-2 extracted by the upper layer feature quantity extraction unit 310-2.

Based on at least one of the feature amount 300a-1 extracted by the feature amount extraction section 310-1 and the feature amount 300a-2 extracted by the feature amount extraction section 310-2, the determination section 34 It is determined whether or not the object to be determined is normal.

FIG. 11 is a flowchart showing an example of determination processing in the second embodiment.
The image determination apparatus 1 receives a determination processing task, for example, based on a user's operation (step S300). The image determination apparatus 1 accepts, for example, a task to determine a small abnormality in a target object or a task to determine a large abnormality in a target object. A minute abnormality of the target object is, for example, a fine flaw of the target object (product or the like). A large abnormality in the target object is, for example, a foreign substance present in a predetermined space such as a room.

After the image is input by the feature quantity extraction unit 310 (step S204), the image determination device 1 selects the feature quantity extraction unit 310 based on the task received in step S300 (step S302). For example, when the image determination apparatus 1 receives a task to determine a minute abnormality of a target object, the image determination apparatus 1 selects the feature quantity extraction unit 310-1 for extracting an abnormality of the first size. For example, when the image determination apparatus 1 receives a task to determine a large abnormality of the target object, it selects the obtained feature quantity extraction unit 310-2 for extracting an abnormality of the second size. Next, the image determination device 1 acquires the feature amount from the selected feature amount extraction unit 310 (step S304). Then, the image determination apparatus 1 performs feature space conversion (step S208) and normality determination processing (step S210) using the acquired feature amount.

FIG. 12 is a flow chart showing another example of determination processing in the second embodiment.
After the image is input by the feature quantity extraction units 310 (step S204), the image determination device 1 selects a plurality of feature quantity extraction units 310 (step S302#). The plurality of feature amount extraction units 310 are, for example, feature amount extraction units 310 that extract feature amounts of different sizes. Next, the image determination apparatus 1 acquires feature amounts from the plurality of selected feature amount extraction units 310 (step S304#).

Next, the image determination apparatus 1 performs feature space conversion (step S208#) and normality determination processing (step S306#) for each of a plurality of feature amounts extracted from one feature amount extraction unit 310. FIG. For example, when the two feature quantity extraction units 310-1 and 310-2 are selected, the image determination device 1 converts the feature space of the plurality of feature quantities extracted by the feature quantity extraction unit 310-1 and converts the first The normality determination process, the transformation of the feature space of the plurality of feature amounts extracted by the feature amount extraction unit 310-2, and the second normality determination process are executed in parallel.

In the normality determination process (step S306#), the image determination apparatus 1 determines whether or not the target object is normal based on the determination results of a plurality of normality determination processes. For example, when the two feature amount extraction units 310-1 and 310-2 are selected, the image determination device 1 performs the determination based on the determination result of the first normality determination process and the determination result of the second normality determination process. , to determine whether the target object is normal. For example, when the determination result of the second normality determination process using the feature amount of the second size in a certain space is "abnormal", the image determination apparatus 1 is used to obtain the second feature amount. A first size feature quantity that has been used is identified. The image determination apparatus 1 detects a first size (minor) abnormality based on the determination result of the first normality determination process using the specified feature amount of the first size.

Note that the image determination apparatus 1 is not limited to executing the first normality determination process and the second normality determination process in parallel. It may be determined whether or not to perform feature space transformation using the feature quantity of the first size extracted by 310-1 and the first normality determination process. That is, when the second normality determination process using the second size feature amount is not determined to be abnormal, the image determination apparatus 1 transforms the feature space using the first size feature amount and performs the first If it is determined to be abnormal by the second normality determination process using the feature amount of the second size without executing the normality determination process, the feature space is transformed using the feature amount of the first size and the first Execute the normality determination process. As a result, the image determination apparatus 1 can first determine a rough abnormality, and if there is a rough abnormality, can determine a minute abnormality. As a result, the image determination device 1 can analyze the abnormality of the target object based on the rough abnormality and the minute abnormality. In addition, the image determination apparatus 1 can reduce processing for minute abnormalities when there is no general abnormality.

Note that the image determination apparatus 1 is not limited to executing the first normality determination process and the second normality determination process in parallel. It may be determined whether or not to perform the feature space conversion using the second size feature quantity extracted in 310-2 and the second normality determination process. That is, when the first normality determination process using the feature amount of the first size is not determined to be abnormal, the image determination apparatus 1 converts the feature space using the feature amount of the second size and performs the second normality determination process. If it is determined to be abnormal by the first normality determination process using the feature amount of the first size without executing the normality determination process of , the feature space is transformed using the feature amount of the second size and the second Execute the normality determination process. As a result, the image determination apparatus 1 can first determine a minute abnormality, and if there is a minute abnormality, roughly determine an abnormality. As a result, according to the image determination apparatus 1, when there is a minute abnormality, it is possible to analyze whether the minute abnormality affects the general abnormality. In addition, the image determination apparatus 1 can reduce the processing for a rough abnormality when there is no minute abnormality.

According to the image determination apparatus 1 of the second embodiment described above, the first feature amount extraction unit (310-1) extracts the feature amount from the determination target data of the first size, the first size and the is a second feature quantity extraction unit (310-2) for extracting a feature quantity from determination object data of a second size different from each other, the feature quantity extracted by the first feature quantity extraction unit, and the second feature quantity and a determination unit 34 that determines whether or not the object to be determined is normal based on at least one of the feature amounts extracted by the extraction unit. Thus, according to the image determination apparatus 1, it is possible to determine whether or not the determination target is normal by using different feature amounts depending on the task, thereby improving convenience for the user.

Further, according to the image determination device 1 of the second embodiment, the feature amount can be extracted using the feature amount extraction section 310 in the convolutional neural network 300A including the feature amount extraction section 310 and the determination section 320. .

Furthermore, according to the image determination device 1 of the second embodiment, the two feature quantity extraction units 310, the feature quantity extraction unit 310-1 and the feature quantity extraction unit 310-2, are generated using the same learning data. be able to.

Furthermore, according to the image determination apparatus 1 of the second embodiment, either the feature amount extracted by the feature amount extraction unit 310-1 or the feature amount extracted by the feature amount extraction unit 310-2 Since the normality determination process is performed using the quantity, the two feature quantity extraction units 310 can be selectively used according to the task desired by the user, and the user's convenience can be improved.

Furthermore, according to the image determination apparatus 1 of the second embodiment, both the feature amount extracted by the feature amount extraction unit 310-1 and the feature amount extracted by the feature amount extraction unit 310-2 are Since the normality determination process is performed using the normality determination process, it is possible to improve the determination accuracy of the normality determination process, for example, to determine a minute abnormality at a location where a rough abnormality is detected.

Furthermore, according to the image determination device 1 of the second embodiment, the positive image is divided into a plurality of divided data according to a predetermined rule, and the processing parameters of the recognition model 32A are learned using each of the divided divided data. do. Further, according to the image determination device 1, an input image can be divided into a plurality of images according to a predetermined rule, and determination processing can be performed on each image. As a result, according to the image determination device 1, it is possible to suppress erroneous determination due to loss of features of the target object due to processing such as reduction of the input image.

In addition, although each embodiment and each modification have been described, these are only examples and are not limited to these. A part may be combined with one or more embodiments or one or more modified examples to realize one aspect of the present invention.

It should be noted that a program for executing each process of the image determination apparatus 1 in this embodiment may be recorded in a computer-readable recording medium, and the program recorded in the recording medium may be read and executed by a computer system. , the above-described various processes related to the image determination apparatus 1 may be performed.

Note that the “computer system” referred to here may include hardware such as an OS and peripheral devices. The "computer system" also includes the home page providing environment (or display environment) if the WWW system is used. In addition, "computer-readable recording medium" means writable non-volatile memory such as flexible disk, magneto-optical disk, ROM, flash memory, portable medium such as CD-ROM, hard disk built in computer system, etc. storage device.

Furthermore, "computer-readable recording medium" means a volatile memory (e.g., DRAM (Dynamic
Random Access Memory)), which holds a program for a certain period of time. Also, the program may be transmitted from a computer system storing the program in a storage device or the like to another computer system via a transmission medium or by a transmission wave in the transmission medium.

Here, the "transmission medium" for transmitting the program refers to a medium having a function of transmitting information, such as a network (communication network) such as the Internet or a communication line (communication line) such as a telephone line. Further, the program may be for realizing part of the functions described above. Further, it may be a so-called difference file (difference program) that can realize the above-described functions in combination with a program already recorded in the computer system.

As described above, the embodiments of the present invention have been described in detail with reference to the drawings, but the specific configuration is not limited to these embodiments, and includes designs within the scope of the gist of the present invention.

1 image determination device 10 learning image storage unit 12 image conversion unit 14 storage unit 14 learning image database 20 learning processing unit 30 recognition processing unit 32 learning result data storage unit 32A recognition model 34 determination unit 40 determination result management unit 42 determination result data storage Unit 50 Interface unit 60 Management terminal 62 User interface unit 70 Learning control unit 72 Re-learning data storage unit 74 Learning data storage unit 300, 300A Neural networks 310, 310-1, 310-2 Feature amount extraction unit 320 Discrimination unit 500 Normal region

Claims (4)

an acquisition unit that acquires a plurality of feature amounts related to a normal determination target;
a conversion unit that converts a feature space of the plurality of selected feature amounts by selecting a plurality of feature amounts from the plurality of feature amounts acquired by the acquisition unit;
A setting for setting a range for determining that the feature amount is normal in the feature space converted by the conversion unit by accepting information representing an anomaly detection rate and/or a normal detection rate based on a user's operation. Department and
A determination target acquisition unit that includes a feature amount extraction unit that extracts the feature amount of the determination target based on determination target data related to the determination target, and that converts the feature space of the extracted feature amount into the feature space transformed by the conversion unit. When,
If the feature amount obtained by converting the feature space by the determination target obtaining unit is within the range set by the setting unit, the feature amount obtained by the determination target obtaining unit is determined to be normal, and the determination is performed. a determination unit that determines that the feature amount acquired by the determination target acquisition unit is abnormal when the feature amount acquired by the target acquisition unit is not within the range set by the setting unit ;
The feature quantity extracting unit determines that the determination target is normal among recognition models whose processing parameters have been learned so that the determination target is determined to be normal when determination target data relating to the normal determination target is input. An information processing apparatus , which is a feature amount extractor that extracts a feature amount to be given to a classifier that determines whether or not . a learning processing unit that divides the determination target data related to the normal determination target into a plurality of divided data according to a predetermined rule, and learns processing parameters of the recognition model using each of the divided divided data,
The feature amount extraction unit divides the determination target data according to the predetermined rule, extracts the feature amount of the determination target using each of the divided data,
The determination unit determines based on the recognition model using feature amounts extracted using each of the divided data.
The information processing device according to claim 1 . a step in which the information processing device acquires a plurality of feature amounts relating to a normal determination target;
a step in which the information processing device selects a plurality of feature amounts from the acquired plurality of feature amounts, thereby transforming a feature space of the selected plurality of feature amounts ;
The information processing device receives information representing an anomaly detection rate and/or a normal detection rate based on a user's operation, thereby setting a range for determining that the feature amount is normal in the transformed feature space. and
a step of extracting a feature amount of the determination target based on determination target data relating to the determination target by a feature amount extraction unit, and converting a feature space of the extracted feature amount of the determination target into the feature space;
If the feature amount to be determined after the feature space conversion is within the range, the feature amount to be determined is determined to be normal, and if the feature amount to be determined is not within the range, A step of determining that the feature amount to be determined is abnormal,
The feature quantity extracting unit determines that the determination target is normal among recognition models whose processing parameters have been learned so that the determination target is determined to be normal when determination target data relating to the normal determination target is input. It is a feature amount extractor that extracts the feature amount to be given to the classifier that determines whether or not
Information processing methods. to the computer,
Acquiring a plurality of feature values related to a normal determination target,
Selecting a plurality of feature values from the acquired plurality of feature values, transforming the feature space of the selected plurality of feature values,
By accepting information representing an anomaly detection rate and/or a normal detection rate based on a user's operation, setting a range for determining that the feature amount is normal in the transformed feature space,
causing a feature amount extraction unit to extract a feature amount of the determination target based on determination target data relating to the determination target, and converting a feature space of the extracted feature amount of the determination target into the feature space;
If the feature amount to be determined after the feature space conversion is within the range, the feature amount to be determined is determined to be normal, and if the feature amount to be determined is not within the range, Determining that the feature amount to be determined is abnormal,
The feature quantity extracting unit determines that the determination target is normal among recognition models whose processing parameters have been learned so that the determination target is determined to be normal when determination target data relating to the normal determination target is input. A feature quantity extractor that extracts a feature quantity to be given to a classifier that determines whether or not
program.

Images