JP7239824B2 - Image inspection system, image inspection device and image inspection program - Google Patents

Description

The present invention relates to an image inspection system, an image inspection apparatus, and an image inspection program.

For example, a business operator that provides services to users (hereinafter simply referred to as a business operator) builds an information processing system for providing services to users. Specifically, for example, a company builds an information processing system that identifies a part including a defect (for example, a stain) from an image of each mass-produced part.

In such an information processing system, for example, a supervised learning model (hereinafter simply referred to as a learning model) is created by performing machine learning on feature values extracted from images to be learned (hereinafter also referred to as learning images). Generate. Then, the information processing system inputs the feature amount extracted from the image to be inspected (hereinafter also referred to as the inspection image) into the learning model, thereby indicating whether or not the component corresponding to the inspection image contains a defect. Get score.

As a result, the business operator can refer to the obtained score to identify the defective part among the mass-produced parts (see, for example, Patent Literatures 1 to 3).

JP 2015-041164 A JP 2010-067068 A JP 2010-102690 A

Here, when generating a learning model as described above, the business operator prepares, for example, learning images of parts including defects for each defect size. Then, the information processing system generates a learning model for each defect size, for example, by using each learning image for each defect size. As a result, the information processing system can accurately determine whether or not the part corresponding to the inspection image contains a defect.

However, the operator may not be able to prepare enough training images corresponding to each size of the defect. Therefore, when the information processing system cannot exhaustively generate a learning model for each defect size, and cannot accurately determine whether or not the part corresponding to the inspection image includes a defect. There is

Accordingly, in one aspect, an object of the present invention is to provide an image inspection system, an image inspection apparatus, and an image inspection program capable of accurately identifying a component including a defect from an image of the component.

In one aspect of the embodiment, in an image inspection system having a plurality of information processing devices that perform machine learning on a plurality of learning images, each of the plurality of information processing devices performs machine learning on the plurality of learning images for each first size. a divided image generator that generates a plurality of first divided images by dividing and generates a plurality of second divided images by dividing the plurality of training images into second sizes different from the first size; From each of a plurality of first normal divided images corresponding to a normal state among the plurality of first divided images and a plurality of second normal divided images corresponding to a normal state among the plurality of second divided images a feature amount extraction unit for extracting feature amounts corresponding to a plurality of types of parameters; and from the plurality of types of parameters, the features between the plurality of first normal segmented images and the plurality of second normal segmented images. By using a parameter identifying unit that identifies a first parameter whose similarity relation in amount satisfies a predetermined condition, and a feature amount corresponding to the identified first parameter, the plurality of first divided images and the plurality of second divided images are a machine learning execution unit that performs machine learning on the divided images.

According to one aspect, it is possible to accurately identify a component containing a defect from an image of the component.

FIG. 1 is a diagram illustrating the configuration of an information processing system 10. As shown in FIG. FIG. 2 illustrates a specific example of generating a learning model for each defect size. FIG. 3 is a diagram for explaining the hardware configuration of the information processing device 1. As shown in FIG. FIG. 4 is a functional block diagram of the information processing apparatus 1. As shown in FIG. FIG. 5 is a flowchart for explaining an outline of image inspection processing in the first embodiment. FIG. 6 is a flowchart for explaining an outline of image inspection processing in the first embodiment. FIG. 7 is a flowchart for explaining details of image inspection processing in the first embodiment. FIG. 8 is a flowchart for explaining details of image inspection processing in the first embodiment. FIG. 9 is a flowchart for explaining details of image inspection processing in the first embodiment. FIG. 10 is a flowchart for explaining details of image inspection processing in the first embodiment. FIG. 11 is a flowchart for explaining details of image inspection processing in the first embodiment. FIG. 12 is a flowchart for explaining details of image inspection processing in the first embodiment. FIG. 13 is a flowchart for explaining details of image inspection processing in the first embodiment. FIG. 14 is a diagram illustrating a specific example of the process of S22. FIG. 15 is a diagram illustrating a specific example of the process of S23. FIG. 16 is a diagram illustrating a specific example of the processing of S34 and S35. FIG. 17 is a diagram illustrating a specific example of the processing of S34 and S35. FIG. 18 is a diagram illustrating a specific example of distribution corresponding to the luminance histogram. FIG. 19 is a diagram explaining a specific example of distribution corresponding to the gradient histogram. FIG. 20 is a diagram illustrating details of image inspection processing in the first embodiment.

[Configuration of information processing system]
First, the configuration of the information processing system 10 will be described. FIG. 1 is a diagram illustrating the configuration of an information processing system 10. As shown in FIG.

As shown in FIG. 1, an information processing system 10 (hereinafter also referred to as an image inspection system 10) includes, for example, one or more information processing apparatuses 1 (hereinafter also referred to as an image inspection apparatus 1) and one or more operation units. terminal 5;

The operation terminal 5 is, for example, a PC (Personal Computer), and is a terminal for inputting information required by the operator. Specifically, the operation terminal 5 inputs learning images and inspection images of parts, for example.

The information processing apparatus 1 generates a learning model, for example, by performing machine learning on feature amounts extracted from learning images input via the operation terminal 5 . Then, for example, when receiving an inspection image input via the operation terminal 5, the information processing apparatus 1 inputs a feature amount extracted from the inspection image into the learning model. After that, the information processing apparatus 1 determines whether or not the part corresponding to the inspection image includes a defect, based on the score output from the learning model along with the input of the feature amount.

Here, when generating a learning model as described above, the business operator prepares, for example, a learning image of a part including a defect for each size of the defect. Then, the information processing apparatus 1 generates a learning model for each defect size, for example, by using each learning image for each defect size. A specific example of generating a learning model for each defect size will be described below.

[Specific examples of learning models for each defect size]
FIG. 2 illustrates a specific example of generating a learning model for each defect size.

Specifically, the learning image IM11 shown in FIG. 2A is an image including the part IM12. A component IM12 shown in FIG. 2A includes a defect IM13. A learning image IM21 shown in FIG. 2C is an image including a part IM22. A component IM22 shown in FIG. 2C includes a defect IM23.

That is, the learning image IM11 shown in FIG. 2A is an image of a part IM12 including a relatively small-sized defect IM13, and the learning image IM21 shown in FIG. 2C is an image of a relatively large-sized defect IM23. is an image of a part IM22 including

Specifically, when the information processing apparatus 1 performs learning on the learning image IM11 shown in FIG. 2A, for example, the defect IM13 (eg, the size closest to the size of defect IM13) corresponding to . Then, as shown in FIG. 2B, the information processing apparatus 1 divides the learning image IM11 for each window size WS11 corresponding to the specified size to generate a plurality of divided images. After that, the information processing device 1 performs machine learning by using feature amounts extracted from each of the plurality of generated divided images.

Further, when performing learning on the learning image IM21 shown in FIG. Identify the size to be used (eg, the size closest to the size of defect IM23). Then, as shown in FIG. 2D, the information processing apparatus 1 divides the learning image IM21 for each window size WS21 corresponding to the specified size to generate a plurality of divided images. After that, the information processing device 1 performs machine learning by using feature amounts extracted from each of the plurality of generated divided images.

Thereby, the information processing apparatus 1 can generate a learning model for each defect size. Therefore, the information processing apparatus 1 can selectively use the learning model according to the situation, and can accurately determine whether or not the part corresponding to the inspection image includes a defect. .

However, operators may not have enough training images for each size of defect. Therefore, the information processing apparatus 1 cannot comprehensively generate a learning model for each defect size, and cannot accurately determine whether or not a component corresponding to an inspection image includes a defect. Sometimes.

Therefore, the information processing apparatus 1 according to the present embodiment generates a plurality of split images (hereinafter also referred to as first split images) by splitting a plurality of learning images for each first size. The information processing device 1 also generates a plurality of divided images (hereinafter also referred to as second divided images) by dividing the plurality of learning images into second sizes different from the first size.

Then, the information processing device 1 selects a plurality of divided images corresponding to the normal state among the plurality of first divided images (hereinafter also referred to as a first normal divided image) and a normal state among the plurality of second divided images. from each of the plurality of split images (hereinafter also referred to as second normal split images) corresponding to . Furthermore, the information processing apparatus 1 selects a parameter (hereinafter referred to as first first segmented image) from a plurality of types of parameters that satisfies a predetermined condition for the similarity relationship of feature amounts between the plurality of first normal segmented images and the plurality of second normal segmented images. (also called parameters).

After that, the information processing device 1 performs machine learning on the plurality of first divided images and the plurality of second divided images by using the feature amount corresponding to the specified first parameter.

That is, the information processing apparatus 1 specifies the first parameter corresponding to the feature amount that does not depend on the size of the defect, among the parameters with which the feature amount can be extracted from the first divided image and the second divided image. Then, the information processing device 1 performs machine learning on the learning image by using the feature amount corresponding to the specified first parameter.

As a result, the information processing apparatus 1 can use the generated learning model to accurately identify defects of sizes for which a sufficient number of learning images were not prepared. Therefore, the information processing apparatus 1 can accurately determine whether or not the component corresponding to the inspection image includes a defect.

[Hardware configuration of information processing system]
Next, the hardware configuration of the information processing system 10 will be described. FIG. 3 is a diagram for explaining the hardware configuration of the information processing device 1. As shown in FIG.

The information processing apparatus 1 has a CPU 101 as a processor, a memory 102, an external interface (I/O unit) 103, and a storage medium 104, as shown in FIG. Each unit is connected to each other via a bus 105 .

The storage medium 104 has, for example, a program storage area (not shown) that stores a program 110 for performing a process (hereinafter also referred to as an image inspection process) to identify parts containing defects from images of the parts. The storage medium 104 also has a storage unit 130 (hereinafter also referred to as an information storage area 130) that stores information used when performing image inspection processing, for example. Note that the storage medium 104 may be, for example, an HDD (Hard Disk Drive) or an SSD (Solid State Drive).

The CPU 101 executes the program 110 loaded from the storage medium 104 to the memory 102 to perform image inspection processing.

Also, the external interface 103 communicates with the operation terminal 5, for example.

[Functions of information processing system]
Next, functions of the information processing system 10 will be described. FIG. 4 is a functional block diagram of the information processing apparatus 1. As shown in FIG.

As shown in FIG. 4, the information processing apparatus 1 is divided into an information receiving section 111, an information managing section 112, and a divided Various functions including an image generation unit 113, a feature amount extraction unit 114, a parameter identification unit 115, a machine learning execution unit 116, a score acquisition unit 117, an abnormality determination unit 118, and an information output unit 119 are realized. .

For example, the information processing apparatus 1 stores a learning image 131, a first divided image 132, a second divided image 133, and a learning model 134 in the information storage area 130, as shown in FIG.

First, functions for performing machine learning of the learning image 131 (generating the learning model 134) will be described.

The information receiving unit 111 receives, for example, a plurality of learning images 131 (learning images about parts) transmitted from the operation terminal 5 .

The information management unit 112 stores the plurality of learning images 131 received by the information reception unit 111 in the information storage area 130, for example.

The divided image generation unit 113 generates a plurality of first divided images 132 by, for example, dividing a plurality of learning images 131 stored in the information storage area 130 for each first size. Also, the divided image generation unit 113 generates a plurality of second divided images 133 by, for example, dividing the plurality of learning images 131 stored in the information storage area 130 for each second size. Then, the information management unit 112 stores, for example, the plurality of first divided images 132 and the plurality of second divided images 133 generated by the divided image generation unit 113 in the information storage area 130 .

The feature quantity extracting unit 114 extracts a plurality of first normal divided images 132 a among the plurality of first divided images 132 generated by the divided image generating unit 113 and a plurality of second divided images 133 generated by the divided image generating unit 113 . Feature amounts corresponding to a plurality of types of parameters are extracted from each of the plurality of second normal divided images 133a among them.

The parameter specifying unit 115 determines the similarity of feature amounts between the plurality of first normal divided images 132a and the plurality of second normal divided images 133a from the plurality of types of parameters from which the feature amount extraction unit 114 has extracted the feature amounts. A first parameter whose relationship satisfies a predetermined condition is identified.

The machine learning execution unit 116 performs machine learning on the plurality of first divided images 132 and the plurality of second divided images 133 by using feature amounts corresponding to the first parameters identified by the parameter identifying unit 115 . The machine learning execution unit 116 then generates a learning model 134 in the information storage area 130 .

Next, a description will be given of the function of specifying a part containing a defect by using the learning model 134. FIG.

The information receiving unit 111 receives, for example, inspection images (inspection images of parts) transmitted from the operation terminal 5 .

The divided image generating unit 113 generates a plurality of divided images by dividing the inspection image received by the information receiving unit 111 into each predetermined size. Specifically, the divided image generating unit 113 divides the inspection image received by the information receiving unit 111 for each size specified in advance by the business operator.

The feature amount extraction unit 114 extracts feature amounts corresponding to the first parameters from each of the divided images generated by the divided image generation unit 113 .

The score acquisition unit 117 obtains the score output from the learning model 134 in accordance with the input of the feature amount corresponding to the first parameter extracted by the feature amount extraction unit 114 for each of the plurality of divided images generated by the divided image generation unit 113. to get That is, the score acquisition unit 117 acquires scores for each of the divided images generated by the divided image generation unit 113 by using the learning model 134 generated in the information storage area 130 .

The abnormality determination unit 118 determines that a divided image whose score acquired by the score acquisition unit 117 is equal to or less than a threshold among the plurality of divided images generated by the divided image generation unit 113 is in an abnormal state.

For example, when the abnormality determination unit 118 determines that a divided image in an abnormal state is included, the information output unit 119 outputs information indicating that the inspection image received by the information reception unit 111 is in an abnormal state to the operation terminal. Output to 5.

[Outline of the first embodiment]
Next, an outline of the first embodiment will be described. 5 and 6 are flowcharts for explaining an outline of image inspection processing in the first embodiment.

As shown in FIG. 5, the information processing device 1 waits until the machine learning execution timing (NO in S1). The machine learning execution timing may be, for example, the timing at which the business operator inputs information to the information processing apparatus 1 to start the image inspection process.

Then, when the machine learning execution timing has come (YES in S1), the information processing apparatus 1 divides the plurality of learning images 131 by the first size to generate a plurality of first divided images 132 (S2). . Also, in this case, the information processing apparatus 1 generates a plurality of second divided images 133 by dividing the plurality of learning images 131 into each second size (S3).

Subsequently, the information processing apparatus 1 generates a plurality of first normal divided images 132a corresponding to the normal state among the plurality of first divided images 132 and a plurality of the plurality of second divided images 133 corresponding to the normal state. and the second normal segmented image 133a, respectively, feature amounts corresponding to a plurality of types of parameters are extracted (S4).

Further, as shown in FIG. 6, the information processing apparatus 1 determines that the similarity relationship of the feature amounts between the plurality of first normal divided images 132a and the plurality of second normal divided images 133a is determined based on the plurality of types of parameters. A first parameter that satisfies the conditions is specified (S5).

After that, the information processing device 1 performs machine learning on the plurality of first divided images 132 and the plurality of second divided images 133 by using the feature amount corresponding to the first parameter specified in the process of S5 (S6). .

That is, the information processing apparatus 1 specifies the first parameter corresponding to the feature amount that does not depend on the size of the defect, among the parameters with which the feature amount can be extracted from the first divided image 132 and the second divided image 133. . Then, the information processing device 1 performs machine learning on the learning image by using the feature amount corresponding to the specified first parameter.

As a result, the information processing apparatus 1 can use the generated learning model to accurately identify even a defect of a size for which a sufficient number of learning images 131 were not prepared. Therefore, the information processing apparatus 1 can accurately determine whether or not the component corresponding to the inspection image includes a defect.

[Details of the first embodiment]
Next, details of the first embodiment will be described. 7 to 13 are flowcharts for explaining the details of image inspection processing according to the first embodiment. 14 to 20 are diagrams for explaining the details of image inspection processing in the first embodiment.

[Image accumulation processing]
First, processing for accumulating the learning images 131 (hereinafter also referred to as image accumulation processing) will be described. FIG. 7 is a flowchart for explaining image storage processing.

As shown in FIG. 7, the information receiving unit 111 of the information processing device 1 waits until the learning image 131 is received (NO in S11). Specifically, the information receiving unit 111 waits until the learning image 131 is transmitted from the operation terminal 5, for example.

Then, when the learning image 131 is received (YES in S11), the information management unit 112 of the information processing device 1 stores the learning image 131 received in the process of S11 in the information storage area 130 (S12).

[Machine learning processing]
Next, processing for performing machine learning on the learning image 131 (hereinafter also referred to as machine learning processing) will be described. 8 to 11 are flowcharts for explaining machine learning processing.

As shown in FIG. 8, the information management unit 112 waits until the machine learning execution timing (NO in S21).

Then, when the machine learning execution timing comes (YES in S21), the divided image generation unit 113 divides the plurality of learning images 131 stored in the information storage area 130 into a plurality of first sizes by dividing the plurality of learning images 131 into a plurality of first sizes. A divided image 132 is generated (S22). Specifically, the divided image generator 113 divides at least some of the plurality of learning images 131 stored in the information storage area 130 . A specific example of the processing of S22 will be described below.

[Specific example of processing in S22]
FIG. 14 is a diagram illustrating a specific example of the process of S22. FIG. 14A is a diagram for explaining a specific example when performing the processing of S22 on the learning image IM31 included in the learning image 131, and FIG. FIG. 12 is a diagram illustrating a specific example of performing the process of S22 on IM32;

Specifically, for example, as shown in FIG. 14A, the divided image generation unit 113 divides the learning image IM31 into each window size WS31 (first size) to generate a plurality of first divided images 132. Generate. In addition, the divided image generation unit 113 generates a plurality of first divided images 132 by dividing the learning image IM32 into each window size WS31, for example, as shown in FIG. 14B.

Returning to FIG. 8, the divided image generating unit 113 generates a plurality of second divided images 133 by dividing the plurality of learning images 131 stored in the information storage area 130 for each second size (S23). Specifically, the divided image generator 113 divides at least some of the plurality of learning images 131 stored in the information storage area 130 . A specific example of the processing of S23 will be described below.

[Specific example of processing in S23]
FIG. 15 is a diagram illustrating a specific example of the process of S23. FIG. 15A is a diagram for explaining a specific example when performing the processing of S23 on the learning image IM31 included in the learning image 131, and FIG. FIG. 13 is a diagram illustrating a specific example of performing the process of S23 on IM32;

Specifically, for example, as shown in FIG. 15A, the divided image generation unit 113 divides the learning image IM31 into each window size WS32 (second size) to generate a plurality of second divided images 133. Generate. Further, the divided image generating unit 113 generates a plurality of second divided images 133 by dividing the learning image IM32 into each window size WS32, for example, as shown in FIG. 15B.

Returning to FIG. 8, the feature quantity extraction unit 114 of the information processing device 1 identifies a plurality of first normal divided images 132a from the plurality of first divided images 132 generated in the process of S22 (S24).

Specifically, for example, the feature amount extraction unit 114 refers to the state information added by the operator for each of the plurality of first divided images 132 generated in the process of S22, and extracts the plurality of first divided images generated in the process of S22. Among the divided images 132, an image associated with state information indicating that each image is in a normal state is specified as a first normal divided image 132a.

Then, the feature amount extraction unit 114 identifies a plurality of second normal divided images 133a from the plurality of second divided images 133 generated in the process of S23 (S25).

Specifically, for example, the feature amount extraction unit 114 refers to the state information added by the operator for each of the plurality of second divided images 133 generated in the process of S23, and extracts the plurality of second divided images generated in the process of S23. Among the divided images 133, an image associated with state information indicating that each image is in a normal state is specified as a second normal divided image 133a.

In addition, for example, among the plurality of first divided images 132 generated in the processing of S22, the business operator adds state information indicating that each image is in a normal state to an image that does not include a component defect. can be anything. In addition, for example, among the plurality of first divided images 132 generated in the process of S22, the business operator provides state information indicating that each image is in an abnormal state for an image including at least a part of a component defect. may be added.

Subsequently, as shown in FIG. 9, the feature amount extraction unit 114 extracts a plurality of types of parameters from each of the plurality of first normal divided images 132a and the plurality of second normal divided images 133a generated in the process of S24. A corresponding feature amount is extracted (S31).

Specifically, the feature amount extraction unit 114 extracts, for example, feature amounts corresponding to the average brightness, brightness variance, etc. of the first divided image 132 and the second divided image 133 .

Then, the parameter specifying unit 115 of the information processing device 1 generates a distribution of the feature amounts of the plurality of first normal divided images 132a extracted in the process of S31 for each parameter for which the feature amount was extracted in the process of S31. (S32).

Further, the parameter specifying unit 115 generates a distribution of the feature amounts of the plurality of second normal divided images 133a extracted in the process of S31 for each parameter for which the feature amount is extracted in the process of S31 (S33).

After that, the parameter specifying unit 115 calculates the difference between the shape of the distribution generated in the process of S32 and the shape of the distribution generated in the process of S33 for each parameter whose feature amount is extracted in the process of S31 (S34 ).

Then, the parameter specifying unit 115 specifies, as a first parameter, a parameter corresponding to a feature amount whose difference calculated in the process of S34 satisfies a predetermined condition, among the parameters for which the feature amount was extracted in the process of S31 (S35) (S35). ). Details of the processing of S34 and S35 will be described below.

[Details of processing in S34 and S35]
FIG. 11 is a flowchart for explaining the details of the processing of S34 and S35.

As shown in FIG. 11, the parameter specifying unit 115 calculates the basic statistic of the distribution generated in the process of S32 for each parameter whose feature amount is extracted in the process of S31 (S101).

Specifically, for example, when luminance variance is included in the parameter for which the feature amount is extracted in the process of S31, the parameter specifying unit 115 determines the average of the luminance variance in the distribution generated in the process of S32 (hereinafter simply referred to as the average μ1 ), the standard deviation of the luminance distribution in the distribution generated in the process of S32 (hereinafter also simply referred to as the standard deviation ρ1), and the skewness of the histogram corresponding to the distribution generated in the process of S32 (hereinafter simply referred to as the skewness s1) and the kurtosis of the histogram corresponding to the distribution generated in the process of S32 (hereinafter also simply referred to as kurtosis k1) are specified as basic statistics corresponding to the luminance variance.

Further, the parameter specifying unit 115 calculates the basic statistic of the distribution generated in the process of S33 for each parameter whose feature amount is extracted in the process of S31 (S102).

Specifically, for example, when luminance variance is included in the parameter for which the feature amount is extracted in the process of S31, the parameter specifying unit 115 determines the average of the luminance variance in the distribution generated in the process of S33 (hereinafter simply referred to as the average μ2 ), the standard deviation of the luminance variance in the distribution generated in the process of S33 (hereinafter also simply referred to as the standard deviation ρ2), and the skewness of the histogram corresponding to the distribution generated in the process of S33 (hereinafter simply referred to as the skewness s2) and the kurtosis of the histogram corresponding to the distribution generated in the process of S33 (hereinafter also simply referred to as kurtosis k2) are specified as basic statistics corresponding to the luminance variance.

Then, the parameter specifying unit 115 uses the distribution generated in the process of S32 and the distribution generated in the process of S33 from the basic statistics calculated in the processes of S101 and S102 for each parameter whose feature amount is extracted in the process of S31. The degree of similarity of the shape with the distribution is calculated (S103).

Specifically, the parameter identifying unit 115 calculates the degree of similarity using, for example, the following formula (1).
Similarity=|μ1−μ2|+|ρ1−ρ2|+|s1−s2|+|k1−k2| (Formula 1)

Subsequently, the parameter identification unit 115 identifies N parameters in descending order of the degree of similarity calculated in the process of S103 (S104).

Specifically, the parameter identifying unit 115 calculates N using, for example, the following equations (2) and (3). Note that M in the following formulas (2) and (3) is, for example, the learning image 131 used when generating the first divided image 132 in the process of S22 (the second divided image 133 is generated in the process of S23). It may be the number of data of the learning image 131) used when performing.
N=M/10+2 (M<80) (Formula 2)
N=10 (M≧80) (Formula 3)

After that, the parameter identification unit 115 identifies the N parameters identified in the process of S104 as first parameters (S105).

In the process of S35, the parameter specifying unit 115 determines, for example, the center of the distribution generated in the process of S32 (for example, the median value, mean value, or mode of the distribution) and the center of the distribution generated in the process of S33. A parameter corresponding to a feature amount for which the difference between is equal to or less than a threshold may be specified as the first parameter. Further, the parameter identifying unit 115 identifies, as the first parameter, a parameter corresponding to a feature quantity for which the difference between the width of the distribution generated in the process of S32 and the width of the distribution generated in the process of S33 is equal to or less than a threshold, for example. It may be something to do. A specific example of the processing of S34 and S35 in the case of using the difference in the central value of the distribution will be described below.

[Specific example of processing in S34 and S35]
16 and 17 are diagrams for explaining specific examples of the processing of S34 and S35. FIG. 16A is a specific example for explaining the distribution of the first normal divided image 132a when the average luminance is used as a parameter, and FIG. It is a specific example for explaining the distribution of normal divided images 133a. Also, FIG. 17A is a specific example for explaining the distribution of the first normal divided image 132a when brightness variance is adopted as a parameter, and FIG. It is a specific example for explaining the distribution of the second normal divided image 133a.

Specifically, in the example shown in FIG. 16, the median value of the histogram shown on the right side of FIG. 16A matches the median value of the histogram shown on the right side of FIG. 16B. Therefore, in this case, the parameter specifying unit 115, for example, determines that the difference between the median value of the distribution of the first normal divided image 132a and the median value of the second normal divided image 133a is equal to or less than the threshold, and sets the average luminance to the It is specified as one parameter.

On the other hand, in the example shown in FIG. 17, the median value of the histogram shown on the right side of FIG. 17(A) does not match the median value of the histogram shown on the right side of FIG. 17(B). Therefore, when the parameter specifying unit 115 determines that the difference between the median value of the distribution of the first divided image 132 and the median value of the second divided image 133 exceeds the threshold, for example, the luminance variance is set as the first parameter. Make a decision not to specify.

Note that the feature amount extraction unit 114 may extract, for example, feature amounts corresponding to the luminance histogram and the gradient histogram of the first divided image 132 and the second divided learning image 132 in the process of S31. Then, in the processing of S32 and S33, the parameter specifying unit 115 may generate, for example, a distribution of feature quantities corresponding to the luminance histogram and the gradient histogram. A specific example of the distribution corresponding to the luminance histogram will be described below.

[Specific example of distribution corresponding to luminance histogram]
FIG. 18 is a diagram illustrating a specific example of distribution corresponding to the luminance histogram. FIG. 18A is a specific example of a luminance histogram for each first normal divided image 132a, and FIGS. 18B and 18C are distributions corresponding to the luminance histograms shown in FIG. 18A. A specific example.

Specifically, the luminance histogram shown in FIG. a section indicating the number of pixels between 128 and 191 (hereinafter also referred to as a third section), and a section indicating the number of pixels with a luminance value between 128 and 191 (hereinafter also referred to as a third section); is between 192 and 255 (hereinafter also referred to as a fourth partition).

Then, in the process of S31, the feature amount extraction unit 114, for example, for each first normal divided image 132a, calculates the number of pixels corresponding to each of the first, second, third, and fourth divisions as a brightness histogram. Extract the feature quantity corresponding to .

After that, in the process of S32, the parameter specifying unit 115 aggregates the number of first normal divided images 132a corresponding to the number of pixels in the first section, for example, as shown in FIG. A distribution is generated for the luminance histogram of the first normal segmented image 132a. In addition, in the process of S33, the parameter specifying unit 115, for example, tallies the number of the second normal divided images 133a corresponding to the number of pixels in the first section, thereby obtaining the brightness histogram of the second normal divided images 133a. generates a distribution (not shown) of

Then, in the processes of S34 and S35, the parameter specifying unit 115 uses the distribution of the luminance histogram of the first normal divided image 132a and the distribution of the luminance histogram of the second normal divided image 133a to obtain the first parameter identification.

In addition, in the process of S31, for example, as shown in FIG. The distribution of the brightness histogram of the first normal divided image 132a may also be generated.

Then, in the processes of S34 and S35, the parameter specifying unit 115 specifies the first parameter by using the distributions of the luminance histograms corresponding to the plurality of sections (eg, the first section and the fourth section). may be

Specifically, the parameter specifying unit 115 may, for example, calculate the difference in shape for each distribution corresponding to a plurality of sections. Then, for example, when at least one of the differences in shapes corresponding to the plurality of sections satisfies a predetermined condition, the parameter specifying unit 115 may specify the brightness histogram as the first parameter. Also, in this case, the parameter specifying unit 115 may specify, as the first parameter, only the brightness histogram corresponding to the section where the shape difference satisfies a predetermined condition, for example, among the brightness histograms.

[Specific example of distribution corresponding to gradient histogram]
Next, a specific example of the distribution corresponding to the gradient histogram will be described. FIG. 19 is a diagram explaining a specific example of distribution corresponding to the gradient histogram. FIG. 19A is a specific example of the gradient histogram for each first normal divided image 132a, and FIGS. 19B and 19C are distributions corresponding to the gradient histograms shown in FIG. 19A. A specific example.

Specifically, the brightness histogram shown in FIG. 19A has a section indicating the number of pixels with a brightness gradient between 0° and 45° (hereinafter also referred to as a first section), and a section indicating the number of pixels with a brightness gradient from 46°. A section indicating the number of pixels up to 90° (hereinafter also referred to as a second section) and a section indicating the number of pixels having a luminance gradient between 91° and 135° (hereinafter also referred to as a third section) and a section indicating the number of pixels whose luminance gradient is between 136° and 180° (hereinafter also referred to as a fourth section).

Then, in the process of S31, for example, the feature amount extraction unit 114 calculates the number of pixels corresponding to each of the first, second, third, and fourth partitions for each first normal divided image 132a as a gradient histogram. Extract the feature quantity corresponding to .

After that, in the process of S32, the parameter specifying unit 115 counts the number of first normal divided images 132a corresponding to the number of pixels in the first section, for example, as shown in FIG. A distribution is generated for the gradient histogram of the first normal segmented image 132a. In addition, in the process of S33, the parameter specifying unit 115, for example, counts the number of second normal divided images 133a corresponding to the number of pixels in the first section, thereby obtaining a gradient histogram of the second normal divided images 133a. generates a distribution (not shown) of

Then, in the processes of S34 and S35, the parameter specifying unit 115 uses the distribution of the gradient histogram of the first normal divided image 132a and the distribution of the gradient histogram of the second normal divided image 133a to obtain the first parameter identification.

In addition, in the process of S31, for example, as shown in FIG. The distribution of the gradient histogram of the first normal divided image 132a may also be generated.

Then, similarly to the case described with reference to FIG. 18, the parameter specifying unit 115 uses distributions of gradient histograms corresponding to multiple sections (for example, the first section and the third section) in the processes of S34 and S35. may specify the first parameter.

Returning to FIG. 10, the feature quantity extraction unit 114 identifies a plurality of first abnormal divided images 132b from the plurality of first divided images 132 generated in the process of S22 (S41). Also, the feature quantity extraction unit 114 identifies a plurality of second abnormal divided images 133b from the plurality of second divided images 133 generated in the process of S23 (S42).

Then, the feature amount extraction unit 114 extracts the feature amount corresponding to the first parameter specified in the process of S35 from the plurality of first abnormal segmented images 132b and the plurality of second abnormal segmented images 133b specified in the process of S41. Extract (S43).

Subsequently, the information management unit 112 generates a plurality of learning data in which the feature amount corresponding to the first parameter extracted in the process of S31 and the state information indicating that each image is in a normal state are associated with each other, and the learning data of S43. A plurality of pieces of learning data are generated in which the feature amount corresponding to the first parameter extracted in the process is associated with state information indicating that each image is in an abnormal state (S44).

That is, the information management unit 112 stores the feature amount corresponding to the first parameter extracted from each of the plurality of first normal divided images 132a and the plurality of second normal divided images 133a in the process of S31, and the normal state of each image. A plurality of pieces of learning data are generated in association with state information indicating that there is. In addition, the information management unit 112 adds the feature amount corresponding to the first parameter extracted from each of the plurality of first abnormal divided images 132b and the plurality of second abnormal divided images 133b in the process of S43, and A plurality of pieces of learning data each associated with state information indicating that there is a state is generated.

After that, the machine learning execution unit 116 of the information processing device 1 generates the learning model 134 by performing machine learning using the plurality of learning data generated in the process of S44 (S45). Then, the information processing device 1 terminates the machine learning process.

That is, the information processing apparatus 1 specifies the first parameter corresponding to the feature amount that does not depend on the size of the defect, among the parameters with which the feature amount can be extracted from the first divided image 132 and the second divided image 133. . Then, the information processing device 1 performs machine learning by using the feature quantity corresponding to the specified first parameter.

As a result, by using the generated learning model 134, the information processing apparatus 1 can accurately identify defects of sizes for which a sufficient number of learning images 131 were not prepared, as shown in FIG. be possible. Therefore, the information processing apparatus 1 can accurately determine whether or not the component corresponding to the inspection image includes a defect.

In addition, in the processes of S34 and S35, for example, for each parameter for which the feature amount was extracted in the process of S31, the parameter specifying unit 115 specifies the number of samples for each section in the histogram of the distribution corresponding to each parameter, It may also create a vector of the number of samples whose dimension is the number of partitions. Then, the parameter identifying unit 115 may identify the first parameter by, for example, calculating cosine similarity between vectors corresponding to each parameter. Also, the parameter identifying unit 115 may identify the first parameter by, for example, calculating the degree of similarity between vectors corresponding to each parameter by a histogram intersection method.

[Abnormality identification process]
Next, the process of specifying an inspection image for a component containing a defect (hereinafter also referred to as an abnormality specifying process) will be described. 12 and 13 are flowcharts for explaining the abnormality identification process.

The information receiving unit 111 waits until an inspection image (new image) is received (NO in S51). Specifically, the information receiving unit 111 waits until an inspection image is transmitted from the operation terminal 5 .

Then, when the inspection image is received (YES in S51), the divided image generation unit 113 generates divided images by dividing the inspection image received in the process of S51 into each predetermined size (S52). In this case, the divided image generator 113 may use, as the predetermined size, the geometric mean of the first size used in the process of S24 and the second size used in the process of S25, for example.

Subsequently, the feature amount extraction unit 114 extracts the feature amount corresponding to the first parameter specified in the process of S35 from the divided images generated in the process of S52 (S53).

After that, the score acquisition unit 117 acquires the score output from the learning model 134 in accordance with the input of the feature amount corresponding to the first parameter extracted in the process of S53 for each divided image generated in the process of S52 ( S54).

Then, as shown in FIG. 13, the abnormality determination unit 118 of the information processing device 1 identifies scores below the threshold from the scores acquired in the process of S54 (S61).

As a result, if it is determined that there is a score below the threshold (YES in S62), the abnormality determination unit 118 determines that the divided image corresponding to the score specified in S61 is abnormal among the divided images generated in S52. It is determined to be in the state (S63).

Also, in this case, the abnormality determination unit 118 determines that the inspection image received in the process of S51 is in an abnormal state (S64). That is, the abnormality determination unit 118 determines that a defect is included in the component corresponding to the inspection image received in the process of S51.

After that, the information output unit 119 of the information processing device 1 outputs, for example, information indicating that the inspection image received in the process of S51 is in an abnormal state to the operation terminal 5 (S64).

Then, the information processing device 1 terminates the abnormality identification process. Note that the information processing apparatus 1 similarly terminates the abnormality identification process (NO in S62) when it is determined in the process of S62, for example, that there is no score equal to or lower than the threshold value.

As a result, the information processing apparatus 1 can accurately determine whether or not the component corresponding to the inspection image includes a defect.

The above embodiments are summarized as follows.

(Appendix 1)
An image inspection system having a plurality of information processing devices that perform machine learning on a plurality of learning images,
Each of the plurality of information processing devices,
A plurality of first divided images are generated by dividing the plurality of training images by a first size, and a plurality of second divided images are generated by dividing the plurality of training images by a second size different from the first size. a divided image generation unit that generates divided images;
From each of a plurality of first normal divided images corresponding to a normal state among the plurality of first divided images and a plurality of second normal divided images corresponding to a normal state among the plurality of second divided images , a feature quantity extraction unit that extracts each feature quantity corresponding to a plurality of types of parameters;
a parameter identification unit that identifies, from the plurality of types of parameters, a first parameter that satisfies a predetermined condition for the similarity relationship of the feature amount between the plurality of first normal segmented images and the plurality of second normal segmented images; and ,
a machine learning execution unit that performs machine learning on the plurality of first divided images and the plurality of second divided images by using the feature amount corresponding to the identified first parameter;
An image inspection system characterized by:

(Appendix 2)
In Appendix 1,
The feature amount extraction unit is configured to extract a plurality of first abnormal divided images corresponding to an abnormal state among the plurality of first divided images and a plurality of second divided images corresponding to an abnormal state among the plurality of second divided images. extracting a feature amount corresponding to the first parameter from each of the abnormal segmented images,
The machine learning execution unit indicates that each image is in a normal state, and a feature quantity corresponding to the first parameter extracted from each of the plurality of first normal segmented images and the plurality of second normal segmented images. a plurality of learning data each associated with state information; a feature amount corresponding to the first parameter extracted from each of the plurality of first abnormal segmented images and the plurality of second abnormal segmented images; Perform machine learning with a plurality of learning data each associated with state information indicating an abnormal state,
An image inspection system characterized by:

(Appendix 3)
In Appendix 1,
The parameter identifying unit identifies, as the first parameter, a parameter corresponding to a feature amount that satisfies a predetermined condition for a difference in distribution shape between the plurality of first normal segmented images and the plurality of second normal segmented images. do,
An image inspection system characterized by:

(Appendix 4)
In Appendix 3,
The parameter identifying unit identifies, as the first parameter, a parameter corresponding to a feature quantity in which a difference in center of distribution between the plurality of first normal segmented images and the plurality of second normal segmented images is equal to or less than a threshold. do,
An image inspection system characterized by:

(Appendix 5)
In Appendix 3,
The parameter identifying unit identifies, as the first parameter, a parameter corresponding to a feature quantity in which a difference in distribution width between the plurality of first normal segmented images and the plurality of second normal segmented images is equal to or less than a threshold. do,
An image inspection system characterized by:

(Appendix 6)
In Appendix 2,
The machine learning execution unit generates a learning model by performing the machine learning,
the divided image generating unit, when receiving a new image, generates a plurality of divided images by dividing the new image into predetermined sizes;
The feature amount extraction unit extracts feature amounts corresponding to the first parameters from the plurality of divided images, and
Each of the plurality of information processing devices,
a score acquisition unit that acquires a score output from the learning model according to the input of the feature quantity corresponding to the first parameter for each of the plurality of divided images;
an abnormality determination unit that determines that a divided image having the obtained score equal to or less than a threshold among the plurality of divided images is in an abnormal state;
An image inspection system characterized by:

(Appendix 7)
In Supplementary Note 6, further,
Each of the plurality of information processing devices,
an information output unit that outputs information indicating that the new image is in an abnormal state when it is determined that the divided image in the abnormal state is included in the plurality of divided images;
An image inspection system characterized by:

(Appendix 8)
generating a plurality of first divided images by dividing a plurality of training images into each first size; and generating a plurality of second divisions by dividing the plurality of training images into each of a second size different from the first size. a divided image generation unit that generates an image;
From each of a plurality of first normal divided images corresponding to a normal state among the plurality of first divided images and a plurality of second normal divided images corresponding to a normal state among the plurality of second divided images , a feature quantity extraction unit that extracts each feature quantity corresponding to a plurality of types of parameters;
a parameter identification unit that identifies, from the plurality of types of parameters, a first parameter that satisfies a predetermined condition for the similarity relationship of the feature amount between the plurality of first normal segmented images and the plurality of second normal segmented images; and ,
a machine learning execution unit that performs machine learning on the plurality of first divided images and the plurality of second divided images by using the feature amount corresponding to the identified first parameter;
An image inspection apparatus characterized by:

(Appendix 9)
In Appendix 8,
The feature amount extraction unit is configured to extract a plurality of first abnormal divided images corresponding to an abnormal state among the plurality of first divided images and a plurality of second divided images corresponding to an abnormal state among the plurality of second divided images. extracting a feature amount corresponding to the first parameter from each of the abnormal segmented images,
The machine learning execution unit indicates that each image is in a normal state, and a feature quantity corresponding to the first parameter extracted from each of the plurality of first normal segmented images and the plurality of second normal segmented images. a plurality of learning data each associated with state information; a feature amount corresponding to the first parameter extracted from each of the plurality of first abnormal segmented images and the plurality of second abnormal segmented images; Perform machine learning with a plurality of learning data each associated with state information indicating an abnormal state,
An image inspection apparatus characterized by:

(Appendix 10)
In Appendix 9,
The machine learning execution unit generates a learning model by performing the machine learning,
the divided image generating unit, when receiving a new image, generates a plurality of divided images by dividing the new image into predetermined sizes;
The feature amount extraction unit extracts feature amounts corresponding to the first parameters from the plurality of divided images, and
a score acquisition unit that acquires a score output from the learning model according to the input of the feature quantity corresponding to the first parameter for each of the plurality of divided images;
an abnormality determination unit that determines that a divided image having the obtained score equal to or less than a threshold among the plurality of divided images is in an abnormal state;
An image inspection apparatus characterized by:

(Appendix 11)
generating a plurality of first divided images by dividing a plurality of training images into each first size; and generating a plurality of second divisions by dividing the plurality of training images into each of a second size different from the first size. generate the image,
From each of a plurality of first normal divided images corresponding to a normal state among the plurality of first divided images and a plurality of second normal divided images corresponding to a normal state among the plurality of second divided images , extract feature quantities corresponding to multiple types of parameters,
identifying a first parameter from the plurality of types of parameters that satisfies a predetermined condition for the similarity relationship of the feature amount between the plurality of first normal segmented images and the plurality of second normal segmented images;
Performing machine learning on the plurality of first divided images and the plurality of second divided images by using the feature amount corresponding to the identified first parameter;
An image inspection program characterized by causing a computer to execute processing.

(Appendix 12)
In Supplementary Note 11, further,
From each of a plurality of first abnormal divided images corresponding to an abnormal state among the plurality of first divided images and a plurality of second abnormal divided images corresponding to an abnormal state among the plurality of second divided images , extracting feature quantities corresponding to the first parameters, respectively;
let the computer do the work,
In the process of performing the machine learning, the feature quantity corresponding to the first parameter extracted from each of the plurality of first normal segmented images and the plurality of second normal segmented images, and the fact that each image is in a normal state. a plurality of learning data respectively associated with state information indicating, a feature amount corresponding to the first parameter extracted from each of the plurality of first abnormal segmented images and the plurality of second abnormal segmented images, and each image Perform machine learning with a plurality of learning data each associated with state information indicating that is in an abnormal state,
An image inspection program characterized by:

(Appendix 13)
In Appendix 12,
In the process of performing the machine learning, a learning model is generated by performing the machine learning, and
when receiving a new image, generating a plurality of divided images by dividing the new image into predetermined sizes;
extracting feature amounts corresponding to the first parameters from the plurality of divided images;
Acquiring a score output from the learning model according to the input of the feature amount corresponding to the first parameter for each of the plurality of divided images;
Determining that a divided image for which the acquired score is equal to or less than a threshold among the plurality of divided images is in an abnormal state;
An image inspection program characterized by causing a computer to execute processing.

1: Information processing device 5: Operation terminal NW: Network 10: Information processing system

Claims (9)

An image inspection system having a plurality of information processing devices that perform machine learning on a plurality of learning images,
Each of the plurality of information processing devices,
A plurality of first divided images are generated by dividing the plurality of training images by a first size, and a plurality of second divided images are generated by dividing the plurality of training images by a second size different from the first size. a divided image generation unit that generates divided images;
From each of a plurality of first normal divided images corresponding to a normal state among the plurality of first divided images and a plurality of second normal divided images corresponding to a normal state among the plurality of second divided images , a feature quantity extraction unit that extracts each feature quantity corresponding to a plurality of types of parameters;
a parameter identification unit that identifies, from the plurality of types of parameters, a first parameter that satisfies a predetermined condition for the similarity relationship of the feature amount between the plurality of first normal segmented images and the plurality of second normal segmented images; and ,
a machine learning execution unit that performs machine learning on the plurality of first divided images and the plurality of second divided images by using the feature amount corresponding to the identified first parameter;
An image inspection system characterized by: In claim 1,
The feature amount extraction unit is configured to extract a plurality of first abnormal divided images corresponding to an abnormal state among the plurality of first divided images and a plurality of second divided images corresponding to an abnormal state among the plurality of second divided images. extracting a feature amount corresponding to the first parameter from each of the abnormal segmented images,
The machine learning execution unit indicates that each image is in a normal state, and a feature quantity corresponding to the first parameter extracted from each of the plurality of first normal segmented images and the plurality of second normal segmented images. a plurality of learning data each associated with state information; a feature amount corresponding to the first parameter extracted from each of the plurality of first abnormal segmented images and the plurality of second abnormal segmented images; Perform machine learning with a plurality of learning data each associated with state information indicating an abnormal state,
An image inspection system characterized by: In claim 1,
The parameter identifying unit identifies, as the first parameter, a parameter corresponding to a feature amount that satisfies a predetermined condition for a difference in distribution shape between the plurality of first normal segmented images and the plurality of second normal segmented images. do,
An image inspection system characterized by: In claim 3,
The parameter identifying unit identifies, as the first parameter, a parameter corresponding to a feature quantity in which a difference in center of distribution between the plurality of first normal segmented images and the plurality of second normal segmented images is equal to or less than a threshold. do,
An image inspection system characterized by: In claim 3,
The parameter identifying unit identifies, as the first parameter, a parameter corresponding to a feature quantity in which a difference in distribution width between the plurality of first normal segmented images and the plurality of second normal segmented images is equal to or less than a threshold. do,
An image inspection system characterized by: In claim 2,
The machine learning execution unit generates a learning model by performing the machine learning,
the divided image generating unit, when receiving a new image, generates a plurality of divided images by dividing the new image into predetermined sizes;
The feature amount extraction unit extracts feature amounts corresponding to the first parameters from the plurality of divided images, and
Each of the plurality of information processing devices,
a score acquisition unit that acquires a score output from the learning model according to the input of the feature quantity corresponding to the first parameter for each of the plurality of divided images;
an abnormality determination unit that determines that a divided image having the obtained score equal to or less than a threshold among the plurality of divided images is in an abnormal state;
An image inspection system characterized by: In claim 6, further
Each of the plurality of information processing devices,
an information output unit that outputs information indicating that the new image is in an abnormal state when it is determined that the divided image in the abnormal state is included in the plurality of divided images;
An image inspection system characterized by: generating a plurality of first divided images by dividing a plurality of training images into each first size; and generating a plurality of second divisions by dividing the plurality of training images into each of a second size different from the first size. a divided image generation unit that generates an image;
From each of a plurality of first normal divided images corresponding to a normal state among the plurality of first divided images and a plurality of second normal divided images corresponding to a normal state among the plurality of second divided images , a feature quantity extraction unit that extracts each feature quantity corresponding to a plurality of types of parameters;
a parameter identification unit that identifies, from the plurality of types of parameters, a first parameter that satisfies a predetermined condition for the similarity relationship of the feature amount between the plurality of first normal segmented images and the plurality of second normal segmented images; and ,
a machine learning execution unit that performs machine learning on the plurality of first divided images and the plurality of second divided images by using the feature amount corresponding to the identified first parameter;
An image inspection apparatus characterized by: generating a plurality of first divided images by dividing a plurality of training images into each first size; and generating a plurality of second divisions by dividing the plurality of training images into each of a second size different from the first size. generate the image,
From each of a plurality of first normal divided images corresponding to a normal state among the plurality of first divided images and a plurality of second normal divided images corresponding to a normal state among the plurality of second divided images , extract feature quantities corresponding to multiple types of parameters,
identifying a first parameter from the plurality of types of parameters that satisfies a predetermined condition for the similarity relationship of the feature amount between the plurality of first normal segmented images and the plurality of second normal segmented images;
Performing machine learning on the plurality of first divided images and the plurality of second divided images by using the feature amount corresponding to the identified first parameter;
An image inspection program characterized by causing a computer to execute processing.

Images