JP7781698B2 - Quality Management Systems and Quality Control Programs - Google Patents

Description

The present invention relates to a quality control system, particularly a quality control system using machine learning. Alternatively, the present invention relates to a quality control program using machine learning.

Many image recognition technologies using machine learning have been reported. For example, Patent Document 1 describes an image processing device that includes an image input means for inputting an image, a pixel combination acquisition means for acquiring combinations of two pixels from the input image, a co-occurrence acquisition means for acquiring the co-occurrence of the gradient direction of the luminance of the two pixels related to the acquired combination, an occurrence frequency acquisition means for acquiring the distribution of the occurrence frequency of the acquired co-occurrence while changing the acquired pixel combination, a probability density function generation means for generating a probability density function corresponding to the acquired distribution of the occurrence frequency, and a probability density function output means for outputting the generated probability density function using parameters that define the probability density function.

Furthermore, as a method for determining whether or not a subject has an abnormality using image recognition technology with machine learning, Patent Document 2 describes an anomaly detection system that includes a feature extraction unit that extracts features of the subject from multiple image data continuously acquired about the subject, and a determination unit that determines whether or not the subject has an abnormality based on the features extracted by the feature extraction unit and a predetermined threshold. The feature extraction unit corresponds to an encoder that, together with a decoder and a classifier, constitutes an adversarial autoencoder. The system is adapted by sequentially repeating the following steps: first learning that enables the decoder to restore input image data from the features extracted by the encoder; second learning that enables the classifier to distinguish between samples from a predetermined distribution and the features extracted by the encoder; and third learning that causes the distribution of the features extracted by the encoder to approach a predetermined distribution based on the classification results of the classifier. The threshold is determined by clustering multiple features extracted from multiple normal subjects by the adapted feature extraction unit.

Once learning is complete, a conventional machine learning system saves the difference data between the original image and the image restored using a deep learning algorithm (AutoEncoder) as feature variance. The feature variance is output in gmm format as the average pixel value and variance information for each subdivided image section.

One gmm file is generated for each image section, but gmm files are data that record numerical values. For this reason, when gmm files containing tens of thousands of points are generated, it is impossible for a person to intuitively grasp how the variation in features is distributed, even if they try to analyze it.

Japanese Patent Application Laid-Open No. 2018-124963 Japanese Patent Application Laid-Open No. 2020-177282

One object of one embodiment of the present invention is to provide a quality control system that uses machine learning to easily understand the variability in the characteristics of test objects. Alternatively, one object of one embodiment of the present invention is to provide a quality control program that uses machine learning to easily understand the variability in the characteristics of test objects.

A quality control system according to one embodiment of the present invention includes an autoencoder that generates differential data containing the difference between multiple pieces of training image data and restored image data generated from the training image data, a file generation unit that calculates the mean and variance of the differential data for each predetermined interval to generate a file, and a visualization processing unit that generates an image based on the data in the file.

The autoencoder may extract features from the multiple training image data for each pixel of the multiple training image data, and generate restored image data based on the extracted features, and the file generator may apply a Gaussian mixture model to calculate the mean and variance of the differential data.

The visualization processing unit may generate an image based on the mean value variation or variance value variation contained in the file.

The system may further include a determination unit that determines whether or not there are abnormal values in the difference data, where the determination unit determines whether or not there are abnormal values in the difference data generated from the first image data of the test object, and the visualization processing unit generates a second image including the abnormal values based on the first image data of the test object and the determination result of the determination unit.

The determination unit may set a second interval having a size different from that of the first interval based on the range of the first interval that includes the differential data determined to be an abnormal value, and calculate the average value and variance value of the differential data for each second interval.

The system may further include a determination unit that determines whether or not there are any abnormal values in the differential data, and the determination unit may compare the design data with the differential data to determine whether or not there are any abnormal values in the differential data.

The design data may have a predetermined range of tolerance values for each portion, and the determination unit may compare the tolerance range with the difference data to determine whether or not there are any abnormal values in the difference data, and may assign weights to portions that include abnormal values, and the visualization processing unit may generate an image that includes the weighted portions.

A quality control program according to one embodiment of the present invention includes causing a computer to generate differential data containing the differences between multiple pieces of training image data and restored image data generated from the training image data, causing the computer to calculate the average and variance of the differential data for each predetermined interval to generate a file, and causing the computer to generate an image based on the data in the file.

It is also possible to have a computer extract features for each pixel from multiple pieces of training image data, generate restored image data based on the extracted features, and apply a Gaussian mixture model to have the computer calculate the mean and variance of the difference data.

The computer may generate an image based on the mean value variation or variance value variation contained in the file.

The computer may be caused to determine whether or not there are any abnormal values in the difference data generated from the first image data of the test object, and to generate a second image containing the abnormal values based on the first image data of the test object and the determination result.

Based on the range of a first interval that includes differential data determined to be an abnormal value, the computer may be configured to set a second interval having a size different from that of the first interval, and calculate the mean and variance of the differential data for each second interval.

The design data and the differential data may be compared, and the computer may determine whether or not there are any abnormal values in the differential data.

The design data may have a predetermined range of tolerance values for each part, and the computer may compare the tolerance range with the difference data, determine whether there are any abnormal values in the difference data, weight the parts that contain abnormal values, and generate an image including the weighted parts.

One embodiment of the present invention provides a quality control system that uses machine learning to easily understand variations in the characteristics of test objects. Alternatively, one embodiment of the present invention provides a quality control program that uses machine learning to easily understand variations in the characteristics of test objects.

1 is a block diagram showing a quality control system 100 according to an embodiment of the present invention. FIG. 1 is a flow diagram illustrating an outline of a process according to an embodiment of the present invention. 1A is a flow diagram illustrating step S10 of generating differential data according to one embodiment of the present invention, and FIG. 1B is a conceptual diagram illustrating step S10 of generating differential data according to one embodiment of the present invention. 10A to 10D are diagrams showing the relationship between one pixel (px) of image data and a predetermined section A used by the file generating unit 130. FIG. 10 is a flow diagram illustrating step S20 of calculating the average value and variance value of the difference data and generating a file according to an embodiment of the present invention. FIG. 10 is a flow diagram illustrating a feature visualization step S30 according to an embodiment of the present invention. 10A is a schematic diagram showing an image generated by the visualization processing unit 150, and FIG. 10B is a schematic diagram showing a top view of the product. 1 is a flow diagram illustrating a method for controlling the quality of a product according to an embodiment of the present invention. 10A is a schematic diagram showing an image generated by the visualization processing unit 150, and FIG. 10B is a schematic diagram showing a top view of a test object having a flaw.

One embodiment of the present invention will be described below with reference to the drawings. The embodiment described below is merely an example of an embodiment of the present invention, and the present invention is not limited to these embodiments.

In the drawings referenced in this embodiment, identical parts or parts with similar functions are designated by the same or similar reference numerals (reference numerals consisting of a number followed by A, B, etc.). Furthermore, for the sake of convenience, the dimensional proportions of the drawings may differ from the actual proportions, and some components may be omitted from the drawings. Note that ordinal numbers such as "first," "second," and "third" are used in this specification and elsewhere only to simplify the explanation and should not be interpreted in a limiting sense.

[Quality Control System]
1 is a block diagram showing a quality control system 100 according to an embodiment of the present invention. The quality control system 100 includes, for example, an autoencoder 120, a file generator 130, and a visualization processor 150. In one embodiment, the quality control system 100 further includes, for example, an input unit 110, a determination unit 140, a control unit 160, and an output unit 170. In one embodiment, the quality control system 100 further includes, but is not limited to, a storage unit 180 and a communication unit 190.

Figure 3(B) shows a diagram explaining the autoencoder 120. Figure 4 shows the relationship between one pixel (px) and the predetermined interval A used by the file generator 130 for image data of a product undergoing quality control. Figures 4(A) to 4(D) show image data of e x f pixels. The autoencoder 120 is one of the well-known algorithms for dimensionality reduction using a neural network, and includes an encoder 127 that extracts features by compressing the input layer 121 into an intermediate layer 123, and a decoder 129 that restores the intermediate layer 123 to the output layer 125. In one embodiment, the autoencoder 120 may be a program or module stored in the memory unit 180 and executed by the control unit 160. In this embodiment, the autoencoder 120 receives multiple training image data 10 as input and performs encoding and decoding for each training image data 10. The autoencoder 120 generates restored image data 30 for each of the multiple training image data 10 by encoding and decoding data for each pixel (first section) of the training image data 10. In this process, the autoencoder 120 extracts features contained in the training image data 10 for each pixel and generates difference data between the training image data 10 and the restored image data 30 for each training image data 10. In one embodiment, the difference data is generated as an integer value for each pixel and may include values less than or equal to 0. Note that because the autoencoder 120 imposes a large computational processing load on the quality control system 100, it may be configured as a processing device located outside the quality control system 100 and capable of communicating with the communication unit 190 via a network.

The file generation unit 130 is a program or module that applies a Gaussian mixture model to calculate the mean and variance of the difference data for each predetermined interval (second interval). In one embodiment, the file generation unit 130 is stored in the storage unit 180 and executed by the control unit 160. The file generation unit 130 generates a file containing the mean and variance of the difference data calculated for each predetermined interval. In this specification, the "predetermined interval" can be set taking into account the quality required for products subject to quality control and the size of defects that occur in the products. Note that Figure 4 shows an example in which 3 x 4 pixels is set as the predetermined interval, but this is not limited to this. The variance calculated by the file generation unit 130 has a unimodal or multimodal normal distribution. Note that the autoencoder 120 and the file generation unit 130 may be a single program or module that executes a series of processes.

The visualization processing unit 150 is a program or module that generates an image based on the mean value variation or variance value variation contained in the file generated by the file generation unit 130. In one embodiment, the file generation unit 130 is stored in the memory unit 180 and executed by the control unit 160. In the quality control system 100, the visualization processing unit 150 converts the mean value variation or variance value variation contained in the file generated by the file generation unit 130 into an image that can be seen by humans and outputs it. This makes it possible to visually recognize product manufacturing errors and defects in product quality control, and allows people to easily understand information that is important for reviewing the manufacturing process.

The input unit 110 is an input device for operating the quality control system 100, and may be a known input device such as a keyboard, mouse, and a display device (e.g., a liquid crystal display or an organic EL display). The input unit 110 may also be equipped with an imaging unit 111 that acquires image data to be provided to the autoencoder 120. The imaging unit 111 is a camera that acquires image data, and may use known imaging elements such as a CCD or CMOS.

The determination unit 140 is a program or module that determines whether or not there are abnormal values in the difference data generated by the file generation unit 130. In one embodiment, the determination unit 140 is stored in the storage unit 180 and executed by the control unit 160. In one embodiment, the determination unit 140 can determine whether or not there are abnormal values in the difference data generated from the image data of the test object. Furthermore, in order to make it easier for the user to recognize abnormal values, the determination unit may generate an image in which the section containing the abnormal value is weighted when it detects an abnormal value that falls outside a predetermined acceptable range.

The control unit 160 is composed of a known central processing unit (CPU), operating system (OS), and control programs or modules for controlling the quality control system 100. Alternatively, the control unit 160 may be provided as a single program including the OS and control programs or modules. The control programs or modules that make up the control unit 160 are stored in the memory unit 180 and executed by the CPU.

The output unit 170 is a known output device for outputting the difference data generated by the autoencoder 120, the file generated by the file generation unit 130, or the image generated by the visualization processing unit 150. The output unit 170 also includes a display unit 171. The display unit 171 may be a known display device (e.g., a liquid crystal display or an organic EL display) and is not particularly limited. In one embodiment, the output unit 170 may be a display device equipped with a touch panel that can also function as the input unit 110.

The memory unit 180 is a storage device that stores the operating system (OS) and control programs or modules that constitute the control unit 160, the programs or modules that constitute the autoencoder 120, the programs or modules that constitute the file generation unit 130, the programs or modules that constitute the visualization processing unit 150, the differential data generated by the autoencoder 120, the files generated by the file generation unit 130, or the images generated by the visualization processing unit 150. The memory unit 180 is configured using a known storage device, and may be, for example, a memory, a hard disk, or a solid-state drive (SSD). The memory unit 180 may be built into a computer that includes the control unit 160, or may be located on a server or network drive that can communicate via the communication unit 190.

The communication unit 190 is a known wired or wireless communication device that can be controlled by the control unit 160. The communication unit 190 can be connected to communication networks such as a local area network (LAN), a wide area network (WAN), and the Internet. The communication unit 190 may be a communication device that conforms to wireless communication standards such as Wi-Fi (registered trademark) (a communication method that uses the IEEE 802.11 standard) or Bluetooth (registered trademark). The communication unit 190 may also perform data communication with a server or network drive located outside the computer that has the communication unit 190.

[Processing by quality control system]
The processing performed by the quality control system 100 will be described with reference to FIGS. 2 to 9. FIG. 2 is a flow diagram outlining the processing according to one embodiment of the present invention. The quality control system 100 generates differential data containing the differences between a plurality of sets of learning image data and restored image data generated from the learning image data (step S10). The quality control system 100 calculates the average value and variance value of the differential data for each predetermined section and generates a file (step S20). The quality control system 100 then generates an image based on the data in the generated file (step S30).

Please refer to Figure 3. Figure 3(A) is a flow diagram illustrating step S10 of generating differential data according to one embodiment of the present invention. Figure 3(B) is a conceptual diagram illustrating step S10 of generating differential data according to one embodiment of the present invention. The autoencoder 120 imports multiple training image data 10 (step S11). The multiple training image data 10 can be acquired by the imaging unit 111 capturing an image of a product. Alternatively, the autoencoder 120 may import multiple training image data 10 that has been acquired in advance and stored in the memory unit 180.

The autoencoder 120 compresses the training image data 10 input to the input layer 121 using the encoder 127 to generate the intermediate layer 123. The autoencoder 120 also decompresses the training image data 10 using the decoder 129 to generate the decompressed image data 30 in the output layer 125 (step S15). Through this process, the autoencoder 120 extracts features from the training image data 10 for each pixel (step S13).

More specifically, for example, training image data 10 of e×f pixels includes subpixel data for m pixels. Each pixel includes values (brightness values) of three components (subpixels): red (R), green (G), and blue (B). When the autoencoder 120 performs training using n pieces of training image data 10, the first to nth pieces of training image data 10 are compressed by the encoder 127 and decompressed by the decoder 129 for each color component. The autoencoder 120 generates difference data from the decompressed image data 30 for each of the multiple pieces of training image data 10. For example, if the first pixel of the first piece of training image data is ( _R11 , _G11 , _B11 ), the mth pixel is ( _R1m , _G1m , _B1m ), the first pixel of the nth piece of training image data is ( _Rn1 , _Gn1 , _Bn1 ), and the mth pixel is ( _Rnm , _Gnm , _Bnm ), the differential data generated by the autoencoder 120 will be ( _ΔR11 , _ΔG11 , _ΔB11 ) for the first pixel of the first piece of training image data and ( _ΔR1m , _ΔG1m , _ΔB1m ) for the mth pixel, and ( _ΔRn1 , _ΔGn1 , _ΔBn1 ) for the first pixel of the nth piece of training image data and ( _ΔRnm , _ΔGnm , _ΔBnm ) for the mth section.

In this embodiment, by performing machine learning by repeating step S10 using multiple pieces of training image data 10, the value of the difference data between the training image data 10 and the restored image data 30 approaches zero. The obtained data may be processed in the next step S20, or may be stored in the memory unit 180.

In one embodiment, a person may use image processing software or the like to process the learning image data 10 to correct the difference data generated by the autoencoder 120. Alternatively, if the difference data contains an abnormal value that is equal to or greater than a specified value, the quality control system 100 may recognize the abnormal value and display an abnormality.

Please refer to Fig. 5. Fig. 5 is a flow diagram illustrating step S20 of calculating the mean value and variance value of difference data and generating a file according to an embodiment of the present invention. The file generation unit 130 calculates the mean value and variance value for each predetermined section of the difference data generated for each pixel by the autoencoder 120. The file generation unit 130 generates a file by applying a Gaussian mixture model for each section based on multiple pieces of difference data for each pixel. For example, the differential data for the first section is defined as (μ _R1 μ _G1 μ _B1 σ _R1 σ _G1 σ _B1 σ _RG1 σ _GB1 σ _BR1 ), the differential data for the second section is defined as (μ _R2 μ _G2 μ _B2 σ _R2 σ _G2 σ _B2 σ _RG2 σ _GB2 σ _BR2 ), and the differential data for the kth section is defined as (μ _Rk μ _Gk μ _Bk σ _Rk σ _Gk σ _Bk σ _RGk σ _GBk σ _BRk ).

For example, the file generating unit 130 calculates the average μ for the color component for each section. For example, the average μ _R1 of the red (R) component for section 1/k is calculated by the following formula (1).

The file generator 130 also calculates the variance σ for the color components in each section. For example, the variance σ _R1 of the red (R) component in the section 1/k is calculated using the following formula (2).

The file generator 130 also calculates the covariance σ for the color components for each interval (step S21). For example, the covariance σ _RG1 of the red (R) and green (G) components for the interval 1/k is calculated using the following formula (3):

The file generation unit 130 outputs the calculation results as a normal distribution (step S23).

The resulting file may be processed in the next step S30 or stored in the memory unit 180.

The predetermined interval (second interval) for generating the file in step S20 may be set manually via the input unit 110, taking into consideration the quality required for the product being quality controlled and the size of defects that may occur in the product. Reducing the size of the predetermined interval increases the accuracy of the generated file, but increases the amount of calculations and the processing time in the file generation unit 130. In one embodiment, when the second interval is provided as the initial value, the determination unit 140 may reset a third interval having a size different from that of the second interval based on the range of the second interval that includes the difference data determined to be an abnormal value. The file generation unit 130 may then perform calculations using the difference data for each more appropriate third interval to generate a file containing the mean and variance of the difference data. By repeating this resetting process (refinement), the optimal section may be set.

Furthermore, since (A) the maximum section size capable of detecting the anticipated defect ≦ (X) the optimal section size ≦ (B) the minimum section size capable of absorbing variation in the training image data 10, the size of (A) is calculated from the product specifications, and a section setting (X) equal to or greater than the size of (A) is provisionally set, taking into account the calculation processing time. Model data is generated from the training image data 10 by the file generation unit 130. From the generated model data, an image is generated and analyzed by the visualization processing unit 150. If the model data shows a distribution with little variation, learning is completed. On the other hand, if the model data shows a distribution with large variation, the size of (X) is determined to be equal to or greater than the size of (B), and the setting value is automatically changed so that the size of (X) is equal to or less than the size of (B), and model data is generated from the training image data 10 by the file generation unit 130. In this way, the quality control system 100 can automatically set the optimal section size.

In one embodiment, when performing the above calculations for each predetermined section of the training image data 10, the file generation unit 130 preferably shifts the predetermined section relative to the training image data 10 so that a portion of the second section overlaps the first section. For example, in FIG. 4(A), section A including the first pixel is set. The file generation unit 130 shifts section A in the X-axis direction to calculate the mean and variance. For example, as shown in FIG. 4(B), the file generation unit 130 sets section A' so that the two pixels on the right side of section A overlap with the two pixels on the left side of the shifted section A'. Furthermore, as shown in FIG. 4(C), the file generation unit 130 sets section B so that the two pixels on the bottom side of section A overlap with the two pixels on the top side of section B after shifting in the Y-axis direction. Furthermore, as shown in FIG. 4(D), the file generation unit 130 sets section B' so that the two pixels on the right side of section B overlap with the two pixels on the left side of the shifted section B'. That is, the file generation unit 130 sets section B so that the two pixels on the right and two pixels below section A overlap with the two pixels on the left and two pixels above section B after movement in the Y-axis direction.

Here, if the size of the learning image data 10 is length (Him) × width (Wim), the size of a predetermined section is length (Hsp) × width (Wsp), and the movement amount of the section is length (Hsd) × width (Wsd), then:
H = [(Him - Hsp) / Hsd] + 1
W = [(Wim - Wsp) / Wsd] + 1
* Square brackets [ ] indicate truncation to the nearest integer.
The number of files generated is H × W. By setting the sections before and after the movement to overlap in this manner, the file generation unit 130 can generate more accurate mean and variance values, preventing product defects from being overlooked. On the other hand, overlapping the sections before and after the movement increases the number of files generated compared to when the sections are not overlapped, which increases the time required for calculation processing and increases the load on the file generation unit 130. For this reason, the ratio at which the sections before and after the movement are overlapped, i.e., the amount of movement of the sections, may be set taking into account the calculation speed, etc. Alternatively, the file generation unit 130 may be configured as a calculation processing device located outside the quality control system 100 and capable of communicating with the communication unit 190 via a network.

The visualization processing unit 150 aggregates the files output by the file generation unit 130 into an image. Figure 6 is a flow diagram illustrating step S30 of visualizing features according to one embodiment of the present invention. The visualization processing unit 150 aggregates the data obtained for each section into a single image (step S31). The visualization processing unit 150 combines the mean values or variance values output by the file generation unit 130 based on the pixel position information of the difference data. As a result, the visualization processing unit 150 outputs the mean values or variance values for each color component as an image (step S33). The number of images of mean values output is the number of color components, and the number of images of variance values output is the square of the number of color components.

The display unit 171 can display the image output by the visualization processing unit 150, allowing the user to visually recognize variations in the characteristics of the test object. Alternatively, the visualization processing unit 150 may store the image in the storage unit 180, or output it to a printer via the output unit 170. Furthermore, the communication unit 190 may send the image to a server or network drive.

The image aggregated by the visualization processing unit 150 may differ from human intuition. In one embodiment, a manually edited image may be overwritten as visualization information for a new file. This allows the file output by the file generation unit 130 to be corrected to an ideal state based on the visualization information, correcting unexpected variations in the image generated from the learning image data 10 and making a correct judgment.

[Evaluation of manufacturing errors]
In one embodiment, when machine learning is performed in the quality control system 100 using non-defective products (good or acceptable products) as the training image data 10, a user can visually recognize variations in the products. Specifically, the visualization processing unit 150 generates an image based on the mean value variation or variance value variation contained in the file. FIG. 7(A) is a schematic diagram showing an image generated by the visualization processing unit 150, and FIG. 7(B) is a schematic diagram of a top view of the product. As an example, in FIG. 7(A), portions of the product with large variations are displayed in dark colors. From the image in FIG. 7(A), it can be recognized that large variations, i.e., large manufacturing errors, occur in the area surrounded by the dashed line in the product shown in FIG. 7(B).

[Product Evaluation]
In one embodiment, after performing the above-described machine learning, the quality control system 100 can evaluate a product as a test object. See FIG. 8 . FIG. 8 is a flow diagram illustrating a product quality control method according to one embodiment of the present invention. The imaging unit 111 captures an image of the product as a test object, and the autoencoder 120 inputs the image data of the test object as the input layer 121 (step S50). The autoencoder 120 compresses and decompresses the image data of the test object. At this time, even if the test object has a flaw, the machine-learned autoencoder 120 generates decompressed image data 30 without the flaw. The file generation unit 130 compares the image data of the test object input to the autoencoder 120 with the decompressed image data 30 to generate differential data including an abnormal value indicating a flaw in the test object (step S60). The determination unit 140 determines that the test object has a flaw based on the abnormal value contained in the differential data. Based on the determination result by the determination unit 140, the visualization processing unit 150 generates an image indicating the presence of a flaw in the image data of the test object (step S70).

In one embodiment, the quality control system 100 may generate differential data by comparing the image data of the test object with the restored image data 30 generated during learning, without using the autoencoder 120. This process omits the calculation process in the autoencoder 120, and although the accuracy of the differential data may be slightly reduced, it is possible to reduce the overall processing time.

In this embodiment, the autoencoder 120 generates differential data between the image data of the test object and the restored image data, and determines whether there are any scratches or other defects on the test object from the abnormal values contained in the differential data. However, as a variant, the autoencoder 120 can also determine whether there are any scratches or other defects on the test object because it is unable to reconstruct the restored image data when it generates the restored image data from the image data of the test object.

Figure 9(A) is a schematic diagram showing an image generated by the visualization processing unit 150, and Figure 9(B) is a schematic diagram of a top view of a test object having a flaw. As an example, in Figure 9(A), in order to make it easier for the user to visually recognize that the test object has a flaw, the visualization processing unit 150 colors the areas where abnormal values are detected and generates an image that is combined with the image data of the test object. From the image in Figure 9(A), it can be recognized that the test object shown in Figure 9(B) has a flaw in the area surrounded by a dashed line. The visualization processing unit 150 may also draw a line in the area adjacent to the area where the abnormal value was detected, and display a circle, oval, or other shape surrounding the flaw in the test object on the image, which is then displayed on the display unit 171.

[Comparison of design data and product]
In one embodiment, machine learning is performed using image data of a non-defective product, and then the design data of the product is evaluated, thereby enabling evaluation of manufacturing errors between the design data and the product. Specifically, the file generation unit 130 compares the design data input to the autoencoder 120 with the restored image data 30 to generate differential data including anomalies indicating differences between the design data and the restored image data 30. The determination unit 140 determines, based on the anomalies included in the differential data, that the product deviates (or differs) from the design data. Based on the determination result by the determination unit 140, the visualization processing unit 150 can generate an image indicating that the product deviates from the design data.

To make the above-mentioned abnormal values and variations easier for users to visually recognize, when the determination unit 140 detects an abnormal value in the difference data, the determination unit 140 may weight the portion containing the abnormal value, and the visualization processing unit 150 may generate an image including the weighted portion. In other words, the visualization processing unit 150 can generate an image in which the weighted portion is emphasized.

[Section resetting process]
As described above, it is possible to reset the size of the intervals used by the file generation unit 130. When the second interval is given as an initial value, the determination unit 140 resets a third interval having a size different from that of the second interval based on the range of the second interval including the difference data determined to be an abnormal value or the range of the second interval including the difference data determined to have a large variation, and the file generation unit 130 can perform arithmetic processing on the difference data for each more appropriate third interval to generate a file including the average value and variance value of the difference data.

[Quality Control Program]
In one embodiment of the present invention, a program for executing each of the above-described processes can be provided. Alternatively, in one embodiment, the program can be provided as a recording medium storing the program. This will be described with reference to FIGS. 2 to 9.

This program causes the quality control system to generate differential data containing the differences between multiple training image data and restored image data generated from the training image data (step S10). This program also causes the autoencoder 120 to import multiple training image data 10 (step S11).

This program causes the autoencoder 120 to compress the training image data 10 input to the input layer 121 using the encoder 127, generating the intermediate layer 123. The program also causes the decoder 129 to decompress the data in the output layer 125, generating the decompressed image data 30 (step S15). Through this process, the program causes the autoencoder 120 to extract features from the training image data 10 for each pixel (step S13).

More specifically, as described above, this program compresses and decompresses the three components of red (R), green (G), and blue (B) for each pixel of the learning image data 10 to generate differential data. This program processes the resulting differential data in the next step S20 or stores it in the memory unit 180.

In this embodiment, by performing machine learning by repeating step S10 using training image data, the program can bring the value of the difference data between the training image data and the restored image data closer to zero.

This program causes the quality control system 100 to calculate the mean and variance of the difference data for each specified interval and generate a file (step S20). This program causes the file generation unit 130 to calculate the mean and variance for each interval from the difference data generated by the autoencoder 120. As described above, this program causes the file generation unit 130 to calculate the mean μ for the color component for each interval from the obtained difference data. This program also causes the file generation unit 130 to calculate the variance σ for the color component for each interval. This program also causes the file generation unit 130 to calculate the covariance σ for the color component for each interval (step S21).

This program causes the file generation unit 130 to output the calculation results as a normal distribution (step S23). The resulting file may be processed in the next step S30, or may be stored in the storage unit 180.

The program causes the quality control system 100 to generate an image based on the data in the generated file (step S30). Because the program calculates the average and variance of the difference data for each specified interval and generates a file, the program may set the intervals by considering the balance between accuracy and processing time. The intervals can also be set manually via the input unit 110. Alternatively, when a second interval is provided as the initial value, the program causes the determination unit 140 to reset a third interval having a different size from the second interval based on the range of the second interval that includes the difference data determined to be an abnormal value, and then causes the file generation unit 130 to perform arithmetic processing using the difference data for each more appropriate third interval, thereby generating a file containing the average and variance of the difference data. By repeating this resetting process (refinement), the optimal intervals can be set.

This program causes the visualization processing unit 150 to aggregate the data obtained for each section into a single image (step S31). This program causes the visualization processing unit 150 to combine the mean values or variance values output by the file generation unit 130 based on the pixel position information of the difference data. This program causes the visualization processing unit 150 to output the mean values or variance values for each color component as an image (step S33).

This program can display the image output by the visualization processing unit 150 on the display unit 171, allowing the user to visually recognize variations in the characteristics of the test object. Alternatively, the visualization processing unit 150 may store the image in the memory unit 180, or output it to a printer via the output unit 170. Furthermore, this program may cause the communication unit 190 to transmit the image to a server or network drive.

[Manufacturing error evaluation program]
In one embodiment, when machine learning is performed in the quality control system 100 using non-defective products (good or acceptable products) as the training image data 10, the user can visually recognize variations in the products. The manufacturing error evaluation program can cause the visualization processing unit 150 to generate an image based on the mean value variation or variance value variation contained in the file. As an example, as shown in FIG. 7A, the manufacturing error evaluation program displays areas with large variations in dark colors. From the image in FIG. 7A, it can be recognized that large variations, i.e., large manufacturing errors, occur in the area surrounded by the dashed line in the product shown in FIG. 7B.

[Product Evaluation Program]
In one embodiment, the product evaluation program performs the above-described machine learning and then causes the quality control system 100 to evaluate the product, which is the test object. The product evaluation program causes the imaging unit 111 to capture an image of the product, which is the test object, and causes the autoencoder 120 to input the image data of the test object as the input layer 121 (step S50). The product evaluation program causes the autoencoder 120 to compress and decompress the image data of the test object. At this time, even if the test object has a flaw, the machine-learned autoencoder 120 generates decompressed image data 30 without the flaw. The product evaluation program causes the file generation unit 130 to compare the image data of the test object input to the autoencoder 120 with the decompressed image data 30 and generate differential data including an abnormal value indicating a flaw in the test object (step S60). The product evaluation program causes the determination unit 140 to determine that the test object has a flaw based on the abnormal value contained in the differential data. Based on the determination result by the determination unit 140, the product evaluation program causes the visualization processing unit 150 to generate an image indicating the presence of a flaw in the image data of the test object (step S70).

As an example, in Figure 9(A), in order to make it easier for the user to visually recognize the presence of a flaw in the test object, the product evaluation program causes the visualization processing unit 150 to color the area where an abnormal value was detected and generate an image that combines this with the image data of the test object. From the image in Figure 9(A), it can be recognized that the test object shown in Figure 9(B) has a flaw in the area surrounded by a dashed line. The product evaluation program also causes the visualization processing unit 150 to draw a line in the area close to the area where the abnormal value was detected, and display a circle, oval, or other shape surrounding the flaw in the test object on the image, which can be displayed on the display unit 171.

[Design data and product comparison program]
In one embodiment, machine learning is performed using image data of a non-defective product, and then the design data of the product is evaluated, thereby enabling evaluation of manufacturing errors between the design data and the product. The comparison program causes the file generation unit 130 to compare the design data input to the autoencoder 120 with the restored image data 30 to generate differential data including an abnormal value indicating a difference between the design data and the restored image data 30. The comparison program causes the determination unit 140 to determine, based on the abnormal value included in the differential data, that the product deviates (or differs) from the design data. Based on the determination result by the determination unit 140, the comparison program causes the visualization processing unit 150 to generate an image indicating that the product deviates from the design data.

To make it easier for users to visually recognize the above-mentioned abnormal values and variations, a program according to one embodiment weights the portion containing the abnormal value when the determination unit 140 detects an abnormal value in the difference data, and causes the visualization processing unit 150 to generate an image containing the weighted portion. In other words, the comparison program causes the visualization processing unit 150 to generate an image in which the weighted portion is emphasized.

[Section resetting processing program]
As described above, it is possible to reset the size of the intervals used by the autoencoder 120. When the second interval is given as an initial value, the resetting processing program causes the determination unit 140 to reset a third interval having a size different from that of the second interval based on the range of the second interval including the difference data determined to be an abnormal value or the range of the second interval including the difference data determined to have a large variation, causes the autoencoder 120 to calculate the difference data for each more appropriate third interval, and causes the file generation unit 130 to generate a file including the average value and variance value of the difference data.

The above describes one embodiment of the present invention with reference to the drawings, but the present invention is not limited to the above embodiment and can be modified as appropriate without departing from the spirit of the present invention. For example, if a person skilled in the art adds, deletes, or modifies components as appropriate based on the quality control system or quality control program of this embodiment, this also falls within the scope of the present invention as long as the gist of the present invention is maintained. Furthermore, the above-mentioned embodiments can be combined as appropriate as long as there are no mutual contradictions, and technical matters common to each embodiment are included in each embodiment even if not explicitly stated.

Even if there are other effects and advantages different from those brought about by the aspects of each of the above-mentioned embodiments, if they are clear from the description in this specification or can be easily predicted by a person skilled in the art, they are naturally understood to be brought about by the present invention.

100 Quality control system, 110 Input unit, 120 Autoencoder, 130 File generation unit, 140 Determination unit, 150 Visualization processing unit, 160 Control unit, 170 Output unit, 180 Storage unit, 190 Communication unit

Claims (12)

an autoencoder that generates differential data including a difference between data for each first section set at a size of one pixel of a plurality of learning image data of a product and restored image data generated from the data for each first section of the learning image data;
a file generating unit that calculates an average value and a variance value of the difference data for each second section set to a predetermined pixel size and generates a file;
a visualization processing unit that generates an image by combining the average value or the variance value included in the file based on position information of the first section of the difference data. the autoencoder extracts features from the plurality of training image data for each of the first sections of the plurality of training image data, and generates restored image data based on the extracted features;
The quality control system according to claim 1 , wherein the file generation unit calculates a mean value and a variance value of the difference data by applying a Gaussian mixture model. a determination unit that determines whether or not there is an abnormal value in the difference data,
the determining unit determines whether or not an abnormal value exists in the difference data generated from the first image data of the test object;
The quality control system according to claim 1 , wherein the visualization processing unit generates a second image including the abnormal value based on the first image data of the test object and the determination result of the determination unit. 4. The quality control system according to claim 3, wherein the determination unit sets the second interval based on the range of the first interval including the differential data determined to be the abnormal value, and calculates the average value and variance value of the differential data for each of the second intervals. a determination unit that determines whether or not there is an abnormal value in the difference data,
The quality control system according to claim 1 , wherein the determining unit compares the design data with the difference data to determine whether or not there is an abnormal value in the difference data. the design data has a predetermined range of tolerance for each part,
the determination unit compares the difference data with the range of allowable values to determine whether or not there is an abnormal value in the difference data, and weights the portion including the abnormal value;
The quality control system according to claim 5 , wherein the visualization processing unit generates the image including the weighted portion. generating differential data including a difference between data for each first section set by one pixel of a plurality of learning image data of a product and restored image data generated from the data for each first section of the learning image data, and
causing the computer to calculate an average value and a variance value of the difference data for each second section set at a predetermined pixel size, and generating a file;
causing the computer to generate an image in which the average value or the variance value included in the file is combined based on position information of the first section of the difference data. causing the computer to extract features for each of the first sections from a plurality of learning image data, and causing the computer to generate restored image data based on the extracted features;
8. The quality control program according to claim 7 , wherein the computer calculates a mean value and a variance value of the difference data by applying a Gaussian mixture model. causing the computer to determine whether or not there is an abnormal value in the difference data generated from the first image data of the test object;
9. The quality control program according to claim 7 , further comprising causing the computer to generate a second image including the abnormal value based on the first image data of the test object and the determination result. 10. The quality control program according to claim 9, wherein the computer is caused to set the second interval based on the range of the first interval including the difference data determined to be the abnormal value, and to calculate the average value and variance value of the difference data for each of the second intervals. 9. The quality control program according to claim 7 , wherein the computer compares design data with the difference data to determine whether or not there is an abnormal value in the difference data. the design data has a predetermined range of tolerance for each part,
causing the computer to compare the range of allowable values with the difference data, determine whether or not there is an abnormal value in the difference data, and weight the portion including the abnormal value;
The quality control program of claim 11 , further comprising causing the computer to generate the image including the weighted portions.