JP7643102B2 - Quality evaluation equipment and inspection management system - Google Patents

Description

The present invention relates to a quality evaluation device and an inspection management system that evaluate the quality of inspection objects such as printed circuit boards.

In the field of FA (Factory Automation), a process is realized in which an image of an inspection target such as a workpiece is captured and input into a pre-trained supervised AI model to inspect the quality of the workpiece. The quality of the workpieces mass-produced at production sites is constantly changing, and as the change becomes greater, the supervised AI model is no longer able to make correct judgments, increasing the risk of overlooking or overlooking something. Therefore, it is necessary to retrain the supervised AI model using newly obtained learning data and use the most appropriate supervised AI model in response to the change in quality. For example, if the workpiece is a printed circuit board, examples of the quality of the workpiece include the appearance of the solder and the performance of visual inspection of the component positions, etc.

For example, in Patent Document 1, a model for image judgment can be generated that can appropriately judge data with new features that have not been seen before by training the model on new images of good and bad products.

However, the characteristics of the workpiece can change due to various factors such as the temperature at the time, variations in the material, and the condition of the inspection equipment and the equipment used in the previous process, making it difficult for users to determine when the workpiece characteristics will change and when the supervised AI model should be retrained.

JP 2020-107104 A

The present invention has been made in consideration of the above problems, and the problem that the present invention aims to solve is to realize highly reliable inspection by capturing changes in the quality of the object to be inspected and encouraging re-learning of a trained learning device using supervised machine learning that is used in the inspection process.

To solve the above problems, the present invention provides:
a good-quality image acquiring unit that acquires a good-quality image, which is an inspection image determined to be a good-quality image in an inspection process using a trained first learning device generated by machine learning the inspection results as teacher data, using an inspection image generated by imaging an inspection object as learning data;
a quality change evaluation unit for evaluating a change in the quality of the inspection object;
a quality evaluation information generating unit that generates quality evaluation information including a result of the evaluation;
The quality evaluation device is provided with the above.

According to this, the user can recognize the change in quality of the inspection object by the quality evaluation information including the result of evaluation by the quality change evaluation unit that evaluates the change in quality of the inspection object. The user can grasp the change in quality of the inspection object by the quality evaluation information and determine the need for re-learning of the first learning device used in the inspection process, so that the user can be prompted to re-learn the first learning device and realize a highly reliable inspection.
Here, the good product images include inspection images generated by imaging an inspection object that has been determined to be good by the trained first learning device, and inspection images generated by imaging an inspection object that has been determined to be defective by the trained first learning device but is subsequently determined to be good in a visual inspection process.

In the present invention,
The quality change evaluation unit
a quality evaluation unit that outputs a quality evaluation index for evaluating the quality based on the non-defective product image;
a quality evaluation index storage unit that stores the quality evaluation index;
a change determination unit that determines a change in the quality based on a change over a predetermined period of time of the quality evaluation index stored in the quality evaluation index storage unit;
The above formula may be used.

This makes it possible to grasp changes in the quality of the test object based on changes over a specified period of time in the quality evaluation index, which changes over time, and determine the need for re-learning the first learning device used in the test process, thereby encouraging the user to re-learn the first learning device and achieving highly reliable testing.

In the present invention,
the quality evaluation index is an anomaly degree output by a trained second learning device generated by machine learning through unsupervised learning using the non-defective product image as training data;
The change determination unit may calculate an average value and a standard deviation of the degree of abnormality over a predetermined period of time, and compare the average value and the standard deviation with a first threshold value and a second threshold value, respectively, to determine the change in the quality.

According to this, the degree of abnormality output by the trained second learning device based on images of good products can be used to capture changes in the quality of the inspection object and determine the need for re-learning the first learning device used in the inspection process, thereby encouraging the user to re-learn the first learning device and achieving highly reliable inspections.

The device may be provided with a setting unit that automatically sets at least one of the first threshold value and the second threshold value.

In this way, the first threshold and the second threshold are automatically set by the setting unit, eliminating the need for the user to set them, and enabling appropriate settings using machine learning, etc.

In the present invention,
The quality change evaluation unit
Clustering may be performed using a trained third learning device generated by unsupervised machine learning using the good product images as training data, and the number of outliers may be compared with a third threshold to evaluate the change in quality.

According to this, by using clustering performed by the trained third learning device based on images of good products, it is possible to capture changes in the quality of the object to be inspected and determine the need for re-training the first learning device used in the inspection process, thereby encouraging the user to re-train the first learning device and achieving highly reliable inspection.

In the present invention,
The image forming apparatus may further include a third threshold setting unit that automatically sets the third threshold.

The third threshold is automatically set by the third threshold setting unit, eliminating the need for user setting and enabling appropriate setting using machine learning, etc.

In the present invention,
The quality evaluation information may include information recommending relearning of the first learning device.

This allows the user to recognize the results of the quality evaluation of the test object, and the information recommending re-learning the first learning device directly urges the user to re-learn the first learning device, which leads to more reliable re-learning of the first learning device and enables highly reliable testing.

In the present invention,
The quality evaluation information may include information recommending improvements to a front-end process related to the inspection target.

This allows the user to recognize the results of the quality evaluation of the inspection object and recommend improvements to the upstream process related to the inspection object, so that improvements to the upstream process can lead to improvements in the quality of the inspection object.

In the present invention,
The apparatus may further include a display unit for displaying the quality evaluation information.

This allows the user to visually recognize the contents of the quality evaluation information via the display unit.

The present invention also provides a method for producing a method for manufacturing a semiconductor device comprising the steps of:
The quality evaluation device;
The inspection management system includes an inspection device having an inspection processing unit that performs inspection processing on the inspection object using the trained first learning device, and a memory unit that stores the inspection image provided to the inspection processing and the inspection result that is the result of the inspection processing.

According to this,
The user can use the quality evaluation information from the quality evaluation device to grasp changes in the quality of the object to be inspected and determine the need for re-learning of the first learning device used in the inspection process, thereby configuring an inspection management system that encourages the user to re-learn the first learning device and enables highly reliable inspections.

In the present invention,
The system may further include a learning device for re-learning the first learning device.

This makes it possible to configure an inspection management system that can realize highly reliable inspections by re-training the first learning device at appropriate times prompted by the quality evaluation information.

According to the present invention, highly reliable inspection can be achieved by capturing changes in the quality of the object to be inspected and encouraging re-learning of a trained learning device using supervised machine learning that is used in the inspection process.

1 is a diagram showing a schematic configuration of a manufacturing facility according to a first embodiment of the present invention; 1 is a functional block diagram of an inspection device according to a first embodiment of the present invention. FIG. 2 is a functional block diagram of a management device according to the first embodiment of the present invention. FIG. 4 is a diagram showing an example of quality evaluation information according to the first embodiment of the present invention. FIG. 11 is a diagram showing another example of the quality evaluation information according to the first embodiment of the present invention. FIG. 11 is a diagram showing another example of the quality evaluation information according to the first embodiment of the present invention. FIG. 11 is a diagram showing another example of the quality evaluation information according to the first embodiment of the present invention. FIG. 1 is a functional block diagram of a supervised learning device according to a first embodiment of the present invention. 1 is a functional block diagram of an unsupervised learning device according to a first embodiment of the present invention. FIG. 11 is a functional block diagram of a management device according to a second embodiment of the present invention. FIG. 11 is a functional block diagram of an unsupervised learning device according to a second embodiment of the present invention. FIG. 11 is a diagram illustrating a quality evaluation according to a second embodiment of the present invention. FIG. 11 is a diagram showing an example of quality evaluation information according to Application Example 2 of the present invention.

[Application example]
Hereinafter, application examples of the present invention will be described with reference to the drawings.

Figure 1 shows a schematic configuration of a management system 1 including a management device 10 to which the present invention is applied. In Figure 1, the management system 1 includes manufacturing equipment in a surface mounting line for printed circuit boards, which are a solder printing device X1, a mounter X2, and a reflow furnace X3, and inspection equipment, which are a solder printing inspection device Y1, a component inspection device Y2, an appearance inspection device Y3, and an X-ray inspection device Y4.

The inspection processing function of the visual inspection device Y3 is realized by the camera 31, the information processing device 32, and the display device 33 as shown in FIG. 2. Here, the inspection program read from the inspection program storage unit 324 is executed by the CPU, and the inspection processing for the inspection image is executed in the inspection unit 323. As the inspection program, a learning device 215 (see FIG. 8) that has been machine-learned by supervised learning using the inspection image as learning data and the inspection result as teacher data is used. Since the quality of the mass-produced printed circuit boards to be inspected is constantly changing, if the quality changes significantly, the learning device using supervised learning will not be able to correctly judge whether the board is good or bad, and the risk of overlooking or overlooking increases. For this reason, it is desirable to retrain the learning device using newly obtained inspection images and perform inspection using an inspection program that corresponds to the changing quality. The inspection unit 323 corresponds to the inspection processing unit of the present invention.

In the management device 10 shown in FIG. 3, the quality evaluation unit 103 performs quality evaluation using a quality evaluation program on good-quality images, which are inspection images of inspection objects determined to be good products among the inspection images acquired from the inspection device. Furthermore, the quality change determination unit 105 evaluates the change in quality of the inspection object based on the accumulated quality evaluation results. Then, the quality evaluation information generation unit 106 generates quality evaluation information including the evaluation results regarding the quality change, and displays it on the display unit 107.

Specifically, the quality assessment program is a learning device that is trained by unsupervised learning using a learning device 22 as shown in FIG. 9, and is an unsupervised AI model such as VAE that uses images of good products for learning as learning data.

The quality evaluation information 71 includes, for example, information as shown in Fig. 4. The control chart 711 plots the average value of the degree of abnormality over a predetermined period, which is output by inputting images of non-defective products, which are mass production data, into a quality evaluation program, and indicates the average value with a dashed line, and indicates the standard deviation of the degree of abnormality over the predetermined period with a solid vertical bar. The threshold value (degree of abnormality) displayed in the display area 712 of the quality evaluation information 71 and the threshold value (variation) displayed in the display area 714 are set to "0.5" and "0.4", respectively. In the control chart 711, the value of the degree of abnormality exceeds the threshold value (degree of abnormality) indicated by the solid line parallel to the horizontal axis, and the standard deviation indicated by the vertical bar also exceeds the threshold value (variation).

The quality evaluation information 71 displays an overall evaluation of the quality of the inspection target based on the set threshold (degree of abnormality) and threshold (variation) and the change in the degree of abnormality shown in the control chart 711. Here, the message "There is variation and change in quality. We recommend reviewing the process and re-learning the AI model" is displayed. This allows the user to recognize that there is a large change in the quality of mass-produced printed circuit boards, and that it is necessary to review the process and re-learn the learning device of the inspection program. Therefore, by re-learning the learning device of the inspection program in a timely manner, highly reliable inspection is possible. Here, the above message corresponds to information recommending improvements to the pre-process related to the inspection target of the present invention.

Example 1
Hereinafter, the management system 1 according to the first embodiment of the present invention will be described in more detail with reference to the drawings.
(System Configuration)
1 is a schematic diagram showing an example of the configuration of manufacturing equipment in a surface mounting line for printed circuit boards according to this embodiment. Surface mounting technology (SMT) is a technique for soldering electronic components to the surface of a printed circuit board, and a surface mounting line is mainly composed of three processes: solder printing, component mounting, and reflow (solder deposition).

As shown in FIG. 1, the surface mounting line is provided with the following manufacturing equipment from the upstream side: a solder printing device X1, a mounter X2, and a reflow furnace X3. The solder printing device X1 is a device that prints solder paste on the electrodes (called lands) on a printed circuit board by screen printing. The mounter X2 is a device that picks up electronic components to be mounted on the board and places the components on the solder paste at the corresponding location, and is also called a chip mounter. The reflow furnace X3 is a heating device that heats and melts the solder paste, cools it, and solders the electronic components onto the board. When there are a large number and types of electronic components to be mounted on the board, multiple mounters X2 may be provided on the surface mounting line.

The surface mounting line is also equipped with a system that inspects the condition of the boards at the exit of each process - solder printing, component mounting, and reflow - and automatically detects defects or potential defects. In addition to automatically separating good and defective products, the system also has the function of providing feedback to the operation of each manufacturing device based on the inspection results and their analysis (for example, by changing the mounting program).

The solder print inspection device Y1 is a device for inspecting the printing condition of the solder paste on the board discharged from the solder printing device X1. The solder print inspection device Y1 measures the solder paste printed on the board in two or three dimensions, and judges whether or not various inspection items are normal (within the acceptable range) based on the measurement results. Inspection items include, for example, the volume, area, height, positional deviation, and shape of the solder. An image sensor (camera) can be used for two-dimensional measurement of the solder paste, and a laser displacement meter, phase shift method, spatial coding method, light section method, etc. can be used for three-dimensional measurement.

The component inspection device Y2 is a device for inspecting the placement state of electronic components on the board carried out from the mounter X2. The component inspection device Y2 measures the components (which may be parts of components such as the component body or electrodes) placed on the solder paste in two or three dimensions, and judges whether or not various inspection items are normal (within the allowable range) based on the measurement results. The inspection items include, for example, component position deviation, angle (rotation) deviation, missing parts (no components placed), different components (different components placed), different polarity (different polarity of electrodes on the component side and the board side), front/back inversion (components placed upside down), component height, etc. As with the solder printing inspection, an image sensor (camera) or the like can be used for two-dimensional measurement of electronic components, and a laser displacement meter, a phase shift method, a spatial coding method, a light cutting method, etc. can be used for three-dimensional measurement.

Visual inspection device Y3 is a device for inspecting the quality of soldering on the board removed from the reflow furnace X3. Visual inspection device Y3 measures the soldered portion after reflow in two or three dimensions, and judges whether various inspection items are normal (within the acceptable range) based on the measurement results. Inspection items include the same items as in component inspection, as well as the quality of the solder fillet shape. In addition to the above-mentioned laser displacement meter, phase shift method, spatial coding method, and light section method, the so-called color highlight method (a method in which R, G, and B lighting are applied to the solder surface at different angles of incidence, and the reflected light of each color is photographed with a zenith camera to detect the three-dimensional shape of the solder as two-dimensional hue information) can be used to measure the solder shape.

Figure 2 is a block diagram showing the schematic configuration of the inspection processing functions of the appearance inspection device Y3. The inspection processing functions of the appearance inspection device Y3 are mainly realized by a camera 31, an information processing device 32, and a display device 33. The information processing device 32 is composed of a general-purpose computer system equipped with a CPU (processor), a main storage device (memory), an auxiliary storage device (hard disk, etc.), an input device (keyboard, mouse, controller, touch panel, etc.), etc.

The image acquisition unit 321 acquires an image of the substrate to be inspected taken by the camera 31. The inspection image generation unit 322 performs a predetermined process on the image acquired by the image acquisition unit 321 to generate an inspection image. The inspection unit 323 executes an inspection program to measure (calculate) a predetermined index based on the inspection image, and uses these measured values to inspect the state of the inspection object and judge whether it is good or bad. Here, the inspection program executed is a program including a learning device learned by supervised learning. The inspection program storage unit 324 stores the inspection program executed in the inspection unit 323. The result output unit 325 outputs the inspection result by the inspection unit 323 to the screen and displays it on the display device 33. The inspection result storage unit 326 associates and stores the inspection image and the inspection result. The communication interface 327 is an interface for communicating with the management device 10, the program management server 20, etc. connected via a network (LAN). Here, the inspection images and inspection results stored in the inspection result storage unit 326 include those for inspection images that were judged to be defective by the inspection program but were later judged to be good in the visual inspection process (over-detected inspection images).

The inspection images and inspection results stored in the inspection result storage unit 326 are transmitted to the management device 10 via the communication interface 327. The inspection program is re-learned by the program management server 20, and transmitted from the program management server 20 to the inspection program storage unit 324 via the communication interface 327 and stored therein. Here, an overview of the functions related to the inspection process for the appearance inspection device Y3 has been described, but other inspection devices have similar functional configurations.

The X-ray inspection device Y4 is a device for inspecting the state of soldering of a board using an X-ray image. For example, in the case of package parts such as BGA (Ball Grid Array) and CSP (Chip Size Package) or multi-layer boards, the solder joints are hidden under the parts or boards, so the appearance inspection device Y3 (i.e., the appearance image) cannot inspect the state of the solder. The X-ray inspection device Y4 is a device for complementing such weaknesses of appearance inspection. Inspection items of the X-ray inspection device Y4 include, for example, component positional deviation, solder height, solder volume, solder ball diameter, back fillet length, and the quality of the solder joint. Note that, as the X-ray image, an X-ray transmission image may be used, and it is also preferable to use a CT (Computed Tomography) image.

(Management Device)
The manufacturing devices X1 to X3 and the inspection devices Y1 to Y4 described above are connected to a management device 10 via a network (LAN). The management device 10 is a system that manages and controls the manufacturing devices X1 to X3 and the inspection devices Y1 to Y4, and is configured by a general-purpose computer system including a CPU (processor), a main storage device (memory), an auxiliary storage device (hard disk, etc.), an input device (keyboard, mouse, controller, touch panel, etc.), and a display device, although not shown. The functions of the management device 10 described below are realized by the CPU reading and executing a program stored in the auxiliary storage device.

The management device 10 may be configured as one computer or as multiple computers. Alternatively, all or part of the functions of the management device 10 may be implemented in a computer built into any of the manufacturing devices X1 to X3 or the inspection devices Y1 to Y4. Alternatively, part of the functions of the management device 10 may be realized by a server on a network (such as a cloud server).

A program management server 20 is connected to the management device 10 via a network (LAN). The program management server 20 is a server that manages the inspection program and the quality evaluation program. The program management server 20 is configured by a general-purpose computer system equipped with a CPU (processor), a main storage device (memory), an auxiliary storage device (hard disk, etc.), an input device (keyboard, mouse, controller, touch panel, etc.), a display device, etc. The inspection program is a program that realizes the inspection process in the inspection devices Y1 to Y4, and is stored in a predetermined storage area of the program management server 20, while being downloaded to each of the inspection devices Y1 to Y4 as necessary, stored in a predetermined storage area of each device, and executed in each device. The quality evaluation program is a program that realizes the quality evaluation process in the management device 10, and is stored in a predetermined storage area of the program management server 20, while being downloaded to the management device 10 as necessary, stored in the quality evaluation program storage unit 109, and executed in the management device. The program management server 20 may manage various programs that control the manufacturing devices X1 to X3 and the inspection devices Y1 to Y4, as well as the inspection program and the quality evaluation program.

The management device 10 of this embodiment has functional units for implementing functions that allow the manager of the manufacturing equipment to efficiently perform equipment maintenance and quality control. Figure 2 shows a block diagram of each functional unit related to quality evaluation possessed by the management device 10. Here, the management device 10 corresponds to the quality evaluation device of the present invention, and the management system 1 corresponds to the inspection management system of the present invention.

As shown in FIG. 3, the management device 10 has an inspection image/inspection result acquisition unit 101, an inspection image/inspection result database (DB), a quality evaluation unit 103, an evaluation result storage unit 104, a quality change determination unit 105, a quality evaluation information generation unit 106, a display unit 107, an input unit 108, and a quality evaluation program storage unit 109. Here, the quality evaluation unit 103, the evaluation result storage unit 104, and the quality change determination unit 105 correspond to the quality change evaluation unit of the present invention. Also, the quality evaluation unit 103, the evaluation result storage unit 104, and the quality change determination unit 105 correspond to the quality evaluation unit, quality evaluation storage unit, and change determination unit of the present invention, respectively. Also, the quality evaluation information generation unit 106 corresponds to the quality evaluation information generation unit of the present invention.

The inspection image and inspection result acquisition unit 101 acquires the inspection images and associated inspection results from the inspection devices Y1 to Y4. The inspection images and inspection results acquired by the inspection image and inspection result acquisition unit 101 are stored in an inspection image and inspection result DB. Here, the inspection images are also associated with information such as the inspection date and time and line-symmetrical product number, and this information is also acquired and stored.

The quality evaluation unit 103 has a function of evaluating the quality of the printed circuit board by the output obtained by inputting an inspection image to a trained learning device that has been trained by unsupervised learning, such as an autoencoder. Here, what is input to the quality evaluation unit 103 are inspection images of printed circuit boards that have been determined to be good products among the inspection images recorded in the inspection image/inspection result DB 102. Note that inspection images that have been determined to be good products also include inspection images of printed circuit boards that have been determined to be bad by the inspection program in the inspection device but that have been determined to be good products in a subsequent visual inspection process. The trained learning device used here outputs a one-dimensional numerical value, which is called the degree of abnormality. The inspection images used for evaluating the quality can be sampled as appropriate. For example, the results of sampling once an hour are accumulated for a predetermined period of time, and the degree of abnormality is calculated. Here, the degree of abnormality corresponds to the quality evaluation index of the present invention.

The degree of anomaly calculated by the quality evaluation unit 103 is stored in the evaluation result storage unit 104. Here, the evaluation result storage unit 104 corresponds to the quality evaluation index storage unit of the present invention.

The quality change determination unit 105 analyzes the change over time in the degree of abnormality accumulated in the evaluation result accumulation unit 104, and evaluates the change in the quality of the printed circuit board. Specifically, the change in the degree of abnormality is compared with a set threshold value to determine the degree of abnormality. As described above, the average value of the degrees of abnormality accumulated over a specified period is taken as the degree of abnormality for that period, and the standard deviation is taken as the variation in the degree of abnormality. Here, two threshold values are set: a threshold value (degree of abnormality) related to the value of the degree of abnormality, and a threshold value (variation) related to the variation in the degree of abnormality. The threshold value (degree of abnormality) and the threshold value (variation) are input in advance by the user via the input unit 108. The threshold value (degree of abnormality) and the threshold value (variation) correspond to the first threshold value and the second threshold value of the present invention, respectively.

The quality evaluation information generating unit 106 generates quality evaluation information based on the evaluation result by the quality change determining unit 105 and causes the display unit 107 to display the information.
4 shows an example of display of the quality evaluation information 71. Frames and leader lines indicating each piece of information in the quality evaluation information 71 are displayed with dotted lines to distinguish them from the elements of the quality evaluation information 71.

A control chart 711 is displayed in the center of the quality evaluation information 71. Here, a control chart for part number A is displayed. In the control chart 711, the horizontal axis represents the period during which the inspection images were acquired, and the vertical axis represents the degree of abnormality. The average value for the target period is plotted and shown as a dashed line for the degree of abnormality, and the solid vertical bar represents the standard deviation. In addition, the vertical axis displays the threshold value for the degree of abnormality as a solid line.

Below the control chart 711 is a threshold (degree of abnormality) display area 712, with the currently set value of "0.5" displayed. The threshold (degree of abnormality) display area 712 is an input field, and the threshold (degree of abnormality) can be changed by entering a numerical value and specifying the setting button 713 on the right side by clicking, etc. Below the threshold (degree of abnormality) display area 712 is a threshold (variation) display area 714, with the currently set value of "0.4" displayed. The threshold (variation) display area 712 is an input field, and the threshold (variation) can be changed by entering a numerical value and pressing the setting button 713 on the right side by clicking, etc.

As described above, the threshold (degree of abnormality) and the threshold (variation) may be input and set by the user, but a history of the threshold (degree of abnormality) and the threshold (variation) may be stored in a predetermined storage area, and appropriate thresholds (degree of abnormality) and thresholds (variation) may be set by the quality change determination unit 105 using machine learning or the like based on the history of the threshold (degree of abnormality) and threshold (variation) and the history of the evaluation results accumulated in the evaluation result accumulation unit 104, or a recommended value may be taught to the user. It is also possible for the user to input only one of the threshold (degree of abnormality) and the threshold (variation), and for the other to be automatically set or a recommended value to be taught. Here, the quality change determination unit 105 corresponds to the setting unit of the present invention.

An overall evaluation display area 716 is disposed above the control chart 711. A message regarding quality is displayed in the overall evaluation display area 716. This overall evaluation is based on the change in the degree of abnormality displayed in the control chart 711. In the control chart 711, the degree of abnormality exceeds the threshold (degree of abnormality), and the variation also exceeds the threshold (variation). For this reason, the overall evaluation display area 716 displays the message "There is variation and change in quality. We recommend that you review the process and retrain the AI model" as the overall evaluation of quality, informing the user that it is time to retrain the learning device of the inspection program used for the inspection.

In this way, changes in the quality of the test object are captured, and quality evaluation information 71, etc. are displayed on the display unit 107, and a message is displayed to the user encouraging re-learning of the test program's learner, thereby enabling highly reliable testing to be achieved.

On the left side of the quality evaluation information 71, a part number display area 717 is arranged. The inspection program generates an AI model for each part type, and one part type includes multiple part numbers. Here, it is shown that the QFP AI model used for the inspection includes those for part numbers A, B, and C. When an alarm for reviewing the process or relearning the AI model is issued, the part number display is displayed in a color other than white, such as red. Here, the display 717a of part number A is displayed in a different color. This display allows the user to clearly recognize the part number that needs to be addressed. In addition, each part number display 717a, etc. is a button, and by clicking or pressing these buttons, the display switches to the control chart of the corresponding part number. In addition, by clicking or pressing the display 717b of the QFP AI model, the entire control chart can be confirmed.

A representative data display area 718 is located to the right of the quality evaluation information 71. Here, representative data for each period of the inspection images sampled by the quality change determination unit 105 is displayed in chronological order. This allows the validity of the overall evaluation to be visually confirmed. By clicking or otherwise pressing the reselect button 719 located below the representative data display area 718, the representative data can be switched to another inspection image for the relevant period.

Figure 5 shows an example of the display of other quality evaluation information 72. Explanation of information common to quality evaluation information 71 will be omitted. Here, the control chart 721 and overall evaluation display area 716 are different from the quality evaluation information 71. In the control chart 721, the degree of abnormality does not exceed the threshold (degree of abnormality), but it does exceed the threshold (variation). For this reason, the overall evaluation display area 726 displays the message "There is variation in quality. We recommend that you review the process" as the overall quality evaluation, informing the user that the process should be reviewed.

FIG. 6 shows a display example of other quality evaluation information 73. Explanation of information common to the quality evaluation information 71 is omitted. Explanation of information common to the quality evaluation information 71 is omitted. Here, the control chart 731 and the overall evaluation display area 736 are different from the quality evaluation information 71. In the control chart 731, the degree of abnormality exceeds the threshold (degree of abnormality), but does not exceed the threshold (variation). Therefore, the overall evaluation display area 736 displays a message as the overall evaluation of quality, saying "There is a change in quality. We recommend retraining the AI model.", informing the user that it is time to retrain the learning device of the inspection program.

Figure 7 shows an example of the display of other quality evaluation information 74. Explanation of information common to quality evaluation information 71 will be omitted. Explanation of information common to quality evaluation information 71 will be omitted. Here, the control chart 731, overall evaluation display area 736, and product number display area 747 are different from the quality evaluation information 71. In control chart 741, the degree of abnormality does not exceed the threshold (degree of abnormality) or the threshold (variation). Therefore, the overall evaluation display area 746 displays the message "Quality is stable" as the overall evaluation of quality. In addition, the display 747a of product number A in the product number display area 747 is displayed in white, indicating that no alarm has been issued.

In this way, the management device 10 can instruct the user when to re-learn the learning device of the inspection program.

When the quality change determination unit 105 recommends re-learning the learning device of the inspection program and the user instructs re-learning of the learning device of the inspection program, the program management server 20, which manages the program, re-learns the inspection program. At this time, the program management server 20 also re-learns the learning device of the quality evaluation program. In this way, by re-learning the quality evaluation program along with the inspection program, inspection and quality evaluation that responds promptly to quality changes become possible.

Figure 8 illustrates a learning device 21 that learns an inspection program. This learning device 21 is configured in the program management server 20. The learning device 21 includes a learning inspection image acquisition unit 211, a learning inspection result acquisition unit 212, a learning data storage unit 213, a learning processing unit 214, and a learning device 215. In the case of re-learning, the learning data is the inspection images and inspection results from the most recent period up to the time of re-learning. Here, the learning device 215 corresponds to the first learning device of the present invention. The learning device 21 corresponds to the learning device of the present invention.

The learning test image acquisition unit 211 acquires the test image for re-learning from the test image/test result DB 102. The learning test result acquisition unit 212 acquires the test results of the test image from the test image/test result DB 102.

The learning test images and learning test results acquired via the learning test image acquisition unit 211 and the learning test result acquisition unit 212 are stored in the learning data storage unit 213, which is a specified area of the auxiliary storage device of the program management server 20.

The learning processing unit 214 performs machine learning of the learning device so that when the learning test image stored in the learning data storage unit 213 is input, the learning processing unit 214 outputs a test result. The learning processing unit 214 is realized by the CPU of the program management server 20 reading and executing a learning model generation program stored in a specified area of the auxiliary storage device. Here, the learning device is, for example, a program that calculates a model using a neural network with the learning test image as learning data and the learning test result as teacher data, but is not limited to this.

The machine learning of the learning device in the learning processing unit 214 is repeated for a large amount of learning data to obtain a trained learning device 215. The learning device 215 thus obtained is stored in the learning result data storage unit 216, which is a predetermined area of the auxiliary storage device. The trained learning device 215 stored in the learning result data storage unit 216 is transmitted to and stored in, for example, the inspection program storage unit 324 of the inspection machine Y3.

Figure 9 illustrates a learning device 22 that learns a quality evaluation program. This learning device 22 is configured in the program management server 20. The learning device 22 includes a learning-use good product image acquisition unit 221, a learning data storage unit 222, a learning processing unit 214, and a learning device 224. In the case of re-learning, the learning data is made up of good product images from the most recent period up until the time of re-learning. Here, the learning device 224 corresponds to the second learning device of the present invention.

The learning good product image acquisition unit 221 acquires inspection images that have been determined to be good products from the inspection image and inspection result DB 102.

The good-quality images for learning acquired via the good-quality image acquisition unit for learning 221 are stored in the learning data storage unit 222, which is a specified area of the auxiliary storage device of the program management server 20.

The learning processing unit 223 performs machine learning of the learning device so that it outputs the degree of abnormality when a learning image of a good product stored in the learning data storage unit 222 is input. The learning processing unit 223 is realized by the CPU of the program management server 20 reading and executing a learning model generation program stored in a specified area of the auxiliary storage device. Here, the learning device is an unsupervised AI model such as VAE that uses learning images of good products as learning data.

The machine learning of the learner in the learning processing unit 223 is repeated for a large amount of learning data to obtain a trained learner 224. The learner 224 obtained in this manner is stored in the learning result data storage unit 225, which is a specified area of the auxiliary storage device. The trained learner 224 stored in the learning result data storage unit 225 is transmitted to the quality evaluation program storage unit 109 and stored therein.

Example 2
A management system 2 according to a second embodiment of the present invention will be described below. Configurations common to the first embodiment will be designated by the same reference numerals and detailed description will be omitted.

The overall configuration of the management system 2 according to the second embodiment and the functional configurations of the inspection devices Y1 to Y4 and the learning device 21 are the same as those of the first embodiment. FIG. 10 shows a block diagram of each functional unit related to quality evaluation possessed by the management device 11 according to the second embodiment. In the second embodiment, the quality evaluation program is different from that of the first embodiment, and therefore the functions of the quality evaluation unit 203, evaluation result storage unit 204, quality change determination unit 205, and quality evaluation information generation unit 206 of the management device 11, the configuration of the quality evaluation information, and the configuration of the learning device 23 are different from those of the first embodiment. Here, the management device 11 corresponds to the quality evaluation device of the present invention, and the management system 2 corresponds to the inspection management system of the present invention.

Figure 11 shows a schematic functional configuration of a learning device 23 that performs machine learning of a quality evaluation program according to Example 2. The learning device 23 is configured in the program management server 20. The learning device 23 includes a learning-use good-product image acquisition unit 231, a learning data storage unit 232, a learning processing unit 233, and a learning device 234. In the case of re-learning, the subject of the learning data is good-product images from the most recent period up until the time of re-learning. Here, the learning device 234 corresponds to the third learning device of the present invention.

The learning good product image acquisition unit 231 acquires inspection images that have been determined to be good products from the inspection image and inspection result DB 102.

The good-quality images for learning acquired via the good-quality image acquisition unit for learning 231 are stored in the learning data storage unit 232, which is a specified area of the auxiliary storage device of the program management server 20.

The learning processing unit 233 performs machine learning of the learning device by inputting the learning images of good products stored in the learning data storage unit 232, mapping the good product images onto a two-dimensional plane by dimensionally compressing the features of the good product images, and clustering each mapped good product image. The learning processing unit 223 is realized by the CPU of the program management server 20 reading and executing a learning model generation program stored in a specified area of the auxiliary storage device. Here, the learning device is an unsupervised AI model such as principal component analysis (PCA) that uses the learning images of good products as learning data.

The machine learning of the learner in the learning processing unit 233 is repeated for a large amount of learning data to obtain a trained learner 234. The learner 234 obtained in this manner is stored in the learning result data storage unit 235, which is a predetermined area of the auxiliary storage device. The trained learner 234 stored in the learning result data storage unit 235 is transmitted to the quality evaluation program storage unit 109 and stored therein. Figure 12 (A) shows an example of the output of the trained learner 234. Points L1, L2, etc., which represent trained images of good products, are mapped onto a two-dimensional surface S, and are clustered by a circle C that surrounds these points L1, etc.

In the quality evaluation unit 203 of the second embodiment, the quality evaluation program generated in this manner maps the good product images of the mass-produced data obtained from the inspection images and inspection results. FIG. 12(B) shows a two-dimensional map in which good product images, which are mass-produced data, are input to the trained learner 234 and mapped onto a two-dimensional surface S. Here, good product images, which are training data, are displayed as black points, and good product images, which are mass-produced data, are displayed as gray points. In the state shown in FIG. 12(B), points P1, P2, etc., which indicate good product images, which are mass-produced data, are mapped onto the two-dimensional surface S, but points P1, etc., which indicate good product images, which are mass-produced data, are all included in the cluster indicated by circle C1.

In the quality evaluation unit 203 of the second embodiment, as described above, the non-defective product images, which are mass-produced data obtained from the inspection image/inspection result DB, are sampled and mapped, and a two-dimensional map is output. The two-dimensional map is stored in the evaluation result storage unit 204.

In the quality change determination unit 205 of the second embodiment, the number of points indicating good quality images that are outliers outside of the clusters is counted in the two-dimensional map generated by the quality evaluation unit 203. As shown in FIG. 11B, when the mass-produced data, which are good quality images, are sequentially mapped, if the quality changes, data that is mapped to a position outside of cluster C as an outlier appears. In FIG. 12C, point O1 indicating a good quality image that is mass-produced data is mapped to a position outside of cluster C that includes point L1 indicating a good quality image for learning, which is mass-produced data. The quality change determination unit 205 counts the number of such outliers, compares the number of outliers with a threshold value, and determines whether the threshold value is exceeded.

In the second embodiment as well, the quality evaluation information generating unit 206 generates quality evaluation information based on the evaluation result by the quality change determining unit 205 and causes the display unit 107 to display the information.
13 shows an example of display of quality evaluation information 75 according to Example 2. Frames and leader lines indicating each piece of information in the quality evaluation information 75 are displayed with dotted lines to distinguish them from the elements of the quality evaluation information 75.

A 2D map 751 is displayed on the right side of the center of the quality evaluation information 75. The 2D map 751 also displays the product number (here, "product number A") that is the subject of the quality evaluation and the date (evaluation date) indicating when the evaluation data is obtained. The user can appropriately specify the evaluation date of the displayed quality evaluation information 75. As described above, the 2D map is generated by inputting a non-defective image, which is mass-produced data, into the quality evaluation program and mapping the image on a 2D map generated from a non-defective image for learning. Cluster C of the 2D map 751 includes points L1 and the like indicating a non-defective image for learning, and points P1 and the like indicating a non-defective image, which is mass-produced data. In addition, four points O1, O2, O3, and O4 indicating non-defective images, which are mass-produced data, are arranged outside of cluster C as outliers in the 2D map 751. When the cursor is placed on point O4 indicating a non-defective image, which is an outlier, a pop-up is displayed with non-defective image data corresponding to the point and the inspection date on which the non-defective image Im4 was inspected.

Below the 2D map 751 of the quality evaluation information 75, a threshold (number of alarms) display area 752 is arranged, with the currently set value of "4" displayed. The threshold (number of alarms) display area 752 is an input field, and the threshold (number of alarms) can be changed by inputting a numerical value and specifying it by clicking or otherwise selecting the setting button 753 on the right side. This threshold (number of alarms) is intended to issue an alarm recommending the user to review the process or retrain the learning device of the inspection program when the number of outliers in the mass production data arranged on the 2D map exceeds the threshold (number of alarms). Here, the threshold (number of alarms) corresponds to the third threshold of the present invention.

As described above, the threshold (number of alarms) may be input and set by the user, but the history of the threshold (degree of abnormality) and threshold (number of alarms) may be stored in a specified storage area, and an appropriate threshold (number of alarms) may be set by the quality change determination unit 205 using machine learning or the like based on the history of the threshold (number of alarms) and the history of the evaluation results stored in the evaluation result storage unit 204, or a recommended value may be instructed to the user. Here, the quality change determination unit 205 corresponds to the threshold setting unit of the present invention.

Above the quality evaluation information 75, an overall evaluation display area 754 is arranged. A message regarding quality is displayed in the overall evaluation display area 754. This overall evaluation is based on the number of outliers in the 2D map 751. As displayed in the threshold (number of alarms) display area 752, the current threshold (number of alarms) is 4. In contrast, four points O1, O2, O3, and O4 are mapped as outliers in the 2D map 751, so the number of outliers is equal to the threshold (number of alarms). Therefore, the quality change determination unit 105 evaluates that the quality has changed, and a message saying "The quality has changed. We recommend that you review the process or retrain the AI model" is displayed in the overall evaluation display area 754, informing the user that they should review the process or retrain the learning device of the inspection program used in the inspection.

In this way, changes in the quality of the test object are captured, quality evaluation information 75 is displayed on the display unit 107, and a message is displayed to the user encouraging re-learning of the test program's learner, thereby enabling highly reliable testing to be achieved.

On the left side of the quality evaluation information 75, a part number display area 755 is arranged. The inspection program generates an AI model for each part type, and one part type includes multiple part numbers. Here, it is shown that the QFP AI model used for the inspection includes those for part numbers A, B, and C. When an alarm for reviewing the process or relearning the AI model is issued, the part number display is displayed in a color different from white, such as red. Here, the display 755a of part number A is displayed in a different color. This display allows the user to clearly recognize the part number that needs to be addressed. In addition, each part number display 755a, etc. is a button, and by clicking or pressing these buttons, the display switches to a 2D map of the corresponding part number. In addition, by clicking or pressing the display 755b of the QFP AI model, the entire 2D map can be confirmed.

<Appendix 1>
a good-quality image acquisition unit (101) for acquiring a good-quality image, which is an inspection image determined to be a good-quality image in an inspection process using a trained first learning device (215) generated by machine learning using an inspection image generated by capturing an image of an inspection object as learning data and an inspection result as teacher data;
A quality change evaluation unit (103, 104, 105) for evaluating changes in the quality of the inspection object;
a quality evaluation information generating unit (106) for generating quality evaluation information including a result of the evaluation;
A quality evaluation device (10) comprising:

10, 11: Management device 103: Quality evaluation unit 104: Evaluation result storage unit 105: Quality change evaluation unit 106: Quality evaluation information generation unit 215: Learning device

Claims (10)

a good-quality image acquiring unit that acquires a good-quality image, which is an inspection image determined to be a good-quality image in an inspection process using a trained first learning device generated by machine learning the inspection results as teacher data, using an inspection image generated by imaging an inspection object as learning data;
a quality change evaluation unit for evaluating a change in the quality of the inspection object;
a quality evaluation information generating unit that generates quality evaluation information including a result of the evaluation;
Equipped with
The quality change evaluation unit
a quality evaluation unit that outputs a quality evaluation index for evaluating the quality based on the non-defective product image;
a quality evaluation index storage unit that stores the quality evaluation index;
a change determination unit that determines a change in the quality based on a change over a predetermined period of time of the quality evaluation index stored in the quality evaluation index storage unit;
A quality evaluation device comprising : the quality evaluation index is an anomaly degree output by a trained second learning device generated by machine learning through unsupervised learning using the non-defective product image as training data;
The quality evaluation device according to claim 1, wherein the change determination unit calculates an average value and a standard deviation of the degree of anomaly over a predetermined period of time, and compares the average value and the standard deviation with a first threshold value and a second threshold value, respectively, to determine the change in the quality. 3. The quality evaluation device according to claim 2 , further comprising a setting unit that automatically sets at least one of the first threshold value and the second threshold value. a good-quality image acquiring unit that acquires a good-quality image, which is an inspection image determined to be a good-quality image in an inspection process using a trained first learning device generated by machine learning the inspection results as teacher data, using an inspection image generated by imaging an inspection object as learning data;
a quality change evaluation unit for evaluating a change in the quality of the inspection object;
a quality evaluation information generating unit that generates quality evaluation information including a result of the evaluation;
Equipped with
The quality change evaluation unit
A quality evaluation device characterized by performing clustering using a trained third learning device generated by unsupervised machine learning using the good product images as learning data, and evaluating the change in quality by comparing the number of outliers with a third threshold. 5. The quality evaluation device according to claim 4 , further comprising a threshold setting unit that automatically sets the third threshold. The quality evaluation device according to claim 1 , wherein the quality evaluation information includes information recommending relearning of the first learning device. 7. The quality evaluation device according to claim 1 , wherein the quality evaluation information includes information recommending an improvement of a pre-process related to the inspection target. 8. The quality evaluation device according to claim 1 , further comprising a display unit for displaying the quality evaluation information. A quality evaluation device according to any one of claims 1 to 8 ;
An inspection management system including an inspection device having an inspection processing unit that performs inspection processing on the inspection object using the trained first learning device, and a memory unit that stores the inspection image provided to the inspection processing and the inspection result that is the result of the inspection processing. The inspection management system according to claim 9 , further comprising a learning device for re-training the first learning device.

Images