JP7726673B2 - Judgment method - Google Patents

Description

This disclosure relates to technology for determining whether work in production equipment is satisfactory.

Patent Document 1 discloses a state determination device that determines the operating state of an injection molding machine. This conventional technology collects a wide variety of equipment data as training data and uses this data to train a machine learning model.

Japanese Patent Application Laid-Open No. 2020-44718

However, depending on the equipment, abnormal values may occur infrequently, and in such cases, collecting training data may take a significant amount of time. On the other hand, when performing pass/fail judgment using an image-based machine learning model, it is easy to collect training data, but when the workpiece is changed, the same machine learning model cannot be used; instead, a machine learning model must be created and trained for each type of workpiece. Therefore, there is still room for consideration in finding a way to create a machine learning model that can be applied to different types of workpieces. Note that this issue is not limited to injection molding machines, but is also common to judgment of the pass/fail of work in other production equipment.

The present disclosure can be realized in the following forms.
The judgment method disclosed herein is a judgment method for judging the quality of work performed on workpieces in production equipment, and includes the steps of: (a) using images of the work results of the production equipment as training data to train a first machine learning model that judges the quality of the work results; (b) using work status information of the production equipment when the trained first machine learning model judges the work to be quality as training data labeled with a quality label, and using work status information of the production equipment when the first machine learning model judges the work to be quality as training data labeled with a quality label to train a second machine learning model that inputs the work status information and outputs the quality of the work results; and (c) inputting work status information of the production equipment for work performed on new workpieces having the same specifications as the workpieces into the trained second machine learning model to judge the quality of the work performed on the new workpieces.

(1) According to a first aspect of the present disclosure, there is provided a judgment system for judging the quality of work performed on workpieces in a production facility, the judgment system including: a detection unit that detects work status information of the production facility; a first machine learning model that judges the quality of the work results using an image of the work results of the production facility as input; and a second machine learning model that is trained using the work status information of the production facility when the first machine learning model judges the work results to be quality as teacher data labeled with quality, and the work status information of the production facility when the first machine learning model judges the work results to be quality as teacher data labeled with quality.
According to this judgment system, the quality of the work results of the production equipment is judged using a first machine learning model trained using images for which training data is relatively easy to collect, and the work status information at that time is used as training data to train a second machine learning model, so it is easy to build a judgment system that judges the quality of work from the work status information of the production equipment.In addition, the second machine learning model can be applied to different types of work.
(2) In the above-described determination system, the production equipment may be a coating equipment equipped with an injection unit that injects a coating liquid onto the workpiece, and the work status information may include the injection pressure of the injection unit and driving information of a driving unit that changes the relative position between the workpiece and the injection unit.
According to this determination system, it is possible to easily construct a determination system that determines whether work is good or bad based on work status information of the coating equipment.
(3) According to a second aspect of the present disclosure, there is provided a method for determining whether work performed on a workpiece in a production facility is passable or failable, the method including: (a) using images of work results of the production facility as training data to train a first machine learning model that determines whether the work results are passable or failable; (b) using work status information of the production facility when the trained first machine learning model determines the work is passable as training data labeled with a pass label and work status information of the production facility when the first machine learning model determines the work is failable as training data labeled with a fail label to train a second machine learning model; and (c) inputting work status information of the production facility for work performed on a new workpiece having the same specifications as the workpiece into the trained second machine learning model to determine whether the work on the new workpiece is passable or failable.
According to this determination method, the quality of the work results of the production equipment is determined using a first machine learning model trained using images for which training data is relatively easy to collect, and the work status information at that time is used as training data to train a second machine learning model, so that the quality of the work can be easily determined from the work status information of the production equipment. In addition, the second machine learning model can be applied to different types of work.
(4) The above judgment method may further include (d) a step of inputting work status information of the production equipment when working on other work having specifications different from the work into the trained second machine learning model, and performing a judgment on the quality of the work on the other work.
According to this determination method, it is possible to easily determine whether work performed on other workpieces with different specifications is good or bad.

FIG. 2 is a functional block diagram of a determination system. FIG. 1 is an explanatory diagram showing a configuration example of a coating facility. 1 is a flowchart showing a processing procedure according to an embodiment.

Figure 1 is a functional block diagram of a determination system according to one embodiment of the present disclosure. This determination system includes an image-based automatic determination device 100, a production status information-based machine learning device 200, and coating equipment 300 as production equipment. The automatic determination device 100, the machine learning device 200, and the coating equipment 300 can send and receive various data via their respective communication units 110, 210, and 310. Equipment other than the coating equipment 300 can also be used as production equipment.

The image-based automatic judgment device 100 includes a communication unit 110, a control unit 120, first teacher data 130, a learning execution unit 140, a first machine learning model 150, a judgment execution unit 160, and an imaging device 170. The first teacher data 130 and the first machine learning model 150 are stored in a storage device such as a hard disk. The control unit 120 includes a processor and memory, and controls the automatic judgment device 100. The learning execution unit 140 uses the first teacher data 130 to train the first machine learning model 150. The judgment execution unit 160 uses the trained first machine learning model 150 to input images of workpieces that have been coated by the coating equipment 300 and judges whether the workpieces are good or bad. The functions of the learning execution unit 140 and the judgment execution unit 160 may be implemented by the processor of the control unit 120 executing a computer program, or may be implemented by hardware circuits.

The first training data 130 is image-based training data. That is, the first training data 130 is data in which an operator photographs a workpiece coated by the coating equipment 300 using the imaging device 170 and labels the image as good or bad. The first machine learning model 150 is trained using the first training data 130. Therefore, the first machine learning model 150 is a model that uses an image of the work results of the coating equipment 300 as input and judges whether the work results are good or bad. The judgment execution unit 160 uses an image of a new workpiece as input data for the trained first machine learning model 150 and judges whether it is good or bad. This input data is an image of the workpiece after coating, photographed by the imaging device 170.

The production status information-based machine learning device 200 includes a communication unit 210, a control unit 220, second teacher data 230, a learning execution unit 240, and a second machine learning model 250. The second teacher data 230 and the second machine learning model 250 are stored on a storage device such as a hard disk. The control unit 220 includes a processor and memory, and controls the machine learning device 200. The learning execution unit 240 executes learning of the second machine learning model 250 using the second teacher data 230. The functions of the learning execution unit 240 may be implemented by the processor of the control unit 220 executing a computer program, or may be implemented by a hardware circuit.

The second training data 230 is training data based on production status information. That is, the second training data 230 is training data in which the work status information of the coating equipment 300 when the first machine learning model 150 judges it to be good in the above-mentioned automatic judgment device 100 is labeled as good, and the work status information of the coating equipment 300 when the first machine learning model 150 judges it to be bad is labeled as bad. The second machine learning model 250 is trained using the second training data 230. Therefore, the second machine learning model 250 is a model that uses the work status information of the coating equipment 300 as input and judges whether the work results are good or bad. The trained second machine learning model 250 is transferred from the machine learning device 200 to the coating equipment 300 via the communication unit 210 and stored in a storage device.

The coating equipment 300 includes a communication unit 310, a control unit 320, a trained second machine learning model 330, a judgment execution unit 340, a work status detection unit 350, a drive unit 360, and a coating liquid injection unit 370. The second machine learning model 330 is transferred to the coating equipment 300 and stored in a storage device after learning in the machine learning device 200 is completed. The work status detection unit 350 detects work status information indicating the work status of the coating equipment 300. The judgment execution unit 340 inputs the work status information detected by the work status detection unit 350 into the second machine learning model 330 and judges whether the work is good or bad. The drive unit 360 moves the members and workpieces of the coating equipment 300. The coating liquid injection unit 370 injects the coating liquid onto the workpiece.

Figure 2 is an explanatory diagram showing an example configuration of a coating equipment 300. This coating equipment 300 includes a base 10, a support mechanism 20 having a support portion 22 that supports the workpiece WK, a motor 30 that moves the support mechanism 20, and an injection nozzle 40 that injects the coating liquid onto the surface of the workpiece WK. The support mechanism 20 and motor 30 are installed on the base 10. Figure 2 shows the X-axis and Y-axis, which indicate the horizontal direction, and the Z-axis, which indicates the vertical upward direction. A drive unit 360 including the motor 30 changes the relative position between the workpiece WK and the injection nozzle 40 at least in the horizontal direction, thereby positioning the tip of the injection nozzle 40 in a predetermined coating area on the surface of the workpiece WK. If the workpiece WK is curved, the drive unit 360 including the motor 30 moves the relative position between the workpiece WK and the injection nozzle 40 in three dimensions. By injecting the coating liquid from the injection nozzle 40 while moving the workpiece WK, the coating liquid can be applied to the entire coating area of the workpiece WK. Note that instead of using such a motor 30, an articulated robot may be used to position the injection nozzle 40 at a desired position on the workpiece WK. In this embodiment, the workpiece WK is a vehicle glass, and the coating liquid is urethane, a glass adhesive. Note that the motor 30 constitutes part of the drive unit 360 in FIG. 1, and the injection nozzle 40 constitutes part of the coating liquid injection unit 370 in FIG. 1.

The work status detection unit 350 of the coating equipment 300 in FIG. 1 detects work status information indicating the status of the drive unit 360 and the coating liquid injection unit 370 during the coating operation. Information indicating the status of the drive unit 360 is detected, for example, as drive information such as the current value of the motor 30. Information indicating the status of the coating liquid injection unit 370 is detected, for example, as injection pressure at the injection nozzle 40. The work status detection unit 350 may also detect the distance between the tip of the injection nozzle 40 and the surface of the workpiece WK as work status information. It is preferable that the work status detection unit 350 be configured to include sensors for detecting these various pieces of work status information.

Urethane is commonly used as an adhesive for glass attached to vehicle bodies. When applying urethane to glass, the introduction of air or foreign matter can cause the urethane to break or chip. When using a machine learning model to determine the quality of such work, two methods are considered: One is to use an image of the glass after urethane application as input to the machine learning model, and the other is to use work status information from the application equipment during the application process as input to the machine learning model. Of these, the method using work status information as input is considered more versatile when producing multiple vehicle models because it is not affected by changes in the shape of the glass. However, due to factors such as the low frequency of abnormalities, it is difficult to train a machine learning model using work status information as training data. Therefore, in this embodiment, as described below, a first image-based machine learning model 150 based on visual inspection is used to determine the quality of the work results, and the second training data 230 is created by labeling the work status information with a quality label using the results. The second training data 230 is then used to train the second machine learning model 250. By using the trained second machine learning model, it is possible to build a highly versatile judgment system that can determine the quality of application work based on work status information.

Various types of models that can make pass/fail judgments can be used as the first machine learning model 150 and the second machine learning model 250, such as models that use neural networks or models that use decision trees.

The division of functions between the automatic determination device 100, machine learning device 200, and coating equipment 300 shown in Figure 1 can be changed as desired. For example, the functions of the machine learning device 200 may be implemented in the automatic determination device 100 or the coating equipment 300. Alternatively, the functions of all three devices 100, 200, and 300 may be implemented in the coating equipment 300.

Figure 3 is a flowchart showing the processing steps of an embodiment. In step S110, the learning execution unit 140 of the automatic judgment device 100 uses image-based first training data 130 to train the first machine learning model 150 for coating work on a workpiece WK with specific specifications. For example, the windshield of a specific vehicle model is used as the workpiece WK with specific specifications. In this case, it is preferable to specify a vehicle model with a high production volume and for which a large amount of first training data 130 can be prepared as the specific vehicle model. The first training data 130 is an image of the workpiece WK coated with urethane, labeled with a pass/fail judgment result from a visual inspection. It is preferable to capture images at an angle of view that captures the urethane defects to be detected. If the number of images containing abnormalities is small, it is preferable to create several defective products that can serve as a standard for determining defects in visual inspection by humans and photograph these as well. Once the required amount of first training data 130 for machine learning has been collected, the first machine learning model 150 is trained. After training, the first machine learning model 150 preferably has improved judgment accuracy to the point where it can detect a variety of defects while still meeting the standards for visual inspection by humans.

In step S120, work status information for the coating equipment 300 during coating operations for a large number of workpieces WK is detected and recorded. The coating results are then evaluated for quality using the trained first machine learning model 150, and the evaluation results are recorded. First, work status information detected by the work status detection unit 350 during coating operations for each workpiece WK is recorded. As described above, the work status information preferably includes drive information such as the current value of the drive unit 360 and the injection pressure of the coating liquid injection unit 370. In step S120, an image of the urethane-coated workpiece WK is captured using the imaging device 170, and the image is input into the first machine learning model 150 for quality evaluation, and the evaluation results are recorded. This evaluation result is linked to work status information for the same workpiece WK. To perform this association, it is preferable to record the quality evaluation results from the first machine learning model 150 and the work status information together with information such as the detailed date and time and the serial number of the vehicle being produced.

In step S130, the second training data 230 is created by assigning a label to the work status information obtained in step S120 according to the pass/fail judgment result by the first machine learning model 150. For example, a "1" is assigned as a good label to work status information related to work that is judged to be a good product, and a "0" is assigned as a bad label to work status information related to work that is judged to be a defective product.

In step S140, the learning execution unit 240 of the machine learning device 200 uses the second training data 230 to learn the second machine learning model 250. The learned second machine learning model 250 is transferred from the machine learning device 200 to the coating equipment 300 and stored in a storage device as the second machine learning model 330.

In step S150, the judgment execution unit 340 of the coating equipment 300 applies the judgment made by the trained second machine learning model 330 to the coating work on a workpiece WK with the same specifications as the workpiece used in steps S110 and S120. Because the training of the second machine learning model 250 is performed using the work results of a large number of workpieces WK with the same specifications, it is possible to make a highly accurate judgment on the pass/fail of the coating work on a new workpiece WK with the same specifications.

In step S160, the judgment execution unit 340 of the coating equipment 300 applies the judgment made by the trained second machine learning model 330 to the coating work of a workpiece having specifications different from those of the workpiece that was the subject of judgment in step S150. The pass/fail judgment using the second machine learning model 330 is a highly versatile judgment that is not affected by differences in the appearance of the workpiece, so by applying this judgment to workpieces with different specifications, it is possible to prevent defective products from being released from the coating equipment 300. However, it is preferable that the "workpieces with different specifications" be parts of the same type as the workpiece that was the subject of judgment in step S150, such as glass.

As described above, in the above-described embodiment, the quality of the work results of the coating equipment 300 is determined using the first machine learning model 150, which has been trained using images for which training data is relatively easy to collect, and the second machine learning model 250 is trained using the work status information at that time as the second training data 230, so that the quality of the work can be easily determined from the work status information of the coating equipment 300. Furthermore, the trained second machine learning model 330 can also be applied to workpieces with different specifications.

This disclosure is not limited to the above-described embodiments, examples, and variations, and can be realized in various configurations without departing from the spirit of the disclosure. For example, the technical features in the embodiments, examples, and variations corresponding to the technical features in each form described in the Summary of Disclosure section can be replaced or combined as appropriate to solve some or all of the above-described problems or achieve some or all of the above-described effects. Furthermore, if a technical feature is not described as essential in this specification, it can be deleted as appropriate.

10... Stand, 20... Support mechanism, 22... Support unit, 30... Motor, 40... Injection nozzle, 100... Automatic judgment device, 110... Communication unit, 120... Control unit, 130... First teacher data, 140... Learning execution unit, 150... First machine learning model, 160... Judgment execution unit, 170... Imaging device, 200... Machine learning device, 210... Communication unit, 220... Control unit, 230... Second teacher data, 240... Learning execution unit, 250... Second machine learning model, 300... Coating equipment, 310... Communication unit, 320... Control unit, 330... Second machine learning model, 340... Judgment execution unit, 350... Work status detection unit, 360... Drive unit, 370... Coating liquid injection unit

Claims (1)

A method for determining whether work performed on a workpiece in a production facility is acceptable or not, comprising:
(a) using images of work results of the production equipment as training data to perform learning of a first machine learning model that determines whether the work results are good or bad;
(b) using the work status information of the production equipment when the trained first machine learning model judges it to be good as teacher data labeled as good, and the work status information of the production equipment when the trained first machine learning model judges it to be bad as teacher data labeled as bad, to train a second machine learning model that uses the work status information as input and outputs the pass/fail of the work result;
(c) inputting work status information of the production equipment in work on a new workpiece having the same specifications as the workpiece into the trained second machine learning model, and performing a quality judgment on the work on the new workpiece;
A determination method including: