JP7628899B2 - Method for generating trained model, learning device, welding robot control device, system, and program - Google Patents

Description

The present invention relates to a method for generating a trained model, a learning device, a control device for a welding robot, a system, and a program.

In recent years, in the field of welding work, automation technology has been developed that automatically creates or adjusts optimal welding conditions. As a method for creating or adjusting parameters as welding conditions without human intervention, a trained model is used that is generated by performing machine learning processing using a large amount of training data. On the other hand, in order to improve the accuracy of the output obtained from the trained model, the effort and cost for preparing training data for machine learning processing will inevitably increase.

For example, Patent Document 1 discloses an automatic welding system that uses a machine learning model to output welding state-related information and controls a robot based on the output results. Patent Document 2 discloses a method for reducing the cost of collecting training data for additional learning in machine learning.

JP 2018-192524 A JP 2020-135494 A

When the target of feature extraction includes at least one of fluids and plasma gas, such as in a welding work site, these often undergo abrupt changes due to disturbances such as the welding work conditions. Therefore, highly accurate feature extraction that can handle abrupt changes is required. In addition to welding, fluids and plasma gases are also used in the field of additive manufacturing. In this way, when welding work is assumed that requires more accurate feature extraction, more training data is required, and the quality of the training data, that is, the accuracy suitable for generating a more accurate trained model, is required. Cited Document 2 focuses on facilitating the additional work of extracting features of human figures, and does not take into account the accuracy of the data extracted by the trained model. For the accuracy of the training data, manual work is also considered, but this requires a lot of effort and cost.

In view of the above problems, the present invention aims to make it possible to extract learning data in consideration of the accuracy of the learning data used in machine learning processing assumed in the welding field. In particular, the present invention aims to provide a method for generating a trained model, a learning device, a control device, a system, and a program.

In order to solve the above problems, the present invention has the following configuration. That is, a method for generating a trained model,
The method includes a generation process for generating a first trained model by performing machine learning using a combination of data obtained during welding and feature information identified from the data as input and a classification of the combination as correct or incorrect as output.

According to another aspect of the present invention, there is provided a learning device, comprising:
A first generation means for generating a first trained model by performing machine learning using a combination of data obtained during welding and feature information identified from the data as an input and a classification of whether the combination is correct or incorrect as an output;
A second generation means for generating a second trained model by performing machine learning using a combination classified as positive in the first trained model as training data, with data obtained during welding as input, and feature information corresponding to the data as output;
has.

Another aspect of the present invention has the following configuration: A control device having a first trained model and a second trained model generated by a learning device,
An acquisition means for acquiring data related to welding;
A control means for outputting control information for a welding robot or a welding power source;
having
The control means
Feature information output by inputting the data acquired by the acquisition means into the second trained model, or
A combination of the data acquired by the acquisition means and feature information output by inputting the data into the second trained model is input into the first trained model, and the control information is output based on any of the feature information of the combination classified as positive.

Another aspect of the present invention has the following configuration. That is, the system includes:
A control device;
A welding robot
A detection means for detecting welding by the welding robot;
A welding power source;
having
The control device includes:
A first trained model and a second trained model generated by a learning device,
An acquisition means for acquiring data related to welding;
a control means for outputting control information for the welding robot or the welding power source;
having
The control means
Feature information output by inputting the data acquired by the acquisition means into the second trained model, or
inputting a combination of the data acquired by the acquisition means and feature information output by inputting the data into the second trained model into the first trained model, and obtaining feature information of the combination classified as positive;
The control information is output based on any one of the above.

In addition, the present invention has the following configuration as another aspect thereof: A trained model is generated by machine learning using a combination of data obtained during welding and feature information identified from the data as input, and a classification of the combination as correct or incorrect as output,
On the computer,
When a combination of data obtained during welding and feature information specified from the data is input, the device is caused to function to output a classification of whether the combination is correct or incorrect.

In addition, another aspect of the present invention has the following configuration. That is, a program comprising:
Computer,
a first generation means for generating a first trained model by performing machine learning using a combination of data obtained during welding and feature information identified from the data as an input and a classification of whether the combination is correct or incorrect as an output;
a second generation means for performing machine learning using the combination classified as positive in the first trained model as training data, with data obtained during welding as input, and feature information corresponding to the data as output, to generate a second trained model;
Function as.

According to the present invention, it is possible to generate a trained model capable of extracting training data, taking into account the accuracy of training data used in machine learning processing intended for the welding field.

1 is a schematic diagram showing an example of the configuration of a welding system according to an embodiment of the present invention; FIG. 2 is a perspective view for explaining an arrangement position of a visual sensor according to an embodiment of the present invention. 1 is a schematic diagram showing an example of the configuration of a robot control device according to an embodiment of the present invention; 1 is a schematic diagram showing an example of the configuration of a data processing device according to an embodiment of the present invention; A table for explaining various trained models according to one embodiment of the present invention. An explanatory diagram for conceptually explaining the generation process of trained models A and C according to one embodiment of the present invention. FIG. 4 is a schematic diagram for explaining an example of a screen used for a teaching operation according to an embodiment of the present invention. FIG. 2 is an explanatory diagram showing an example of an image that can be used in a learning process according to an embodiment of the present invention. FIG. 2 is an explanatory diagram showing an example of an image that can be used in a learning process according to an embodiment of the present invention. FIG. 2 is an explanatory diagram for explaining a process flow according to an embodiment of the present invention. 4 is a flowchart of the processing of a robot control device and a power supply device according to an embodiment of the present invention. 5 is a flowchart showing a process for generating correction information in the data processing device according to the embodiment of the present invention. FIG. 11 is a graph showing a comparison of results of a learning process according to an embodiment of the present invention. FIG. 11 is a graph showing a comparison of results of a learning process according to an embodiment of the present invention. A table for explaining the accuracy of trained model B according to one embodiment of the present invention.

The following describes the embodiment of the present invention with reference to the drawings. Note that the embodiment described below is one embodiment for explaining the present invention, and is not intended to be interpreted as limiting the present invention, and not all of the configurations described in each embodiment are necessarily essential configurations for solving the problems of the present invention. In addition, in each drawing, the same components are given the same reference numbers to indicate their correspondence.

In addition, the method of measuring welding behavior according to the present invention is useful not only for welding, but also for additive manufacturing technology using GMAW, specifically, metal additive manufacturing technology (WAAM: Wire and Arc Additive Manufacturing). The term additive manufacturing is sometimes used broadly as the term additive manufacturing or rapid prototyping, but in the present invention, the term additive manufacturing is used uniformly. When the method according to the present invention is used for additive manufacturing technology, "welding" can be replaced with "deposition," "additive manufacturing," or "additive manufacturing." For example, when treated as welding, it becomes "welding behavior," but when the present invention is used as additive manufacturing, it can be replaced with "deposition behavior," or when treated as welding, it becomes "welding system," but when the present invention is used as additive manufacturing, it can be replaced with "additive manufacturing system."

In the following description, "learning" or "machine learning" refers to generating a "trained model" by performing learning using training data and an arbitrary learning algorithm. The trained model is updated from time to time as learning progresses using multiple training data, and its output changes even for the same input. Therefore, the trained model is not limited to a state at any point in time. Here, a model used in learning is described as a "learning model", and a learning model that has undergone a certain degree of learning is described as a "trained model". In addition, specific examples of "learning data" will be described later, but the configuration may be changed depending on the learning algorithm used. In addition, the training data may include teacher data used for the learning itself, verification data used to verify the trained model, and test data used to test the trained model. In the following description, when data related to learning is collectively referred to, it is described as "learning data", and when data when performing learning itself is referred to, it is described as "teacher data". Note that it is not intended to clearly classify the teacher data, verification data, and test data contained in the training data, and for example, depending on the learning, verification, and testing methods, all of the training data may also be teacher data.

First Embodiment
Hereinafter, an embodiment of the present invention will be described with reference to the drawings.

[Welding system configuration]
Fig. 1 shows an example of the configuration of a welding system 1 according to this embodiment. The welding system 1 shown in Fig. 1 includes a welding robot 10, a robot control device 20, a power supply device 30, a visual sensor 40, and a data processing device 50. When the method according to the present invention is applied to additive manufacturing, for example, the welding system 1 may be read as an additive manufacturing system, and the welding robot 10 may be read as an additive manufacturing robot.

The welding robot 10 shown in FIG. 1 is a six-axis articulated robot, and a GMAW welding torch 11 is attached to the tip of the robot. GMAW includes, for example, MIG (Metal Inert Gas) welding and MAG (Metal Active Gas) welding, and in this embodiment, MAG welding is used as an example. The welding robot 10 is not limited to a six-axis articulated robot, and may be, for example, a portable small robot. An example of a portable small robot is an orthogonal robot with three or less axes. A welding device having a drive unit such as a cart may also be used. Furthermore, the welding method may be electrogas welding or electroslag welding in addition to GMAW.

The welding wire 13 is supplied to the welding torch 11 from the wire feeder 12. The welding wire 13 is fed from the tip of the welding torch 11 toward the welding point. The power supply unit 30 supplies power to the welding wire 13. This power applies an arc voltage between the welding wire 13 and the workpiece W, generating an arc. The power supply unit 30 is provided with a current sensor (not shown) that detects the welding current flowing from the welding wire 13 to the workpiece W during welding, and a voltage sensor (not shown) that detects the arc voltage between the welding wire 13 and the workpiece W.

The power supply device 30 has a processing unit and a storage unit (not shown). The processing unit is, for example, a CPU (Central Processing Unit). The storage unit is, for example, a volatile or non-volatile memory such as a HDD (Hard Disk Drive), a ROM (Read Only Memory), or a RAM (Random Access Memory). The processing unit controls the power applied to the welding wire 13 by executing a computer program for power supply control stored in the storage unit. The power supply device 30 is also connected to the wire feeder 12, and the processing unit controls the feed speed and feed amount of the welding wire 13. The composition and type of the welding wire 13 are selected according to the welding target.

The visual sensor 40 is composed of, for example, a CCD (Charge Coupled Device) camera. The placement position of the visual sensor 40 is not particularly important, and it may be attached directly to the welding robot 10, or may be fixed to a specific location in the vicinity as a surveillance camera. When the visual sensor 40 is attached directly to the welding robot 10, the visual sensor 40 moves to capture images of the area around the tip of the welding torch 11 in accordance with the operation of the welding robot 10. The visual sensor 40 may be composed of multiple cameras.

The direction from which the visual sensor 40 takes the image is not particularly important. For example, if the welding proceeds in the forward direction, the visual sensor 40 may be positioned to take images of the front side, or the side or rear side. Therefore, the imaging range of the visual sensor 40 may be determined appropriately. Note that it is preferable to take images from the front side in order to prevent interference with the welding torch 11.

In this embodiment, a visual sensor 40 is attached directly to and fixed on the welding robot 10, and a moving image is captured as a welding image so that the imaging range includes at least the workpiece W, the welding wire 13, and the arc. Note that various imaging settings related to the welding image may be specified in advance, or may be switched according to the operating conditions of the welding system 1. Examples of imaging settings include the frame rate, the number of pixels of the image, the resolution, and the shutter speed.

The components constituting the welding system 1 are communicatively connected by various wired/wireless communication methods. The communication method here is not limited to one, and multiple communication methods may be combined for connection.

Figure 2 is a perspective view for explaining the position of the visual sensor 40. In this embodiment, the workpiece W is a butt joint. The workpiece W is two metal plates that are butted together with a groove between them. A ceramic backing material 14 is attached to the back side of the two butted metal plates. A metal backing material may be used on the back side, or no backing material may be used. In a butt joint, arc welding is performed in one direction along the groove. In the following, the direction in which the welding proceeds is referred to as "forward." In Figure 2, the direction in which the welding proceeds is indicated by an arrow. For this reason, the welding torch 11 is located behind the visual sensor 40. In this embodiment, the side where the right hand is located when looking backward from the visual sensor 40 is referred to as the "right side," and the side where the left hand is located when looking forward is referred to as the "left side."

In this embodiment, the workpiece W is placed horizontally so that the surface on the side to be welded faces vertically upward. Therefore, the welding robot 10 welds the workpiece W from above the workpiece W. As shown in FIG. 2, the visual sensor 40 may be placed diagonally upward with respect to the welding position of the workpiece W. The imaging range of the visual sensor 40 includes the welding position of the workpiece W, and captures an image of the welding position during arc welding. This image includes the molten pool, the welding wire 13, and the arc. The visual sensor 40 in this embodiment can continuously capture still images of, for example, 1024 x 768 pixels. In other words, the visual sensor 40 can capture the welding image as a moving image.

[Configuration of the robot control device]
3 shows an example of the configuration of robot control device 20 that controls the operation of welding robot 10. Robot control device 20 includes a CPU 201 that controls the entire device, a memory 202 that stores data, an operation panel 203 including a plurality of switches, a teaching pendant 204 used in teaching work, an input/output unit 205, and a communication unit 206. Memory 202 is formed of, for example, a volatile or non-volatile storage device such as a ROM, a RAM, or a HDD. Memory 202 stores a control program 202A used to control welding robot 10. CPU 201 controls various operations performed by welding robot 10 by executing control program 202A.

The operation panel 203 and the teaching pendant 204 are used to input instructions to the robot control device 20. The operator can input a teaching program using the teaching pendant 204. The robot control device 20 controls the welding robot 10 according to the teaching program input from the teaching pendant 204. Note that the teaching program can also be automatically created based on CAD (Computer-Aided Design) information, for example, using a computer (not shown). The operation contents defined in the teaching program are not particularly limited and may differ depending on the specifications of the welding robot 10 and the welding method.

The power supply device 30 and the drive circuit of the welding robot 10 are connected to the input/output unit 205. The power supply device 30 provides the CPU 201 with, for example, the current value of the welding current detected by a current sensor (not shown) and the voltage value of the arc voltage detected by a voltage sensor (not shown) via the input/output unit 205. The CPU 201 also outputs a control signal based on the control program 202A to the drive circuit (not shown) provided in the welding robot 10.

The communication unit 206 is a communication module for wired or wireless communication. The communication unit 206 is used for data communication with the power supply device 30 and the data processing device 50. There are no particular limitations on the communication method or standard used by the communication unit 206, and multiple methods may be combined.

The robot controller 20 also controls the movement speed and protruding direction of the welding torch 11 by controlling each axis of the welding robot 10. The robot controller 20 also controls the weaving operation of the welding robot 10 according to the set period, amplitude, and welding speed. The weaving operation refers to alternately swinging the welding torch 11 in a direction intersecting the welding progress direction. The robot controller 20 performs weaving operation and weld line tracing control. The weld line tracing control is an operation to control the left and right position of the welding torch 11 with respect to the progress direction so that a bead is formed along the weld line. The robot controller 20 also controls the feed speed of the welding wire 13 by controlling the wire feeder 12 via the power supply 30.

[Configuration of data processing device]
4 is an explanatory diagram for explaining an example of the configuration of the data processing device 50. The data processing device 50 is configured by, for example, a computer. The computer is configured to include a main body 510, an input unit 520, and a display unit 530. The main body 510 is configured to include a CPU 511, a ROM 512, a RAM 513, a non-volatile storage device 514, an input/output interface 515, a communication interface 516, and a video output interface 517. The CPU 511, the ROM 512, the RAM 513, the non-volatile storage device 514, the input/output interface 515, the communication interface 516, and the video output interface 517 are connected to each other via a bus or a signal line so as to be able to communicate with each other.

The non-volatile storage device 514 stores a learning program 514A that performs deep learning using predetermined learning data, a trained model 514B that is generated through the execution of the learning program 514A, an information generation program 514C that generates characteristic information related to welding using the trained model 514B, and image data 514D. Details will be described later, but in this embodiment, three trained models are used. Therefore, the learning programs for generating each trained model and the training data used in the learning process are also different. In addition, an operating system and application programs are also installed in the non-volatile storage device 514.

By executing the program by the CPU 511, the data processing device 50 realizes various functions. In the case of this embodiment, the data processing device 50 realizes a function of generating each trained model by machine learning and a function of performing various processes during actual welding using the trained model. The contents of these functions will be described later. The data processing device 50 may be divided according to the function of generating a trained model and the function of performing control processing based on information output from the trained model during actual welding. From the viewpoint of versatility, it is more preferable to divide the data processing device 50 according to each function. The ROM 512 stores a BIOS (Basic Input Output System) executed by the CPU 511, etc. The RAM 513 is used as a working area for the program read from the non-volatile storage device 514.

The input/output interface 515 is connected to an input unit 520 consisting of a keyboard, mouse, etc. The visual sensor 40 is also connected to the input/output interface 515. Image data output from the visual sensor 40 is provided to the CPU 511 via the input/output interface 515. The communication interface 516 is a communication module for wired or wireless communication. The video output interface 517 is connected to a display unit 530 consisting of, for example, a liquid crystal display or an organic EL (Electro-Luminescence) display, and outputs a video signal corresponding to the video data provided by the CPU 511 to the display unit 530.

[Welding System Operation]
The operation of the welding system 1 according to this embodiment will be described below.

[Trained model]
First, the generation of the trained model according to this embodiment will be described. The trained model is generated by executing a learning program. In this embodiment, three trained models are handled. For convenience, the three trained models are described as "trained model A" (also referred to as "third trained model"), "trained model B" (also referred to as "first trained model"), and "trained model C" (also referred to as "second trained model"). For convenience, the training data used to generate each of the three trained models will be described as "trained data A", "trained data B", and "trained data C". In addition, when the teacher data needs to be separately described, they will be similarly referred to with reference symbols. FIG. 5 is a table summarizing an outline of each trained model according to this embodiment.

The trained model A is a trained model generated by training using training data A consisting of a pair of a welding image and welding information that is added to the welding image by manually specifying an area. The welding information here corresponds to feature information indicating areas corresponding to various elements contained in the welding image, which will be described in detail later, and includes, for example, coordinate information of each area in the image. Elements contained in the welding image include flow parts and plasma gas parts that are generated by welding. The trained model A inputs a welding image and outputs the welding information of the welding image. As described above, the task of manually specifying areas in a welding image and adding welding information generally involves a high load and cost. Therefore, in this embodiment, a relatively small amount of training data A is used in the training process to generate the trained model A. In this embodiment, as data obtained during welding, data of a welding image that is easy to capture the characteristics of a welding phenomenon is used as a preferred example, but for example, acoustic data such as an arc sound, numerical data such as current and voltage may be used, or a combination of these data may be used.

Trained model B is a trained model that is generated by training using training data B that consists of a combination of a welding image and the welding information of that welding image, and correct/incorrect information for that combination. The correct/incorrect information here is information that indicates whether or not the combination of a welding image and the welding information of that welding image is correct, as will be described in detail later. Thus, trained model B takes as input a combination of a welding image and the welding information of that welding image, and outputs a result indicating whether or not the combination is correct.

Trained model C is a trained model that is generated by training using training data C consisting of combinations of welding images and their welding information obtained using trained model A that are determined to be correct by trained model B. In other words, trained model C and trained model A have the same type of input information and the same type of output information, but use different data as training data. In particular, when training trained model A, a small amount of training data is used, taking into account the effort and cost of preparing the training data. On the other hand, when training trained model C, a large amount of highly accurate and unmanual training data output using trained model A and trained model B is used.

Figure 6 is a diagram conceptually explaining the generation process of trained model A and trained model C, i.e., the learning process. The learning process in this embodiment is performed using a convolutional neural network (CNN), which includes multiple convolutional layers, multiple pooling layers, and multiple fully connected layers. Note that the configuration of the convolutional neural network is not limited to the above, and the number of layers and the configuration may be different.

In the learning process for trained model A and trained model C, the learning model inputs a welding image and outputs welding information showing various characteristics that appear in the welding image. The welding image input to the learning model includes a molten pool, a welding wire 13, and an arc. In the case of this embodiment, welding is performed with different welding condition settings, and welding images are prepared by capturing images of each welding state. The welding conditions in this embodiment include, for example, the extension length of the welding wire 13, the brightness (F value) of the image data, the shape of the workpiece W, the shape of the contact tip, the welding current, the arc voltage, the welding speed, the weaving conditions, and the attitude (welding attitude) of the welding wire 13 relative to the workpiece W. The shape of the contact tip includes, for example, a straight nozzle and a tapered nozzle. In addition, the information that specifies the welding attitude includes, for example, the retreat angle, the advance angle, and the deviation of the welding line relative to the groove.

In this embodiment, the welding information used is the tip of the welding wire 13 (hereinafter also referred to as the wire tip), the center point of the arc (hereinafter also referred to as the arc center), the positions of the left and right tips of the molten pool, and the positions of the left and right ends of the molten pool. The welding information used as teaching data A when generating trained model A is input by the operator, for example, by following instructions on an operation screen that supports the teaching work and specifying a specific position on the welding image.

Figure 7 is an explanatory diagram for explaining an example of a screen used for teaching work. The welding image shown in Figure 7 includes a molten pool 15, a welding wire 13, and an arc 16. In Figure 7, the molten pool 15 is shown shaded. Also, Figure 8 is an explanatory diagram showing a specific example of a welding image obtained by welding and an example of welding information within the welding image. Here, for ease of explanation, positions corresponding to the coordinates indicated in the welding information are drawn on the welding image.

In this embodiment, the operator teaches the coordinate position of the arc center (ArcX, ArcY), the coordinate position of the wire tip (WireX, WireY), the coordinate position of the left end of the molten pool tip (Pool_Lead_Lx, Pool_Lead_Ly), the coordinate position of the right end of the molten pool tip (Pool_Lead_Rx, Pool_Lead_Ry), the coordinate position of the left end of the molten pool (Pool_Lx), and the coordinate position of the right end of the molten pool (Pool_Rx) as welding information constituting the teacher data A used for learning the learned model A. The operator inputs welding information by specifying a specific position on the screen. The coordinates that give the boundary between the welding wire 13 and the arc are an example of the position coordinates of the wire tip. In addition, the left end of the molten pool tip, the right end of the molten pool tip, the left end of the molten pool, and the right end of the molten pool are examples of information regarding the behavior of the molten pool 15. Furthermore, if the left and right ends of the molten pool are known, the width of the molten pool 15 can be calculated.

In the learning process for generating trained model A, a welding image prepared as teacher data A is input, and various welding information is output. The output welding information is then compared with the welding information prepared as teacher data A, and the error is calculated using an error function and fed back. This feedback adjusts various parameters in each layer of the learning model, and learning progresses. This process is repeated the number of times as many times as the number of teacher data A, thereby generating trained model A. The learning process for generating trained model C is basically the same as trained model A, except that the teacher data used and the amount of data used are different.

The learning process for generating trained model B is performed using a convolutional neural network (CNN) like trained model A and trained model C, and includes multiple convolution layers, multiple pooling layers, and multiple fully connected layers. In the learning process for trained model B, the learning model inputs a combination of welding images and welding information, and outputs correct/incorrect information. The correct/incorrect information here is classified into two categories: OK and NG. In other words, when "OK" is output, it indicates that the combination of the input welding images and welding information is correct. On the other hand, when "NG" is output, it indicates that the combination of the input welding images and welding information is incorrect. Note that the correct/incorrect information is not limited to two categories, and three or more categories may be used depending on the accuracy of the classification.

9 is an explanatory diagram for explaining data used as teacher data B in the learning process for generating the trained model B. The teacher data used to generate the trained model B is prepared by, for example, manually labeling the combination of the welding image and the welding information as correct or not. Teacher data 901 shows an example of data treated as NG. In the combination of the welding image and the welding information shown in the teacher data 901, a ceramic joint is identified as a position on the tip of the molten pool. Since this means that there is an error in the coordinate position shown as the welding information, this combination of the welding image and the welding information is labeled as "NG" and used as teacher data B. In the combination of the welding image and the welding information shown in the teacher data 902, the position on the tip of the molten pool is not identified. Since the position on the tip of the molten pool is not visible due to the arc light, it is correct that it is not identified. Therefore, this combination of the welding image and the welding information is labeled as "OK" and used as teacher data B.

［Overall Overview］
FIG. 10 is a diagram for explaining an overall outline of the process according to the present embodiment. The process according to the present embodiment is divided into two phases: a learning phase and a control phase. The learning phase is a phase in which three trained models are generated, and the control phase is a phase in which actual welding control is performed using the trained model C generated in the learning phase. The process of each phase is realized by the CPU 511 of the data processing device 50 reading and executing various programs stored in the non-volatile storage device 514 or the like. Note that, in each learning process step described later, a configuration may be adopted in which a verification process or a test process is further performed on the obtained trained model. In that case, each learning data may include verification data or test data.

First, the learning phase will be described. In S1001, the data processing device 50 performs a learning process to generate a trained model A using training data A prepared in advance. The learning process here is as described with reference to FIG. 6. In this embodiment, the training data A is a relatively small amount of data that has been prepared manually in advance.

In S1002, the data processing device 50 inputs the welding image to be processed to the trained model A generated in S1001, thereby acquiring welding information for the welding image. The number of welding images input here is not particularly limited, but it is desirable to use more welding images in order to generate a trained model C with higher accuracy.

In S1003, the data processing device 50 performs a learning process to generate a trained model B using previously prepared learning data B. The learning process here is as described with reference to FIG. 9 and the like. When preparing the learning data B, it may be created by manually labeling, i.e., assigning "OK" or "NG", using a combination of the welding image and welding information obtained in the process of S1002.

In S1004, the data processing device 50 inputs the combination of the welding image and welding information output in S1002 to the trained model B generated in S1003, and obtains correct/incorrect information for the combination.

In S1005, the data processing device 50 extracts learning data C from the combination of the welding image and the welding information based on the correct/incorrect information obtained as a result of the processing in S1004. More specifically, the data processing device 50 extracts the combination of the welding image and the welding information that is determined to be correct, i.e., "OK", in the correct/incorrect information, and identifies it as learning data C. Note that in the flow of FIG. 10, the correct/incorrect information and the combination corresponding to the correct/incorrect information are input separately to the processing in S1005, but this is not limited to this. For example, in the processing in S1004, the combination of the obtained welding image and the welding information, which is the input data of the trained model B, and the correct/incorrect information, which is the output data, may be associated with each other, retained, and managed, and the data may be output collectively to the processing in S1005.

In S1006, the data processing device 50 performs a learning process to generate a trained model C using the training data C identified as a result of the process in S1005. The learning process here is as described with reference to FIG. 6. The trained model C here may be trained based on a trained model A that has already been generated, or may be trained completely separately from the trained model A. In addition, the training data C used in the learning process in this step may include the training data A used in S1001.

Next, the control phase will be described. In this control phase, actual welding is performed by the welding robot 10. In this embodiment, the welding speed is mainly controlled as the automatic welding control, so a control signal is input to the welding robot 10 to control the robot's movement. However, when controlling the welding current or arc voltage, a control signal may be input to the welding power source to control the welding conditions such as the welding current and arc voltage. In S1011, the data processing device 50 inputs the welding image to be processed to the learned model C generated in the learning phase, thereby acquiring welding information for the welding image. The welding image here corresponds to a welding image captured while the welding robot 10 is operating in real time.

In S1012, the data processing device 50 uses the welding information obtained in the processing of S1011 to control the welding robot 10 and cause it to perform welding.

(Welding Control)
The welding of the welding system 1 during actual welding will be described below. This operation is performed in the control phase shown in FIG. 10. When performing arc welding, the operator starts the robot control device 20, the power supply device 30, and the data processing device 50. The robot control device 20 controls the movement of the welding robot 10, and the power supply device 30 performs welding according to preset welding conditions. The data processing device 50 also acquires welding images captured by the visual sensor 40, and sequentially calculates welding information related to arc welding using the trained model C. In this embodiment, as described above, the position of the wire tip, the arc center, the right and left end positions of the tip of the molten pool, and the right and left end positions of the molten pool are used as the welding information.

Figure 11 is a flowchart explaining the processing operations of the robot control device 20 and the power supply device 30. When starting arc welding, the operator operates the teaching pendant 204 provided on the robot control device 20 to input a teaching program, various setting values, and a welding start instruction to the robot control device 20. The teaching program here specifies the movement of the welding robot 10, etc.

At S1101, the robot control device 20 receives the teaching program, various setting values, and an instruction to start welding.

At S1102, the robot control device 20 commands the power supply device 30 to start welding (apply voltage).

At S1103, the power supply unit 30 receives a command to start welding from the robot control unit 20.

In S1104, the power supply device 30 controls a built-in power supply circuit (not shown) to supply power, thereby starting welding. This applies a voltage between the welding wire 13 (see FIG. 1) and the workpiece W (see FIG. 1), generating an arc.

At S1105, the robot control device 20 transmits a control signal to the power supply device 30 or the welding robot 10 to perform welding control. The welding control includes automatic welding control (S1120), weaving operation control (S1121), and weld line tracking control (S1122). In automatic welding control, the data processing device 50 automatically moves the welding torch 11 in the welding direction while transmitting a correction signal to the welding robot 10 or the power supply device 30 to control at least one of the welding speed, welding current, or arc voltage, and the welding robot 10 or the power supply device 30 performs welding according to the correction signal. From the viewpoint of ease of control, it is preferable that the automatic welding control includes welding speed control, and in this embodiment, only welding speed control is performed.

In S1106, the robot controller 20 determines whether or not it is necessary to stop welding. For example, it may determine that it is necessary to stop welding when an instruction to stop welding is received from the operator, when the welding end position is detected by a sensor provided on the welding robot 10, or when a welding abnormality is detected, or when both are detected. If it is not necessary to stop welding (NO in S1106), the processing of the robot controller 20 proceeds to S1107. On the other hand, if it is necessary to stop welding (YES in S1106), the processing of the robot controller 20 proceeds to S1108.

In S1107, the robot control device 20 receives welding information from the data processing device 50. The welding information received here is the welding information output by the data processing device 50 using the trained model C. Details of the generation of the welding information received in this process will be described later with reference to FIG. 12. After that, the process returns to S1105, and the robot control device 20 repeats the process using the received welding information.

At S1108, the robot control device 20 stops the welding control.

At S1109, the robot control device 20 commands the power supply device 30 to stop welding. Welding is stopped by stopping the supply of welding power.

At S1110, the robot control device 20 instructs the data processing device 50 to stop generating welding information.

At S1111, the power supply unit 30 receives a command to stop welding from the robot control unit 20.

At S1112, the power supply device 30 stops welding by controlling the power supply circuit via a CPU (not shown). This ends the operation of the robot control device 20 and the power supply device 30.

(Welding information generation process)
The process of generating the welding information received by the robot control device 20 in S1107 in Fig. 11 will be described with reference to Fig. 12. Fig. 12 is a flow chart for explaining the process of generating the welding information by the data processing device 50, executing welding control based on the welding information, and transmitting correction signals for each control factor to the power supply device 30. The control factors refer to welding conditions, such as the welding speed, welding current, arc voltage, weaving operation, and weld line tracking.

At S1201, the data processing device 50 receives a welding image of the welding position from the visual sensor 40 while automatic welding is being performed.

In S1202, the data processing device 50 reduces the size of the received welding image and converts it into a grayscale image. For example, the data processing device 50 reduces the size of the welding image to 120 x 100 pixels. The image processing here is an example, and other processing may be performed further, or part of the processing may be omitted depending on the processing load, etc.

At S1203, the data processing device 50 inputs the reduced welding image into the trained model C generated by the above-mentioned learning phase, and obtains the welding information output as a result. In this embodiment, the trained model C outputs the following welding information: the coordinate position of the arc center (ArcX, ArcY), the coordinate position of the wire tip (WireX, WireY), the coordinate position of the left end of the molten pool tip (Pool_Lead_Lx, Pool_Lead_Ly), the coordinate position of the right end of the molten pool tip (Pool_Lead_Rx, Pool_Lead_Ry), the coordinate position of the left end of the molten pool (Pool_Lx), and the coordinate position of the right end of the molten pool (Pool_Rx).

In S1204, the data processing device 50 performs welding control based on the acquired welding information, and calculates correction signals for each control factor. In the case of this embodiment, the correction signals calculated are a left-right position correction amount ΔX of the welding torch 11, a speed correction amount ΔV of the welding torch 11, and a set value w of the width of the weaving operation (hereinafter referred to as "weaving width"). The left-right position correction amount ΔX of the welding torch 11 is calculated by the following formula (1).
PoolCenX=(Pool_Lx+Pool_Rx)/2
ΔX=PoolCenX−X0 (1)
Here, X0 is a reference value for the left-right position of the center of the molten pool, and is a preset value. The calculated left-right position correction amount ΔX is used to move the welding torch 11 left-right so that the center PoolCenX of the molten pool image coincides with X0.

When calculating the speed correction amount ΔV, first, ΔL is calculated by the following equation (2).
LeadY=(Pool_Lead_Ly+Pool_Lead_Ry)/2-ArcY
ΔL=LeadY-L0 (2)
Here, L0 is the distance between the tip of the molten pool and the center of the arc, and is preset to a value that can form the best quality back bead. LeadY is also the distance between the tip of the molten pool and the center of the arc. Here, LeadY is calculated by formula (2) using Pool_Lead_Ly, Pool_Lead_Ry, and ArcY, which are image feature information extracted from the welding image.

ΔL is the difference between LeadY and L0. Once the distance ΔL is calculated, the speed correction amount ΔV is calculated by the following equation (3) using a conversion coefficient β.
ΔV=ΔL×β (3)

The set value w of the weaving width is calculated by the following formula (4) or (5).
Leadw=Pool_Lead_Rx-Pool_Lead_Lx
IF Leadw<w0 THEN w=0 (4)
IF Leadw≧w0 THEN w=(Leadw−α) (5)
Here, w0 is the gap width of the groove, and is preset to the minimum value at which weaving is required.

Leadw also indicates the gap width, and is a value calculated as described above because the coordinate positions (Pool_Lead_Lx, Pool_Lead_Rx) of the welding information extracted from the welding image correspond to the left and right positions of the groove. According to the above formula (4), when the gap width Leadw is less than w0, the weaving width setting value w becomes 0, and no weaving operation is performed. On the other hand, according to the above formula (5), when the gap width Leadw is w0 or more, the weaving width setting value w is set to a value α smaller than the gap width Leadw. Note that α is a value that is set in advance.

The process of S1204 may be performed by generating a trained model by a learning process separate from the learning process that generates trained models A to C described above. For example, a trained model may be generated using information that associates welding information with welding result information, etc., as teacher data. Then, the welding information may be input to this trained model, and correction signals for each control factor may be output to obtain optimal welding results. Here, welding result information may include welding current waveform data, arc voltage waveform data, arc sound data during welding, bead shape data, spatter generation data, fume generation data, welding defect data, etc., and it is preferable to treat at least one of these data as welding result information. In this case, the learning process may be implemented by a learning process using reinforcement learning, in addition to the supervised learning described above.

In S1205, the data processing device 50 transmits the correction signal calculated in S1204 to the robot control device 20.

In S1206, the data processing device 50 determines whether or not a stop command has been received from the robot control device 20. The stop command here corresponds to the command to stop generating welding information sent from the robot control device 20 in the processing of S1110 in FIG. 11. If a stop command has not been received (NO in S1206), the processing of the data processing device 50 returns to S1201. On the other hand, if a stop command has been received (YES in S1206), this processing flow is terminated.

[verification]
An operation verification was performed by applying the method according to the present embodiment. The verification results will be described below with reference to the drawings. In the verification of the present embodiment, first, 3700 welding images were prepared as training data A for generating a trained model A, and welding information was manually added to the training data A. Then, a trained model A was generated by a learning process using the training data A. Next, the welding images were input to the trained model A to obtain the welding information of the welding images, and 17000 combinations of these were obtained.

Separately, combinations of welding images and welding information were prepared as training data B for generating trained model B, and correct/incorrect information for the combinations was added. In this embodiment, 397 combinations with correct/incorrect information of "OK" and 274 combinations with correct/incorrect information of "NG" were prepared, and these 671 pieces of data were used as training data B. Trained model B was then generated by a learning process using training data B. Next, the 17,000 combinations obtained by using trained model A were input to trained model B, and correct/incorrect information for each combination was obtained. Based on the results, 14,400 combinations judged to be "OK", i.e., correct, were obtained.

Furthermore, as the training data C for generating the trained model C, the 14,400 combinations determined to be correct in the trained model B and the 3,700 training data A used when generating the trained model A were used. Then, the trained model C was generated by performing a learning process using this total of 18,100 training data C.

In other words, with conventional learning methods, 18,100 pieces of training data must be manually created to generate a trained model to be used in actual welding control. However, with the method of this embodiment, it is only necessary to prepare 3,700 pieces of such training data. As a result, the cost of creating training data can be reduced by as much as 80%. If more training data is required to further improve the accuracy of the trained model, further cost reductions are possible.

Next, an example of improved accuracy of welding control according to this embodiment will be described. Figure 13 shows the recognition results of the molten pool tip width in a tapered workpiece in which the groove gap width changes continuously from 3 mm to 10 mm. The molten pool tip width is the Y coordinate above the molten pool tip minus the Y coordinate below the molten pool tip, that is, a feature value that corresponds to the gap width. In Figure 13, the vertical axis indicates the molten pool tip width [mm], and the horizontal axis indicates the passage of time [s]. Here, the focus is on the latter half of the welding, from 100 seconds to 170 seconds.

Line 1301 shows the recognition result when welding is performed using the trained model C generated by the learning method according to this embodiment. Line 1302 shows the recognition result when welding is performed using a trained model generated by a conventional learning method, i.e., a model equivalent to the trained model A described above. The hatching around each of the lines 1301 and 1302 indicates the variation range, i.e., the variation, of the molten pool tip width of each line. When the trained model generated by the conventional learning method is used, as shown by the line 1302, the variation in the molten pool tip width shown on the vertical axis is large. Since it is impossible for the groove width of an actual welded joint to vary in this way, it can be seen that the molten pool tip position cannot be correctly recognized and the recognition accuracy is unstable. On the other hand, when the trained model C generated by the learning method according to this embodiment is used, the line 1301 has less variation in the molten pool tip width compared to the line 1302, and the molten pool tip position is stably recognized.

The gap width at the end of welding around 170 seconds is expected to be approximately 10 mm. Referring to line 1302, the conventional learning method recognizes the gap as approximately 8.5 mm, which is narrower than the actual width. On the other hand, when trained model C generated by the learning method according to this embodiment is used, the gap width is recognized as approximately 10.5 mm, which is closer to the actual width of the tip of the molten pool. In light of the above, the use of the trained model generated by the learning method according to this embodiment enables more stable and accurate recognition than the conventional learning method.

Figure 14 is a diagram showing the error between the manual molten pool tip width value and the molten pool tip width recognition result using the trained model in the welding images around 120 seconds and 160 seconds in the graph shown in Figure 13. In Figure 14, the vertical axis shows the recognition error [mm] of the molten pool tip width. The error between the recognition result using the trained model according to the conventional method and the manual value is about 1.4 mm. On the other hand, the error between the recognition result using the trained model C according to this embodiment and the manual value is about 0.4 mm, which means that the molten pool tip width can be recognized more accurately than the conventional method.

Figure 15 is a confusion matrix regarding the judgment accuracy of the trained model B generated by the learning method according to this embodiment. When a combination of a welding image labeled "OK" as correct/incorrect information and the welding information is input, the trained model B correctly judges the welding image to be "OK" at a rate of 0.9, i.e., 90%. When a combination of a welding image labeled "NG" as correct/incorrect information and the welding information is input, the trained model B correctly judges the welding image to be "NG" at a rate of 0.7, i.e., 70%. That is, in this example, a trained model B capable of making judgments with high accuracy can be generated using a relatively small number of training data B, 671 pieces. In particular, the training data B for generating the trained model B can be prepared by simply labeling the welding image with OK/NG, which reduces the effort required.

As described above, this embodiment makes it possible to generate a trained model capable of extracting training data, taking into account the accuracy of training data used in machine learning processing assuming a welding site. In particular, it is possible to determine whether or not the combination of welding images and welding information is correct, and to extract appropriate training data, thereby significantly reducing the burden and cost of manually creating training data. Then, by using the trained model generated using that training data, it becomes possible to achieve more accurate welding.

<Other embodiments>
In the above embodiment, the learning process is performed using a welding image. However, the present invention is not limited to this configuration. In addition to the welding image, the data obtained by welding may include audio information and numerical information as data during learning. For example, the learning data for generating the trained model A and the trained model C may include the control parameters of the welding robot 10 when the welding image is acquired. Examples of the control parameters include information on the welding posture of the welding robot 10, welding conditions, welding materials, and shielding gas. Examples of the welding posture include information on whether the welding torch 11 is facing upward, downward, or sideways. Examples of the welding conditions include current, voltage, and welding speed. Examples of the audio information include arc sound. In addition, the welding information output from the trained model A and the trained model C is not limited to the welding information shown in FIG. 6 and the like, and may be other characteristic information that is correlated with the data obtained by welding.

In addition, each trained model described in the above embodiment may be updated as appropriate. In other words, after a certain amount of welding images has been accumulated, trained model A or trained model B may be used again to extract trained data C and update trained model C. This method reduces the cost of generating trained data C, making it easy to retrain trained model C to improve its accuracy.

In the above embodiment, the trained model B is configured to extract training data from the output result of the trained model A. This is not limited to the configuration, and may also be applied to the output result of the trained model C. With this configuration, for example, it is possible to select appropriate training data C when re-training the trained model C. As a more specific example, the data selected here may be used as verification data or test data for calculating the accuracy of the trained model C.

The present invention can also be realized by providing a program or application for implementing the functions of one or more of the above-described embodiments to a system or device via a network or storage medium, and having one or more processors in the computer of the system or device read and execute the program.

It may also be realized by a circuit that realizes one or more functions. Examples of circuits that realize one or more functions include an ASIC (Application Specific Integrated Circuit) and an FPGA (Field Programmable Gate Array).

As described above, the present specification discloses the following:
(1) A method for generating a trained model, comprising: a generation step of performing machine learning using a combination of data obtained during welding and feature information identified from the data as input, and a classification of the combination as correct or incorrect as output, to generate a first trained model.
According to this configuration, it is possible to extract learning data taking into consideration the accuracy of the learning data used in machine learning processing assuming a welding site.

(2) the data is a welding image,
The feature information is welding information indicating elements included in the welding image.
A method for generating a trained model according to (1).
According to this configuration, it is possible to carry out learning processing using the welding image and the welding information indicated therein.

(3) The method for generating a trained model according to (1) or (2), wherein the feature information is information relating to at least one of a flow portion and a plasma gas portion generated by welding.
According to this configuration, it is possible to perform a learning process using information relating to at least one of the flow section and the plasma gas section.

(4) The characteristic information of the flow portion includes positions of both ends of a width of the molten pool and positions of both ends of a tip of the molten pool,
The characteristic information relating to the plasma gas portion includes a center position of the arc.
A method for generating a trained model according to (3).
According to this configuration, it is possible to perform a learning process using information relating to at least one of the flow section and the plasma gas section.

(5) In the training data for generating the first trained model,
The combinations we label as positive include
Data identifying at least the positions of both ends of the molten pool width, the positions of both ends of the molten pool tip, the center position of the arc, and the position of the wire tip, or
The data includes data in which at least one of the positions of both ends of the molten pool width or the positions of both ends of the molten pool tip cannot be recognized due to the influence of the arc light,
The combinations labeled as "no" include data in which the positions of both ends of the molten pool were misidentified due to the influence of a certain material.
A method for generating a trained model according to (4).
With this configuration, the accuracy of the first trained model can be improved in response to actual conditions in welding.

(6) A second generation step of performing machine learning using the combination classified as positive in the first trained model as training data, with data obtained during welding as input, and feature information corresponding to the data as output, to generate a second trained model.
A method for generating a trained model according to any one of (1) to (5).
With this configuration, it is possible to use the first trained model to generate a second trained model for controlling the welding robot with high accuracy.

(7) The combination classified using the first trained model is generated using a third trained model generated by performing machine learning using data obtained during welding as input and feature information identified from the data as output.
A method for generating a trained model according to any one of (1) to (6).
With this configuration, it is possible to generate data to which the first trained model is applied without human intervention.

(8) a first generation means for generating a first trained model by performing machine learning using a combination of data obtained during welding and feature information identified from the data as an input and a classification of the combination as correct or incorrect as an output; and
A second generation means for generating a second trained model by performing machine learning using a combination classified as positive in the first trained model as training data, with data obtained during welding as input, and feature information corresponding to the data as output;
having
A learning device characterized by:
With this configuration, it is possible to generate a first trained model and then use this to generate a second trained model for controlling a welding robot with high accuracy.

(9) Further comprising a third generation means for generating a third trained model by performing machine learning using data obtained during welding as an input and feature information identified from the data as an output,
The learning device described in (8), characterized in that a combination of input and output of the third trained model is a target to be classified using the first trained model.
With this configuration, it is possible to generate data to which the first trained model is applied without human intervention.

(10) The learning device described in (9), wherein the training data for generating the third trained model includes data obtained under different conditions for at least one element of the welding posture, welding conditions, welding material, and shielding gas in the welding operation.
According to this configuration, by using training data corresponding to various conditions, it is possible to enable the third trained model to perform highly accurate training with a relatively small amount of data.

(11) The training data for generating the second trained model includes:
A combination of data obtained during welding and characteristic information manually set on the data; and
The learning device according to (9) or (10), characterized in that it includes at least one of a combination classified as incorrect in the first trained model and a combination manually corrected.
According to this configuration, by including highly accurate manually prepared training data, it is possible to improve the accuracy of the second trained model.

(12) A control device having a first trained model and a second trained model generated by the learning device according to any one of (8) to (11),
An acquisition means for acquiring data related to welding;
A control means for outputting control information for a welding robot or a welding power source;
having
The control means
Feature information output by inputting the data acquired by the acquisition means into the second trained model, or
A control device characterized in that a combination of data acquired by the acquisition means and feature information output by inputting the data into the second trained model is input into the first trained model, and the control information is output based on any of the feature information of the combination classified as positive.
With this configuration, it is possible to control the welding robot with high accuracy using a trained model that has been trained using highly accurate teacher data.

(13) The welding robot control device further includes a learning means for performing a learning process using the control information output by the control means and the data acquired by the acquisition means as a result of controlling the welding robot or the welding power source based on the control information,
The control device according to (12) is further characterized in that the control means outputs control information for the welding robot or the welding power source based on the result of the learning process by the learning means.
According to this configuration, a learned model that has been trained using highly accurate teacher data is used to control a welding robot or welding power source, and then the learning process can be further carried out to further improve the accuracy of control.

(14) The control device according to (12) or (13), characterized in that the control information is at least one of a welding posture of the welding robot, welding conditions, and a control amount for a shielding gas.
According to this configuration, it is possible to improve the accuracy of the control amount for the welding posture of the welding robot, the welding conditions, and the shielding gas.

(15) A control device according to any one of (12) to (14),
A welding robot
A detection means for detecting welding by the welding robot;
A welding power source;
A system comprising:
According to this configuration, it is possible to control the welding robot with high accuracy using a trained model that has been trained using highly accurate teacher data.

A trained model generated by machine learning using a combination of data obtained during welding and feature information identified from the data as input and a classification of whether the combination is correct or incorrect as output,
On the computer,
A trained model that functions to output a classification of whether a combination of data obtained during welding and feature information identified from the data is correct when the combination is input.
According to this configuration, it is possible to extract learning data taking into consideration the accuracy of the learning data used in machine learning processing assuming a welding site.

Computer,
a first generation means for generating a first trained model by performing machine learning using a combination of data obtained during welding and feature information identified from the data as an input and a classification of whether the combination is correct or incorrect as an output;
a second generation means for performing machine learning using the combination classified as positive in the first trained model as training data, with data obtained during welding as input, and feature information corresponding to the data as output, to generate a second trained model;
A program to function as a
With this configuration, it is possible to generate a first trained model and then use this to generate a second trained model for controlling a welding robot with high accuracy.

Reference Signs List 1 Welding system 10 Welding robot 11 Welding torch 12 Wire feeder 13 Welding wire 20 Robot control device 30 Power supply device 40 Visual sensor 50 Data processing device 510 Main body 511 CPU
512 ROM
513 RAM
514 Non-volatile storage device 514A Learning program 514B Learned model 514C Information generation program 514D Image data 515 Input/output interface 516 Communication interface 517 Video output interface 520 Input unit 530 Display unit W Work

Claims (9)

a first generation step of performing machine learning using a combination of a welding image obtained during welding and feature information identified from the welding image as an input and a true/false classification indicating whether the combination is correct or not as an output to generate a first trained model ;
a second generation step of performing machine learning using the combination classified as positive in the first trained model as training data, a welding image obtained during the welding as an input, and the feature information corresponding to the welding image as an output to generate a second trained model;
In addition to providing
a third generation step of generating a combination to be classified using the first trained model by using a third trained model generated by performing machine learning using a welding image obtained during the welding as an input and the feature information identified from the welding image as an output;
the characteristic information includes welding information indicating information related to at least one of a flow portion generated by welding and a plasma gas portion, which are elements included in the welding image;
the welding information relating to the flow portion includes positions of both ends of a width of the molten pool and positions of both ends of a tip of the molten pool, and the welding information relating to the plasma gas portion includes a center position of an arc;
A method for generating a trained model. In the training data for generating the first trained model,
The combinations we label as positive include
Data identifying at least the positions of both ends of the molten pool width, the positions of both ends of the molten pool tip, the center position of the arc, and the position of the wire tip, or
The data includes data in which at least one of the positions of both ends of the molten pool width or the positions of both ends of the molten pool tip cannot be recognized due to the influence of the arc light,
The combinations labeled as "no" include data in which the positions of both ends of the molten pool were misidentified due to the influence of a certain material.
The method for generating a trained model according to claim 1 . a first generation means for performing machine learning using a combination of a welding image obtained during welding and feature information identified from the welding image as an input and a classification of correct or incorrect indicating whether the combination is correct or not as an output to generate a first trained model;
a second generation means for performing machine learning using a combination classified as positive in the first trained model as training data, a welding image obtained during the welding as an input, and feature information corresponding to the welding image as an output to generate a second trained model;
having
A third generation means for generating a third trained model by performing machine learning using a welding image obtained during welding as an input and feature information identified from the welding image as an output,
A combination of inputs and outputs of the third trained model is an object to be classified using the first trained model,
The training data for generating the third trained model includes data obtained under different conditions for at least one element of a welding posture, a welding condition, a welding material, and a shielding gas in the welding,
the characteristic information is welding information indicating information related to at least one of a flow portion and a plasma gas portion generated by welding, which are elements included in the welding image;
A learning device characterized in that the welding information related to the flow portion includes the positions of both ends of the molten pool width and the positions of both ends of the molten pool tip, and the welding information related to the plasma gas portion includes the center position of the arc . The training data for generating the second trained model includes:
A combination of data obtained during welding and characteristic information manually set on the data; and
The learning device according to claim 3 , further comprising at least one of a combination classified as incorrect in the first trained model and a combination manually corrected. A control device having a first trained model and a second trained model generated by the learning device according to claim 3 or 4,
An acquisition means for acquiring data related to welding;
A control means for outputting control information for a welding robot or a welding power source;
having
The control means
Feature information output by inputting the data acquired by the acquisition means into the second trained model, or
inputting a combination of the data acquired by the acquisition means and feature information output by inputting the data into the second trained model into the first trained model, and obtaining feature information of the combination classified as positive;
The control device outputs the control information based on any one of the above. a learning means for performing a learning process using the control information output by the control means and the data acquired by the acquisition means as a result of controlling the welding robot or the welding power source based on the control information;
6. The control device according to claim 5 , wherein the control means further outputs control information for the welding robot or the welding power source based on a result of the learning process performed by the learning means. 7. The control device according to claim 5 , wherein the control information is at least one of a welding posture of the welding robot, welding conditions, and a control amount for a shielding gas. A control device according to any one of claims 5 to 7 ;
A welding robot
A detection means for detecting welding by the welding robot;
A welding power source;
A system comprising: A program for causing a computer to generate a trained model,
Computer,
a first generation means for performing machine learning using a combination of a welding image obtained during welding and feature information identified from the welding image as an input and a true/false classification indicating whether the combination is correct or not as an output to generate a first trained model;
a second generation means for performing machine learning using a combination classified as positive in the first trained model as training data, a welding image obtained during the welding as an input, and the feature information corresponding to the welding image as an output, thereby generating a second trained model;
a third generation means for generating a combination to be classified using the first trained model by using a third trained model generated by performing machine learning using a welding image obtained during the welding as an input and the feature information identified from the welding image as an output;
Function as a
the characteristic information includes welding information indicating information related to at least one of a flow portion generated by welding and a plasma gas portion, which are elements included in the welding image;
the welding information relating to the flow portion includes positions of both ends of a width of the molten pool and positions of both ends of a tip of the molten pool, and the welding information relating to the plasma gas portion includes a center position of an arc;
A program characterized by:

Images