JP7611872B2 - WELDING APPARATUS, WELDING METHOD, AND PROGRAM - Google Patents

Description

This disclosure relates to a welding device, a welding method, and a program.

There is an increasing need for automating welding in order to improve welding quality (reduce defects). For example, Patent Document 1 describes an automatic welding system that detects characteristic points such as the electrodes and the edge of the molten pool from an image of the welding point, and controls the weaving width of the electrode based on the coordinate positions of the characteristic points.

JP 2018-192524 A

Experienced welders make adjustments to the position of the welding torch, etc., based on information that is difficult to quantify, such as not only the positions of the electrode and the edge of the molten pool, but also the shape of the molten pool and how the molten pool fits into the groove boundary of the base material. However, conventional automatic welding systems cannot use information that is difficult to quantify, other than coordinate information, for automatic control. For this reason, it has been difficult to improve welding quality to a level close to that of an experienced welder with conventional automatic welding systems.

This disclosure was made in consideration of these issues, and provides a welding device, welding method, and program that can perform automatic welding with higher welding quality by using a learning model that learns the welding skills of experienced welders.

According to one aspect of the present disclosure, a welding device that performs welding by controlling at least one of the control objects, a welding wire and an electrode, includes a camera that captures a welding image including the tip of the control object and the welding point, a data generation unit that generates learning data including a learning image that is a welding image captured in the past and information indicating the appropriateness of the operation of the control object judged by a welder by looking at the learning image, a learning unit that uses the learning data to construct a learning model that takes the welding image as an input and outputs the appropriateness of the operation of the control object, an estimation unit that uses the learning model to estimate the appropriateness of the operation of the control object in the acquired welding image, and a control unit that controls the control object based on the estimation result of the appropriateness of the operation.

According to one aspect of the present disclosure, a welding method for controlling at least one of the control objects, a welding wire and an electrode, to perform welding includes the steps of: capturing a welding image including the tip of the control object and the welding location; generating learning data including a learning image that is a previously captured welding image and information indicating the appropriateness of the operation of the control object judged by a welder by looking at the learning image; using the learning data to construct a learning model that inputs the welding image and outputs the appropriateness of the operation of the control object; using the learning model to estimate the appropriateness of the operation of the control object in the acquired welding image; and controlling the control object based on the estimated result of the appropriateness of the operation.

According to one aspect of the present disclosure, the program causes a welding device that performs welding by controlling at least one of the control objects, a welding wire and an electrode, to execute the steps of: capturing a welding image including the tip of the control object and the welding location; generating learning data including a learning image that is a previously captured welding image and information indicating the appropriateness of the operation of the control object judged by a welder by looking at the learning image; using the learning data, constructing a learning model that inputs the welding image and outputs the appropriateness of the operation of the control object; using the learning model, estimating the appropriateness of the operation of the control object in the acquired welding image; and controlling the control object based on the estimated result of the appropriateness of the operation.

The welding device, welding method, and program disclosed herein use a learning model that has learned the welding skills of experienced welders to perform automatic welding with higher welding quality.

1 is a schematic diagram showing an overall configuration of a welding device according to a first embodiment of the present disclosure. 2 is a block diagram showing a functional configuration of a control device according to the first embodiment of the present disclosure. FIG. 5 is a flowchart illustrating an example of a process in a learning mode of the welding device according to the first embodiment of the present disclosure. FIG. 2 is a diagram illustrating an example of learning data according to the first embodiment of the present disclosure. 5 is a flowchart illustrating an example of a process in a control mode of the welding device according to the first embodiment of the present disclosure. FIG. 11 is a block diagram showing a functional configuration of a control device according to a second embodiment of the present disclosure. FIG. 13 is a diagram illustrating an example of learning data according to a third embodiment of the present disclosure. FIG. 11 is a schematic diagram showing an overall configuration of a welding device according to a fourth embodiment of the present disclosure. FIG. 13 is a diagram illustrating an example of learning data according to a fourth embodiment of the present disclosure. FIG. 13 is a schematic diagram showing an overall configuration of a welding device according to a fifth embodiment of the present disclosure. FIG. 13 is a diagram illustrating an example of learning data according to a fifth embodiment of the present disclosure. FIG. 13 is a diagram for explaining functions of a control device according to a sixth embodiment of the present disclosure. FIG. 11 is a block diagram showing a functional configuration of a welding device according to another embodiment of the present disclosure.

First Embodiment
Hereinafter, a welding device 1 according to a first embodiment of the present disclosure will be described with reference to FIGS. 1 to 5. FIG.

(Overall configuration of welding equipment)
FIG. 1 is a schematic diagram showing an overall configuration of a welding device according to a first embodiment of the present disclosure.
As shown in FIG. 1 , the welding device 1 includes a main body 10 and a control device 20 .

The main body 10 performs arc welding on the base material 3 under the control of the control device 20. The main body 10 includes a welding torch 100, a welding wire supply unit 110, and a camera 120.

An electrode 101 made of tungsten or the like is attached to the welding torch 100.

The welding wire supply unit 110 delivers the welding wire 40 through the wire nozzle 111 toward the welding point of the base material 3.

When the welding torch 100 supplies power to the electrode 101, the base material 3 (groove 31) melts due to arc discharge from the electrode 101, and the tip of the welding wire 40 is supplied to the molten part of the base material 3 and melts, forming a molten pool 41 in which the base material and welding wire are diluted. This molten pool cools and solidifies, welding the base material 3.

The camera 120 captures a welding image including the electrode 101, the tip of the welding wire 40, and the welded portion of the base material 3.

The control device 20 outputs a control signal for controlling at least one of the control objects, the electrode 101 and the welding wire 40, based on the welding image captured by the camera 120. In this embodiment, as shown in FIG. 1, an example is described in which the welding device 1 is a so-called TIG welding device that uses an electrode 101 made of tungsten or the like. In addition, an example is described in which the control device 20 according to this embodiment controls both the electrode 101 and the welding wire 40. In addition, the control of the electrode 101 at this time also includes control of swinging the electrode left and right (weaving width).

In other embodiments, the welding device 1 may be a MAG welding device, a MIG welding device, or the like. In this case, the welding torch 100 uses a welding wire 40 fed into the welding torch 100 from a welding wire supply unit 110 instead of the electrode 101. In this case, the control device 20 controls only the welding wire 40.

The control signal is a signal that indicates, for example, the position of the electrode 101, the position of the welding wire 40, and the amount of welding wire 40 that is pulled out (fed out). The welding torch 100 moves the position of the electrode 101 in the left-right direction (±Y direction) in accordance with the control signal of the control device 20. The welding wire supply unit 110 moves the position of the welding wire 40 in the up-down direction (±Z direction) and left-right direction (±Y direction) in accordance with the control signal of the control device 20. The welding wire supply unit 110 also feeds the welding wire 40 toward the base material 3 in accordance with the control signal of the control device 20.

(Functional configuration of the control device)
FIG. 2 is a block diagram showing a functional configuration of the control device according to the first embodiment of the present disclosure.
As shown in FIG. 2, the control device 20 includes a processor 21, a memory 22, a storage 23, and a communication interface 24.

The processor 21 operates according to a predetermined program to perform the functions of an acquisition unit 210, an estimation unit 211, a control unit 212, a data generation unit 213, and a learning unit 214.

The acquisition unit 210 acquires the welding image captured by the camera 120 of the main body unit 10.

The estimation unit 211 estimates the appropriateness of the operation of the welding wire 40 and the electrode 101 in the acquired welding image using a learning model M that takes a welding image as input and outputs the appropriateness of the operation of the welding wire 40 and the electrode 101. The estimation unit 211 reads out from the storage 23 the learning model M that has been previously learned by the learning unit 214 described below and uses it.

For example, the estimation unit 211 estimates the appropriateness of the operation for the movement of the electrode 101 in each of the left-right and up-down directions in three stages: "excessive," "appropriate," and "inadequate." The estimation unit 211 also estimates the operation for the movement of the welding wire 40 in each of the left-right and up-down directions in three stages: "excessive," "appropriate," and "inadequate." The estimation unit 211 may further express the operation regarding the amount of withdrawal of the welding wire 40 in three stages: "excessive," "appropriate," and "inadequate." In other embodiments, the evaluation of the operation may be divided into more than three stages.

The control unit 212 controls the welding wire 40 and the electrode 101 based on the estimation result of the estimation unit 211. For example, the control unit 212 generates a control signal to move the electrode 101 to the right if the operation of the electrode 101 to the left is estimated to be "excessive," and outputs the control signal to the welding torch 100 to move the electrode 101 to the left if the operation is estimated to be "insufficient." The welding torch 100 then moves the electrode 101 to the left or right in accordance with the control signal. Also, the control unit 212 does not control the left or right movement of the electrode 101 if the operation of the electrode 101 to the left is evaluated to be "appropriate" (does not output a control signal, or outputs a control signal that maintains the left or right position). The welding torch 100 then does not move the position of the electrode 101 to the left or right until it receives the next control signal.

The data generation unit 213 generates learning data that includes learning images, which are welding images captured in the past, and information indicating the degree of appropriateness of the operation of the welding wire 40 and the electrode 101 performed by the welder when the learning image was captured.

The learning unit 214 constructs a learning model M by performing deep learning using the learning data.

Memory 22 has memory space necessary for the operation of processor 21.

Storage 23 is a so-called auxiliary storage device, such as a hard disk drive (HDD) or a solid state drive (SSD). Storage 23 stores welding images captured by camera 120, a trained learning model M, and the like.

The communication interface 24 is an interface for sending and receiving various information (signals) with an external device (such as the main body 10).

The predetermined program executed by the processor 21 of the control device 20 is stored in a computer-readable recording medium. A computer-readable recording medium refers to a magnetic disk, a magneto-optical disk, a CD-ROM, a DVD-ROM, a semiconductor memory, etc. The computer program may be distributed to a computer via a communication line, and the computer that receives the program may execute the program. Furthermore, the program may be for realizing part of the above-mentioned functions. Furthermore, the program may be a so-called differential file (differential program) that can realize the above-mentioned functions in combination with a program already recorded in the computer system.

The control device 20 according to this embodiment has two modes: a learning mode and a control mode. In the learning mode, the control device 20 constructs a learning model M by the learning unit 214. In addition, in the control mode, the control device 20 controls the main body 10 of the welding device 1 (automatic execution of welding) using the learning model M that has been learned by the learning unit 214. The details of the processing in each mode are described below.

(Learning mode process flow)
FIG. 3 is a flowchart showing an example of a process in a learning mode of the welding device according to the first embodiment of the present disclosure.
Here, the flow of processing by the welding device 1 in the learning mode will be described with reference to FIG.

First, the learning unit 214 of the control device 20 acquires welding images and operation information to be used for learning (step S100). For example, the learning unit 214 acquires a plurality of welding images that have been captured of welding performed in the past by a welder (an experienced welder) and the contents of the operation (operation information) that the welder performed on the electrode 101 and the welding wire 40 when capturing each welding image, in association with each other.

The data generation unit 213 also creates learning data D1 and evaluation data D2 based on the acquired welding image and operation information (step S101).

FIG. 4 is a diagram illustrating an example of learning data according to the first embodiment of the present disclosure.
For example, as shown in FIG. 4, the data generating unit 213 generates learning data D1 by adding a label (correct answer data) indicating the degree of appropriateness of the operation of the electrode 101 and the welding wire 40 to each welding image. For example, one of "excessive", "appropriate", and "inadequate" is input for each of the electrode 101 and the welding wire 40. The data generating unit 213 similarly labels all welding images to indicate the direction of the operation as "excessive", "appropriate", or "inadequate". Note that the welder may view each welding image and perform an operation to specify a label. The data generating unit 213 generates learning data D1 by adding a label specified by the welder to each welding image. The data generating unit 213 also divides a part of the labeled learning data D1 into evaluation data D2.

The data generating unit 213 may determine the appropriateness of the operation of the electrode 101 and the welding wire 40 in each welding image based on the time series of the operation information, and automatically add a label. For example, it is assumed that the operation information associated with the welding image shown in FIG. 4 and the operation information at the previous time detect that the welder stops moving the electrode 101 to the right and moves it to the left, and stops the welding wire 40 (no operation). In this case, the data generating unit 213 adds a label to this welding image indicating that the operation of the electrode 101 in the left-right direction (right direction) is "excessive" and that the operation of the welding wire 40 in the left-right direction is "appropriate". Also, for example, when the operation of continuing to move the electrode 101 to the right is detected from the time series of the operation information, the data generating unit 213 adds a label indicating that the operation of the electrode 101 in the left-right direction (right direction) is "insufficient".

Next, the learning unit 214 constructs a learning model M through deep learning using learning data D1 consisting of welding images (learning images) and labels (correct answer data) (step S102).

For example, as shown in FIG. 4, a welding image of a welding point includes various elements such as the base material 3, the groove 31, the electrode 101, the welding wire 40, the molten pool 41, the bead 42, and the arc A. As described above, an experienced welder adjusts the amount of operation of the electrode 101 and the welding wire 40 by taking into account not only positional information such as the end position of the molten pool 41 and the tip positions of the electrode 101 and the welding wire 40, but also information that is difficult to quantify, such as the degree to which the molten pool 41 touches the boundary of the groove 31 (wettability of the groove) and the degree to which the molten pool 41 touches the welded portion (on the bead 42). In the welding device 1 according to this embodiment, the learning unit 214 learns the relationship between the feature amount of the welding image and the operation performed by the welder by looking at this welding image, thereby making it possible to incorporate information that is difficult to quantify into the learning model M. As a result, the learning unit 214 can build a learning model M that can make an estimate closer to the judgment of an experienced welder based on the welding image.

Next, the learning unit 214 evaluates the learning model M constructed in step S102 with the learning data D1 (step S103). The same data set as in step S102 is used for this learning data D1. Specifically, the learning unit 214 inputs only the welding images of the learning data D1 to the learning model M, and outputs the appropriateness of the operation amount for each welding image. The learning unit 214 performs a performance evaluation of the learning data D1 based on the output results of each welding image of the learning model M and the correct answer data.

Furthermore, the learning unit 214 evaluates the learning model M constructed in step S102 with the evaluation data D2 (step S104). This evaluation data D2 is a data set that has not been used in learning the learning model M. The learning unit 214 inputs only the welding images of the evaluation data D2 to the learning model M, and causes it to output the appropriateness of the operation amount for each welding image. The learning unit 214 performs a performance evaluation of the evaluation data D2 based on the output results of each welding image of the learning model M and the correct answer data. The learning unit 214 calculates, for example, an accuracy rate, a loss function, etc. as performance evaluation results for each of the learning data D1 and the evaluation data D2.

The learning unit 214 also links the learning model M, the performance evaluation results of the learning data D1, and the performance evaluation results of the evaluation data D2, and stores them in the storage 23 (step S105).

Next, the learning unit 214 determines whether the termination condition is satisfied (step S106). For example, the learning unit 214 determines that the termination condition is satisfied when the performance evaluation result of the learning data D1 and the performance evaluation result of the evaluation data D2 both converge to a predetermined value or higher (step S106; YES). In this case, the learning unit 214 adopts the learning model M with the highest performance evaluation result among the multiple learning models M constructed as the model to be used in the control mode (step S107).

On the other hand, if the termination condition is not met (step S106; NO), the learning unit 214 returns to step S102 and constructs a new learning model M.

(Control mode process flow)
FIG. 5 is a flowchart showing an example of a process in the control mode of the welding device according to the first embodiment of the present disclosure.
Here, the flow of processing by the welding device 1 in the control mode will be described with reference to FIG.

When the welding device 1 automatically performs welding of the base material 3, the camera 120 captures an image of the welding point of the base material 3. The acquisition unit 210 of the control device 20 acquires the welding image captured by the camera 120 (step S110).

Next, the estimation unit 211 estimates the appropriateness of the operation of the electrode 101 and the welding wire 40 at the current time (step S111). Specifically, the estimation unit 211 inputs the acquired welding image to the learning model M and obtains an output estimation result (information indicating either "excessive," "appropriate," or "insufficient").

Next, the control unit 212 generates control signals for controlling the electrode 101 and the welding wire 40 based on the estimation result of the estimation unit 211, and outputs the control signals to the welding torch 100 and the welding wire supply unit 110 (step S112). For example, as in the example of FIG. 4, if the operation of the electrode 101 in the right direction is "excessive", the control unit 212 generates a control signal to move the electrode 101 in the left direction and outputs the control signal to the welding torch 100. Also, as in the example of FIG. 4, if the operation of the welding wire 40 in the downward direction is "insufficient", the control unit 212 generates a control signal to move the electrode 101 in the downward direction and outputs the control signal to the welding torch 100. Furthermore, as in the example of FIG. 4, if the downward movement of the electrode 101 and the rightward movement of the welding wire 40 are "proper," the control unit 212 determines that the up-down movement of the electrode 101 and the left-right movement of the welding wire 40 are unnecessary, and does not need to output control signals related to these movements to the welding wire supply unit 110.

The welding device 1 performs welding automatically by repeatedly executing the series of steps shown in FIG. 5.

(Action, Effect)
As described above, the welding apparatus 1 according to this embodiment includes a camera 120 that captures a welding image including the electrode 101 and the tip of the welding wire 40 and the welding point of the base material 3, a learning unit 214 that constructs a learning model M using learning data including learning images that are welding images captured in the past and the operation of the electrode 101 and the welding wire 40 determined by the welder by looking at the learning images, an estimation unit 211 that uses the learning model M to estimate the appropriateness of the operation in the acquired welding image, and a control unit 212 that controls the electrode 101 and the welding wire 40 based on the estimation result of the estimation unit 211.

For example, even if the positional relationship of the electrode, welding wire, molten pool, etc. is the same, the welder may make different judgments as to whether the operation of the electrode and welding wire is appropriate, excessive, or insufficient, based on the combination of various conditions of the welding point obtained from the welding image (e.g., wettability of the groove, etc.). For this reason, if the electrode and welding wire are controlled based only on the position coordinates of each part, as in the conventional technology, control different from the welder's judgment may be performed, and the welding quality may be lower than if the welder were to weld manually. In contrast, as described above, the welding device 1 according to this embodiment learns the relationship between the feature amount of the learning image and the operation performed by the welder while looking at the state of the welding point in the learning image, and it is possible to incorporate not only the position information of each part but also information that the welder uses to judge welding and that is difficult to quantify into the learning model M. As a result, the welding device 1 in the control mode can estimate the appropriateness of the operation (e.g., whether the operation is "excessive", "appropriate", or "insufficient") at a level close to the welder's judgment, using the learning model M. Furthermore, the welding device 1 controls the electrode 101 and the welding wire 40 based on the estimation result to perform automatic welding, thereby achieving welding quality close to that achieved by a manual welder. In other words, the welding device 1 according to this embodiment can perform automatic welding with high welding quality compared to conventional technology that performs control based only on the position information of each part in the welding image.

Second Embodiment
Next, a welding device 1 according to a second embodiment of the present disclosure will be described with reference to FIG.
Components common to the first embodiment are denoted by the same reference numerals and detailed description thereof will be omitted.

FIG. 6 is a diagram illustrating an example of learning data according to the second embodiment of the present disclosure.
As shown in Figure 6, the data generation unit 213 in this embodiment generates learning data D1 in step S101 of Figure 3, which includes learning images in which areas of a welding image captured in the past that are not of interest to the welder are masked.

The welder judges whether the amount of manipulation of the electrode 101 and the welding wire 40 is appropriate by observing the fit of the molten pool 41 to the side of the groove 31 (change in the shape of the groove 31) and the way the molten pool 41 covers the bead 42. For example, as shown in FIG. 7, the welder judges the amount of manipulation by referring to the focus region R1 in which these features appear in the welding image. However, with deep learning, it is possible that the welder will learn easy-to-understand features contained in the welding image (for example, areas with large changes in brightness, such as around the arc A), and will not be able to fully learn the features of the area that the welder is actually focusing on (focus region R1).

For this reason, the data generating unit 213 according to this embodiment generates learning data D1 in which non-focused regions R2 of the welding image that the welder does not focus on are arbitrarily masked, as shown in FIG. 6. For example, the data generating unit 213 sets the region designated by the welder as the non-focused region R2 for each welding image. There may be one or more non-focused regions R2. For example, a region that includes the electrode 101 or the arc A with a large change in brightness, or a region that includes the groove shoulder (a portion that is not welded) is set as the non-focused region R2.

In addition, in step S102 of FIG. 3, the learning unit 214 according to this embodiment learns the learning model M based on the learning data D1 including the masked welding image (learning image).

In this way, even when the welding device 1 can only learn from welding images under specific conditions (the number of samples of the learning data D1 is small), it can effectively perform deep learning so that the learning model M can acquire common features in the region of interest R1. This also enables the welding device 1 to accurately estimate whether the operation of the electrode 101 and the welding wire 40 is appropriate or not, based on the features contained in the region of interest R1 in the welding image, during the control mode.

Third Embodiment
Next, a welding device 1 according to a third embodiment of the present disclosure will be described with reference to FIG.
Components common to the above-described embodiments are designated by the same reference numerals and detailed description thereof will be omitted.

FIG. 7 is a diagram illustrating an example of learning data according to the third embodiment of the present disclosure.
As shown in FIG. 7, the data generating unit 213 according to this embodiment generates learning data D1 including a learning image obtained by extracting the contour of a welding image captured in the past in step S101 of FIG.

As described in the second embodiment, deep learning may learn easily recognizable features contained in the welding image (e.g., areas with large changes in brightness, such as around arc A), and may not be able to fully learn the features of the area that the welder actually focuses on. For this reason, the data generation unit 213 in this embodiment generates learning data D1 by omitting information unnecessary for determining the operation from the welding image, in order to prevent learning of areas that the welder does not focus on.

Specifically, the data generating unit 213 performs a known edge detection process on the original welding image to detect and extract only the contours of each part included in the welding image to generate an edge image. The data generating unit 213 may further perform a process to remove information unnecessary for determining the operation from the edge image. In the example of FIG. 7, the data generating unit 213 performs a process to remove the mirror image of the electrode reflected on the side wall of the groove.

In addition, in step S102 of FIG. 3, the learning unit 214 according to this embodiment learns the learning model M based on the learning data D1 including a welding image (learning image) that is an edge image.

In this way, the welding device 1 can efficiently learn the learning model M by using an image that contains only the edges of each part, which is the minimum amount of information required, for the input image, omitting areas that the welder is not paying attention to. This allows the welding device 1 to accurately estimate whether the operation of the electrode 101 and the welding wire 40 is appropriate in the control mode. As a result, the welding device 1 can appropriately control the electrode 101 and the welding wire 40 based on the appropriateness of the operation.

Fourth Embodiment
Next, a welding device 1 according to a fourth embodiment of the present disclosure will be described with reference to FIGS.
Components common to the above-described embodiments are designated by the same reference numerals and detailed description thereof will be omitted.

FIG. 8 is a schematic diagram showing an overall configuration of a welding device according to a fourth embodiment of the present disclosure.
8, the welding device 1 according to this embodiment further includes a torque sensor 112 (first sensor) provided on the wire nozzle 111 of the welding wire supply unit 110. The torque sensor 112 detects a force generated in the wire nozzle 111 when the tip of the welding wire 40 comes into contact with the base material 3.

In this embodiment, in step S100 of FIG. 3, the acquisition unit 210 further acquires first sensor information from the torque sensor 112 that can detect the contact state of the welding wire 40 with the base material 3.

FIG. 9 is a diagram illustrating an example of learning data according to the fourth embodiment of the present disclosure.
In step S101 of FIG. 3, the data generating unit 213 detects the operation of the welding wire 40 based on the first sensor information, and specifies the appropriateness of the operation in the up-down direction (±Z direction) of the welding wire 40. The data generating unit 213 also generates learning data D1 (FIG. 9) by adding information (label) indicating the appropriateness of the operation in the up-down direction (±Z direction) of the welding wire 40 to the welding image acquired at the same time as the sensor information of the torque sensor 112. For example, when it is detected from the time series of the first sensor information of the torque sensor 112 that the welder has performed an operation of continuously moving the welding wire 40 downward, the data generating unit 213 adds a label indicating that the operation in the up-down direction (downward direction) of the welding wire 40 is "too small" to the welding image corresponding to this sensor information. In another embodiment, the welder may specify a label by looking at a combination of the welding image and the first sensor information. In this case, the data generating unit 213 generates learning data D1 by adding a label specified by the welder to the welding image.

In step S102 of FIG. 3, the learning unit 214 learns the learning model M based on the learning data D1 generated by the data generation unit 213.

As in the example of FIG. 8, when the camera 120 is positioned diagonally above and in front of the welding point (in the +Z and +X directions), the difference in the position of the welding wire 40 in the welding image is slight, especially when the welding wire 40 has low rigidity and is soft (easily melted). This may make it difficult to identify the vertical (±Z) position of the welding wire 40 from the welding image. In response to this, the welding device 1 according to the present embodiment is capable of constructing a learning model M that learns the appropriateness of the operation of the welding wire 40 in the vertical direction from the contact state of the welding wire 40 detected by the torque sensor 112.

In addition, in the control mode, the acquisition unit 210 acquires the first sensor information of the torque sensor 112 together with the welding image in step S110 of FIG. 5. In step S111 of FIG. 5, the estimation unit 211 inputs the welding image and the first sensor information into the learning model M and obtains the current degree of appropriateness of the operation of the electrode 101 and the welding wire 40 as an output. In addition, in step S112 of FIG. 5, the control unit 212 generates and outputs a control signal for controlling the electrode 101 and the welding wire 40 based on the degree of appropriateness of the operation of the electrode 101 and the welding wire 40. This allows the welding device 1 to accurately estimate whether the operation of the welding wire 40 in the up-down direction is appropriate in the control mode. Therefore, the welding device 1 can also appropriately control the up-down direction of the welding wire 40 to further improve the welding quality.

In the fourth embodiment, the data generating unit 213 generates the learning data D1 to which information (label) on the appropriateness of the operation of the welding wire 40 in the up-down direction is added based on the sensor information of the torque sensor 112, but the present invention is not limited to this. In other embodiments, the data generating unit 213 may directly include the first sensor information of the torque sensor 112 in the learning data D1. The learning unit 214 learns the appropriateness of the operation of the electrode 101 in the left-right direction and the appropriateness of the operation of the welding wire 40 in the left-right direction and the up-down direction based on the welding image, the label, and the first sensor information of the torque sensor 112. Even in this embodiment, the welding device 1 can accurately estimate whether the operation of the welding wire 40 in the up-down direction is appropriate.

Fifth embodiment
Next, a welding device 1 according to a fifth embodiment of the present disclosure will be described with reference to FIGS.
Components common to the above-described embodiments are designated by the same reference numerals and detailed description thereof will be omitted.

FIG. 10 is a schematic diagram showing an overall configuration of a welding device according to a fifth embodiment of the present disclosure.
As shown in Fig. 10, the welding device 1 according to this embodiment further includes a sensor 102 (second sensor) that is provided on the welding torch 100 and detects at least one of the position and posture of the welding torch 100. The sensor 102 is, for example, an acceleration sensor that can detect the posture (angle) of the welding torch 100. The sensor 102 is also an encoder that can detect the position and posture of the welding torch 100. The sensor 102 may also be a combination of an acceleration sensor and an encoder. The position and posture of the welding torch 100 are represented, for example, by a relative position and a relative angle with respect to a reference position (for example, an initial position and an initial angle when the welding device 1 is installed).

In this embodiment, in step S100 of FIG. 3, the acquisition unit 210 further acquires second sensor information from the sensor 102 that can detect the position or posture of the welding wire 40.

FIG. 11 is a diagram illustrating an example of learning data according to the fifth embodiment of the present disclosure.
The data generator 213 generates learning data D1 (FIG. 11) including the second sensor information of the sensor 102 in step S101 of FIG.

In step S102 of FIG. 3, the learning unit 214 learns the learning model M based on the learning data D1 generated by the data generation unit 213.

When the position or posture of the welding torch 100 changes during welding, the state of the welded area (e.g., the flow of the molten pool 41) changes. For this reason, in cases where welding is performed while changing the position and posture of the welding torch 100, if the learning model M is trained based only on the welding image, the accuracy of estimating the appropriateness of the operation may decrease. In contrast, the welding device 1 according to this embodiment can learn the learning model M that takes into account the position and posture of the welding torch 100 by using the learning data D1 that includes second sensor information that can detect the position and posture of the welding torch 100.

In addition, in the control mode, the acquisition unit 210 acquires the second sensor information of the sensor 102 together with the welding image in step S110 of FIG. 5. In step S111 of FIG. 5, the estimation unit 211 inputs the welding image and the second sensor information into the learning model M, and obtains as an output the degree of appropriateness of the operation of the electrode 101 and the welding wire 40 according to the position and posture of the welding device 1 at the time the welding image was captured. In addition, in step S112 of FIG. 5, the control unit 212 generates and outputs a control signal for controlling the electrode 101 and the welding wire 40 based on the degree of appropriateness of the operation of the electrode 101 and the welding wire 40.

With this configuration, the welding device 1 is able to accurately estimate the appropriateness of the operation of the electrode 101 and the welding wire 40, even when welding is performed while changing the position and posture of the welding torch 100. As a result, the welding device 1 can appropriately control the electrode 101 and the welding wire 40 in response to changes in the position and posture of the welding torch 100, thereby improving welding quality.

Sixth Embodiment
Next, a welding device 1 according to a sixth embodiment of the present disclosure will be described with reference to FIG.
Components common to the above-described embodiments are given the same reference numerals and detailed description thereof will be omitted.

If the welding image changes due to variations in the settings of the welding device 1 (such as the placement of the welding torch 100, welding wire supply unit 110, and camera 120) or the settings of the base material 3, it may not be possible to effectively train the learning model M. For this reason, in step S101 of FIG. 3, the data generation unit 213 according to this embodiment generates learning data D1 including learning images obtained by applying image processing to previously captured welding images that simulates variations in the relative positions of the welding device 1 and the base material 3.

FIG. 12 is a diagram for explaining functions of the control device according to the sixth embodiment of the present disclosure.
Specifically, as shown in FIG. 12, the data generating unit 213 generates a plurality of processed images by performing various image processing on the original welding image, and generates learning data D1 including each processed image.

For example, the data generating unit 213 generates a processed image by translating or rotating the welding image to generate learning data D1 that simulates the variation in the setting of the welding device 1 and the base material. The data generating unit 213 generates a processed image by enlarging or reducing the welding image to generate learning data D1 that simulates the variation in the angle of view due to focusing. The data generating unit 213 generates a processed image by changing the brightness of the welding image to generate learning data D1 that simulates the variation in brightness. The data generating unit 213 generates a processed image by adding Gaussian noise to the welding image to generate learning data D1 that simulates a case where the welding image is taken in a defocused state. The data generating unit 213 generates a processed image by changing the aspect ratio of the welding image to generate learning data D1 that simulates the variation in the distance between the electrode 101 and the welding wire 40.

In addition, in step S102 of FIG. 3, the learning unit 214 according to this embodiment learns the learning model M based on the learning data D1 including the original welding image and the learning data D1 including the processed image (learning image).

In this way, the welding device 1 can construct a learning model M that takes into account variability in settings, etc. This allows the welding device 1 to precisely control the electrode 101 and the welding wire 40 in the control mode without being affected by variability in settings, etc.

<Other embodiments>
FIG. 13 is a block diagram showing a functional configuration of a welding device according to another embodiment of the present disclosure.
In the above-described embodiments, the control device 20 has the data generating unit 213 that generates the learning data D1 and the evaluation data D2, but the present invention is not limited to this. For example, as shown in FIG. 13, the welding device 1 may include a data generating server 50 having the data generating unit 213 in addition to the control device 20. In this case, the learning data D1 and the evaluation data D2 are generated in the data generating unit 213 of the data generating server 50. The control device 20 also acquires the learning data D1 and the evaluation data D2 from the data generating server 50 via the communication interface 24 and stores them in the storage 23. Even with this configuration, the welding device 1 can obtain the same effects as those of the above-described embodiments.

As described above, several embodiments of the present disclosure have been described, but all of these embodiments are presented as examples and are not intended to limit the scope of the invention. These embodiments can be implemented in various other forms, and various omissions, substitutions, and modifications can be made without departing from the gist of the invention. These embodiments and their modifications are included in the scope of the invention and its equivalents as described in the claims, as well as in the scope and gist of the invention.

<Additional Notes>
The welding apparatus, the welding method, and the program described in the above-described embodiments can be understood, for example, as follows.

(1) According to a first aspect of the present disclosure, a welding device 1 that performs welding by controlling at least one of the control objects, the welding wire 40 and the electrode 101, includes a camera 120 that captures a welding image including the tip of the control object and the welding point, a data generation unit that generates learning data D1 including a learning image that is a welding image captured in the past and information indicating the appropriateness of the operation of the control object judged by a welder by looking at the learning image, a learning unit 214 that uses the learning data D1 to construct a learning model M that takes the welding image as input and outputs the appropriateness of the operation of the control object, an estimation unit 211 that uses the learning model M to estimate the appropriateness of the operation of the control object in the acquired welding image, and a control unit 212 that controls the control object based on the estimation result of the appropriateness of the operation.

By doing this, the welding device 1 learns the relationship between the feature amounts of the learning image and the operations that the welder performs while viewing the state of the welded point in the image when the learning image is captured, and is able to incorporate into the learning model M not only the positional information of each part in the welding image, but also information that the welder uses to make welding decisions and that is difficult to quantify. In this way, the welding device 1 can perform automatic welding with high welding quality in control mode by estimating the appropriateness of operations using the learning model M that has learned the decisions of experienced welders.

(2) According to the second aspect of the present disclosure, in the welding device 1 according to the first aspect, the data generator 213 generates learning data D1 including learning images in which areas of a previously captured welding image that are not of interest to the welder are masked.

In this way, even when the welding device 1 can only learn from welding images under specific conditions (the number of samples of the learning data D1 is small), it can effectively perform deep learning so that the learning model M can acquire common features in the region of interest R1. This also enables the welding device 1 to accurately estimate whether the operation of the electrode 101 and the welding wire 40 is appropriate or not, based on the features contained in the region of interest R1 in the welding image, during the control mode.

(3) According to the third aspect of the present disclosure, in the welding device 1 according to the first aspect, the data generator 213 generates learning data D1 including a learning image in which the contour of a previously captured welding image is extracted.

In this way, the welding device 1 can efficiently learn the learning model M by using an image that contains only the edges of each part, which is the minimum amount of information required, for the input image, omitting areas that the welder is not paying attention to. This allows the welding device 1 to accurately estimate whether the operation of the electrode 101 and the welding wire 40 is appropriate in the control mode. As a result, the welding device 1 can appropriately control the electrode 101 and the welding wire 40 based on the appropriateness of the operation.

(4) According to a fourth aspect of the present disclosure, in a welding device 1 according to any one of the first to third aspects, the data generator 213 generates learning data D1 including learning images obtained by applying image processing to previously captured welding images that simulates variations in the relative positions of the control object, the camera 120, and the welding point.

In this way, the welding device 1 can construct a learning model M that takes into account variability in settings, etc. This allows the welding device 1 to precisely control the electrode 101 and the welding wire 40 in the control mode without being affected by variability in settings, etc.

(5) According to a fifth aspect of the present disclosure, the welding device 1 according to any one of the first to fourth aspects further includes a first sensor 112 provided on a wire nozzle 111 that delivers the welding wire 40 and that outputs first sensor information capable of detecting the contact state of the welding wire 40 with the welding point, and the learning unit 214 constructs a learning model M using learning data D1 that further includes the operation of the welding wire 40 identified based on the first sensor information.

(6) According to a sixth aspect of the present disclosure, the welding device 1 according to any one of the first to fifth aspects further includes a second sensor 102 that is provided on the welding device 1 and outputs second sensor information capable of detecting at least one of the position and posture of the welding device 1, and the learning unit 214 constructs a learning model M using learning data D1 that further includes the second sensor information.

In this way, the welding device 1 can accurately estimate the appropriateness of the operation of the electrode 101 and the welding wire 40 even when welding is performed while changing the position and posture of the welding torch 100. As a result, the welding device 1 can appropriately control the electrode 101 and the welding wire 40 by following the changes in the position and posture of the welding torch 100.

(7) According to a seventh aspect of the present disclosure, a welding method for controlling at least one of the control objects, the welding wire 40 and the electrode 101, to perform welding includes the steps of: capturing a welding image including the tip of the control object and the welding location; generating learning data D1 including a learning image that is a previously captured welding image and information indicating the appropriateness of the operation of the control object judged by a welder by looking at the learning image; constructing a learning model M using the learning data D1 that takes the welding image as input and outputs the appropriateness of the operation of the control object; estimating the appropriateness of the operation of the control object in the acquired welding image using the learning model M; and controlling the control object based on the estimated result of the appropriateness of the operation.

(8) According to an eighth aspect of the present disclosure, the program causes a welding device 1 that performs welding by controlling at least one of the control objects, the welding wire 40 and the electrode 101, to execute the following steps: taking a welding image including the tip of the control object and the welding location; generating learning data D1 including a learning image that is a welding image taken in the past and information indicating the appropriateness of the operation of the control object judged by a welder by looking at the learning image; using the learning data D1, constructing a learning model M that takes the welding image as input and outputs the appropriateness of the operation of the control object; using the learning model M to estimate the appropriateness of the operation of the control object in the acquired welding image; and controlling the control object based on the estimated result of the appropriateness of the operation.

1 Welding device 10 Main body 100 Welding torch 101 Electrode 102 Sensor (second sensor)
110: Welding wire supply unit 111: Wire nozzle 112: Torque sensor (first sensor)
Reference Signs List 120 Camera 20 Control device 21 Processor 210 Acquisition unit 211 Estimation unit 212 Control unit 213 Data generation unit 214 Learning unit 22 Memory 23 Storage 24 Communication interface 3 Base material 40 Welding wire 50 Data generation server

Claims (5)

A welding device that performs welding by controlling at least one of a welding wire and an electrode,
A camera that captures a welding image including the tip of the control object and a welding point;
A data generating unit that generates learning data including learning images that are welding images captured in the past and information indicating the appropriateness of the operation of the control target judged by a welder by looking at the learning images;
a learning unit that uses the learning data to construct a learning model that receives the welding image as an input and outputs a degree of suitability of the operation of the control object;
an estimation unit that estimates a degree of appropriateness of an operation of the control target in the acquired welding image by using the learning model;
a control unit that controls the control target based on an estimation result of the appropriateness of the operation;
Equipped with
The data generation unit generates learning data including the learning image in which an area that is not of interest to the welder in the previously captured welding image is masked.
Welding equipment. The data generation unit generates learning data including the learning image obtained by extracting a contour of the welding image captured in the past.
2. The welding apparatus of claim 1. the data generation unit generates learning data including the learning image obtained by performing image processing on the previously captured welding image, the image processing simulating variations in the relative positions of the control object, the camera, and the welding point;
3. The welding device according to claim 1 or 2 . A welding method for performing welding by controlling at least one of a welding wire and an electrode,
Taking a welding image including the tip of the control object and a welding point;
A step of generating learning data including learning images which are welding images captured in the past and information indicating the appropriateness of the operation of the controlled object judged by a welder by looking at the learning images;
constructing a learning model using the learning data, the learning model inputting the welding image and outputting a degree of suitability of the operation of the control object;
estimating a degree of appropriateness of the operation of the control target in the acquired welding image by using the learning model;
controlling the control target based on a result of the estimation of the appropriateness of the operation;
It has
The step of generating the learning data generates the learning data including the learning image in which an area that the welder does not pay attention to in the welding image captured in the past is masked.
Welding method. A welding device for controlling at least one of a welding wire and an electrode to perform welding,
Taking a welding image including the tip of the control object and a welding point;
A step of generating learning data including learning images which are welding images captured in the past and information indicating the appropriateness of the operation of the control target judged by a welder by looking at the learning images;
constructing a learning model using the learning data, the learning model inputting the welding image and outputting a degree of suitability of the operation of the control object;
estimating a degree of appropriateness of the operation of the control target in the acquired welding image by using the learning model;
controlling the control target based on a result of the estimation of the appropriateness of the operation;
A program for executing
The step of generating the learning data generates the learning data including the learning image in which an area that the welder does not pay attention to in the welding image captured in the past is masked.
program .

Images