JP7150666B2 - Hume amount estimation system, Hume amount estimation method, learning device, estimation device, and program - Google Patents

Description

The present invention relates to a Hume amount estimation system, a Hume amount estimation method, a learning device, an estimation device, and a program.

The fumes generated by welding, which are metal fine particles floating in the air, are considered important from the viewpoint of occupational health. Clarification is required. In this standard, a welding fume collection device is installed, manual welding is performed in a welding fume collection box, the total amount of welding fumes generated in the box is collected with a filter material, and the mass is measured. It is prescribed to measure the amount of fume generation.

Internationally, ISO 15011-4 (Health and safety in welding and allied processes－Laboratory method for sampling fume and gases－Part 4: Fume data sheets) was enacted. Hume's data sheet format is also established.

JP 2011-503581 A

Although the above measuring method can accurately measure the amount of generated fume by collecting all the fume generated by welding, it cannot be implemented without dedicated equipment. In addition, the amount of fume generated cannot be measured in real time during welding, nor can it be used for feedback when an abnormality occurs.

Patent Document 1 discloses a particle detection method that irradiates a beam into a space where particles such as smoke exist and measures the scattered light. Measurement may be disturbed.

The present invention has been made in view of the above problems, and its main objects are a fume amount estimation system, a fume amount estimation method, a learning device, An object of the present invention is to provide an estimation device and a program.

In order to solve the above-described problems, a learning device according to one aspect of the present invention includes acquisition means for acquiring a synthesized image in which a Hume image and a background image are synthesized; Learning means for constructing a trained model for estimating information related to fumes generated by welding from an image of a welding state as training data by machine learning.

Further, a program according to another aspect of the present invention includes acquisition means for acquiring a composite image in which a Hume image and a background image are combined; The computer functions as learning means for constructing a learned model for estimating information related to fumes generated by welding from images obtained by photographing the state of welding by machine learning.

A fume amount estimation system according to another aspect of the present invention uses, as input data, a camera for photographing the state of welding and a composite image obtained by synthesizing a fume image and a background image. Estimating means for estimating information related to fume generated by welding from images captured by the camera using a learned model constructed in advance by learning, and outputting the amount of fume generated by welding based on the estimation result. and output means.

In another aspect of the fume amount estimation method of the present invention, a welding process is photographed by a camera, a synthesized image obtained by synthesizing a fume image and a background image is used as input data, and information related to fume is used as teacher data. Using a learned model constructed in advance by learning, information related to fume generated by welding is estimated from the image captured by the camera, and the amount of fume generated by welding is output based on the estimation result.

An estimating apparatus according to another aspect of the present invention includes an obtaining means for obtaining an image generated by a camera for photographing a state of welding; estimating means for estimating information related to fumes generated by welding from the image acquired by the acquiring means, using a learned model constructed in advance by machine learning using information related to welding as teacher data.

According to another aspect of the present invention, there is provided a program that uses, as input data, a composite image obtained by combining a Hume image and a background image; The computer functions as estimation means for estimating information related to fume generated by welding from the image acquired by the acquisition means, using a learned model constructed in advance by machine learning using information related to welding as teacher data.

According to the present invention, it becomes easy to measure the amount of fume generated even during welding.

1 is a block diagram showing an example of a Hume amount estimation system according to an embodiment; FIG. FIG. 4 is a diagram for explaining a learning phase; FIG. FIG. 10 is a flow diagram showing an example procedure of a learning phase; FIG. 4 is a diagram showing an example of a table for managing data sets; FIG. 4 is a diagram showing an example of a background image; FIG. FIG. 4 is a diagram showing an example of a Hume image; FIG. 10 is a diagram showing an example of a synthesized image; FIG. FIG. 10 is a diagram showing an example of a background image (model output); FIG. 10 is a diagram showing an example of a Hume image (model output); FIG. 4 is a diagram for explaining an inference phase; FIG. 10 is a flow diagram showing an example procedure of an inference phase; It is a figure which shows the example of a picked-up image.

BEST MODE FOR CARRYING OUT THE INVENTION Hereinafter, embodiments of the present invention will be described with reference to the drawings.

[system]
FIG. 1 is a block diagram showing an example of a Hume amount estimation system 100 according to an embodiment. A Hume amount estimation system 100 includes an estimation device 1 , a database 2 , a camera 3 , an output unit 4 and a learning device 5 . The fume amount estimation system 100 photographs the state of welding by the welding device 9 with the camera 3 , calculates the amount of fume H generated by welding, and outputs the amount to the output unit 4 .

The estimation device 1 includes a control unit 10 . The control unit 10 is a computer including a CPU, RAM, ROM, nonvolatile memory, input/output interface, and the like. The control unit 10 includes an acquisition unit 11 , an estimation unit 12 and a calculation unit 13 . These functional units are realized by the CPU of the control unit 10 executing information processing according to programs loaded from the ROM or nonvolatile memory to the RAM.

The learning device 5 includes a control section 50 . The control unit 50 is a computer including a CPU, RAM, ROM, nonvolatile memory, an input/output interface, and the like. The control unit 10 includes a synthesis unit 51 , an acquisition unit 52 and a learning unit 53 . These functional units are realized by the CPU of the control unit 10 executing information processing according to programs loaded from the ROM or nonvolatile memory to the RAM.

The program may be supplied via an information storage medium such as an optical disk or memory card, or may be supplied via a communication network such as the Internet or LAN.

The estimation device 1 and the learning device 5 can access the database 2 . The database 2 may be provided inside the estimation device 1 or the learning device 5, or may be provided outside and accessed via a communication network.

Part of the acquiring unit 11, the estimating unit 12, and the calculating unit 13 implemented by the control unit 10 of the estimating device 1 may be implemented by another device capable of communicating with the estimating device 1. FIG. Similarly, part of the synthesizing unit 51 , acquiring unit 52 and learning unit 53 implemented by the control unit 50 of the learning device 5 may be implemented by another device capable of communicating with the learning device 5 .

The output unit 4 is, for example, a display device such as a liquid crystal display. The output unit 4 is not limited to this, and may be an image forming device such as a printer.

[Learning phase]
The learning phase realized by the learning device 5 will be described below. FIG. 2 is a diagram for explaining the learning phase. FIG. 3 is a flow diagram showing an example procedure of the learning phase. FIG. 4 is a diagram showing an example of a table for managing datasets.

As shown in FIG. 2, in the present embodiment, machine learning is performed using a composite image as input data and a background image, a Hume image, a composite ratio α, and a feature amount β as teacher data. Of these, the Hume image or feature quantity β is an example of information related to Hume.

A background image is an image showing a background. FIG. 5 is a diagram showing an example of a background image. The background image is preferably an image of the welding device 9 (see FIG. 1), for example. The background image is not limited to this, and various images such as an image of another device or an image of a person can be applied.

A Hume image is an image showing Hume. FIG. 6 is a diagram showing an example of a Hume image. The fume image is preferably an image of fume actually generated by welding, for example. Also, the fume image is preferably an image obtained by photographing a fume against a monotonous background. For example, an image obtained by photographing fume when manual welding is performed in a welding fume collection box based on JIS Z 3930 is suitable as the fume image.

Further, the fume image may be an image simulating fume generated by welding. The fume-like image may be, for example, a photographed image of smoke or an artificially generated image.

A synthesized image is an image obtained by synthesizing a background image and a Hume image. FIG. 7 is a diagram showing an example of a synthesized image. The compositing ratio α is the compositing ratio between the background image and the hume image in the composite image, and is represented by a value of 0 or more and 1 or less, for example.

A feature amount β is a feature amount of the Hume image. Specifically, the feature amount β is a feature amount related to the amount of Hume shown in the Hume image, such as the average luminance or the total luminance of the Hume image.

The model used for machine learning is a network composed of all convolutional layers, and outputs multi-channel images of the same size as the input data. The first half of the network corresponds to the encoder, which generates a feature map by compressing the input data. The second half of the network corresponds to the decoder, which restores the feature map to its original size and outputs it.

In the example shown, the model outputs an 8-channel image. Of these, the 3-channel image (for example, the RGB image) becomes the background image, and the other 3-channel image becomes the Hume image. FIG. 8 is a diagram showing an example of a background image output from the model. FIG. 9 is a diagram showing an example of a Hume image output from the model.

Of the remaining two channels, the image of one channel becomes the composite ratio α by mean value pooling, and the image of the other one channel becomes the feature amount β by mean value pooling.

The learning device 5 implements the learning phase according to the procedures shown in FIGS. The control unit 50 of the learning device 5 functions as a synthesis unit 51, an acquisition unit 52, and a learning unit 53 by executing the information processing shown in FIG. 3 according to a program.

First, the control unit 50 synthesizes the background image and the hume image to generate a synthesized image (S11). The generated composite image is associated with a background image, a Hume image, a composite ratio α, and a feature amount β that serve as teacher data, and managed as a data set (see FIG. 4).

Next, the control unit 50 inputs the composite image into the model (S12), performs calculations, and outputs the background image, the Hume image, the composite ratio α, and the feature amount β (S13).

Next, the control unit 50 calculates the difference between the output data and the teacher data for each of the background image, the Hume image, the synthesis ratio α, and the feature amount β (S14), and performs learning so that the difference decreases ( S15).

The learning phase is repeated while checking the accuracy of the model using a validation set drawn from the dataset.

After the learning phase is over, the accuracy of the trained model is evaluated using a test set extracted from the dataset.

As described above, the trained model separates the input image into the background image and the Hume image and outputs them. be built.

Since the learning phase requires a large number of data sets, it is preferable to prepare a large number of background images and a large number of Hume images and combine them to create a large number of synthesized images. Also, it is preferable to create a large number of synthesized images using various synthesis ratios α for synthesizing the background image and the Hume image.

Various methods can be applied to create Hume images. For example, a fume image may be created by photographing fume or smoke with a stereo camera and separating the background portion and the fume portion. Alternatively, a fume image may be created by inverting an image of fume or smoke in a symmetrical background, taking a difference, and separating the background portion and the fume portion.

Alternatively, a fume image may be created by creating a two-dimensional map of the particle concentration of fume or smoke by a laser scattering method and coloring it according to the particle concentration. Alternatively, a Hume image may be created using an image generation model such as a generative adversarial network (GAN).

[Inference phase]
The inference phase realized by the estimation device 1 will be described below. FIG. 10 is a diagram for explaining the inference phase. FIG. 11 is a flow diagram showing an example procedure of the inference phase. The control unit 10 of the estimation device 1 functions as an acquisition unit 11, an estimation unit 12, and a calculation unit 13 by executing the information processing shown in FIG. 11 according to a program.

As shown in FIGS. 10 and 11, first, the control unit 10 acquires a captured image generated by the camera 3 that captures the state of welding by the welding device 9 (S21). FIG. 12 is a diagram showing an example of a captured image.

If arc light is included in the captured image, the estimation accuracy of the learned model may be degraded. Preferably, a suppressing filter is used.

Next, the control unit 10 inputs the captured image to the learned model constructed in the learning phase (S22), performs calculations, and outputs the background image, the Hume image, the synthesis ratio α, and the feature amount β (S23). . The Hume image or feature quantity β is an example of information related to Hume. It should be noted that output of the background image, composition ratio α, etc., which is not used for calculating the amount of fume in the next step, may be omitted.

Next, the control unit 10 calculates the amount of fume generated by welding based on the estimation result of the learned model (S24). The output unit 4 outputs the amount of fume calculated by the control unit 10 on the screen.

When a Hume image is used, the control unit 10 extracts the feature amount of the Hume image output from the trained model, and calculates the Hume amount based on the extracted feature amount. The feature amount to be extracted is, like the feature amount β, a feature amount related to the amount of Hume shown in the Hume image, such as the average luminance or the total luminance of the Hume image.

When using the feature amount β, the control unit 10 calculates the Hume amount based on the feature amount β output from the trained model.

The correspondence relationship between the feature amount and the Hume amount is put together, for example, in a lookup table and referred to by the control unit 10 . The amount of fume associated with the feature amount is preferably the amount of fume generation measured by the method specified in JIS Z 3930. That is, based on JIS Z 3930, the feature amount extracted from the image of the fume taken when manual welding is performed in the welding fume collection box, and the amount of fume generated by collecting and measuring the fume with a filter material. is preferably associated.

According to the embodiment described above, since the amount of fume is output based on the photographed image of the state of welding, it becomes easy to measure the amount of fume in real time during welding. Therefore, it is possible to use the amount of fume measured in real time for feedback or the like when an abnormality occurs.

Further, according to the embodiment, since the Hume amount is measured based on the Hume image separated from the captured image or the feature amount β thereof, it is possible to suppress the influence of the background included in the captured image on the measurement of the Hume amount. becomes possible. For this reason, the measurement of the amount of fume can be applied not only to manual welding inside a dedicated collection box, but also to manual welding outside the collection box, automatic welding by a welding device, and the like.

[Modification]
In the above embodiment, the Hume amount is calculated based on the Hume image or the feature amount β estimated by the trained model. Alternatively, the amount of fume generated by welding may be directly estimated.

The amount of fume used as training data is preferably the amount of generated fume measured by the method specified in JIS Z 3930. As a result, the amount of fume estimated by the trained model can be set to a value corresponding to the amount of fume generated according to JIS Z 3930.

Although the embodiments of the present invention have been described above, the present invention is not limited to the embodiments described above, and it goes without saying that various modifications can be made by those skilled in the art.

1 estimation device, 10 control unit, 11 acquisition unit, 12 estimation unit, 13 calculation unit, 2 database, 3 camera, 4 output unit, 5 learning device, 50 control unit, 51 synthesis unit, 52 acquisition unit, 53 learning unit, 9 welding equipment, 100 fume amount estimation system

Claims (19)

Acquisition means for acquiring a synthesized image obtained by synthesizing a Hume image and a background image;
Learning to construct a trained model for estimating information related to fumes generated by welding from an image of welding by machine learning, using the synthesized image as input data and learning information related to fumes as teacher data. means and
A learning device with further comprising synthesizing means for synthesizing the Hume image and the background image to generate the synthetic image;
A learning device according to claim 1. The learning information related to the Hume is the Hume image,
3. The learning device according to claim 1 or 2. the training data includes the Hume image and the background image;
4. A learning device according to any one of claims 1 to 3. The training data further includes a synthesis ratio of the Hume image and the background image in the synthesized image,
5. The learning device according to claim 4. The learning information related to the Hume is a feature amount of the Hume image,
3. The learning device according to claim 1 or 2. The learning information related to the Hume is the amount of Hume contained in the Hume image,
3. The learning device according to claim 1 or 2. The fume image is an image of fume generated by welding,
A learning device according to any one of claims 1 to 7. The fume image is an image simulating fume generated by welding,
A learning device according to any one of claims 1 to 7. Acquisition means for acquiring a synthesized image obtained by synthesizing the Hume image and the background image; and
Learning to construct a trained model for estimating information related to fumes generated by welding from an image of welding by machine learning, using the synthesized image as input data and learning information related to fumes as teacher data. means,
A program that allows a computer to function as a A camera that captures the state of welding,
A synthesized image obtained by synthesizing a Hume image and a background image is used as input data, and a trained model that is pre-constructed by machine learning is used with training information related to Hume as teacher data. an estimating means for estimating information related to the generated fume;
output means for outputting the amount of fume generated by welding based on the estimation result;
A Hume quantity estimation system comprising a The Hume information estimated by the estimation means is the Hume image.
The Hume amount estimation system according to claim 11 . The information related to the Hume estimated by the estimation means is a feature amount of the Hume image,
The Hume amount estimation system according to claim 11 . Further comprising calculating means for calculating the amount of fume generated by welding based on the information related to the fume estimated by the estimating means,
A Hume amount estimation system according to any one of claims 11 to 13. The information related to the fume estimated by the estimation means is the amount of fume generated by welding,
The Hume amount estimation system according to claim 11 . A shield that shields the arc light or a filter that suppresses the transmission of the arc light is used for photographing the state of welding with the camera,
The Hume amount estimation system according to any one of claims 11 to 15. A camera captures the welding process,
A synthesized image obtained by synthesizing a Hume image and a background image is used as input data, and a trained model that is pre-constructed by machine learning is used with training information related to Hume as teacher data. Estimating information related to the generated fumes,
Output the amount of fume generated by welding based on the estimation result,
Hume amount estimation method. Acquisition means for acquiring an image generated by a camera that captures the state of welding;
Welding is performed from the image obtained by the obtaining means using a trained model that has been constructed in advance by machine learning using a synthetic image obtained by synthesizing a Hume image and a background image as input data and learning information related to Hume as teacher data. Estimating means for estimating information related to Hume generated by
An estimating device comprising: Acquisition means for acquiring an image generated by a camera that captures the state of welding; and
Welding is performed from the image obtained by the obtaining means using a trained model that has been constructed in advance by machine learning using a synthetic image obtained by synthesizing a Hume image and a background image as input data and learning information related to Hume as teacher data. Estimation means for estimating information related to Hume generated by
A program that allows a computer to function as a

Images