JP7810089B2 - Steel material determination device, trained model generation method, steel material determination method, and steel material manufacturing method - Google Patents

Description

Embodiments of the present invention relate to a steel material determination device, a method for generating a steel type trained model, a steel material determination method, and a steel material manufacturing method.

For example, when manufacturing and shipping pipe-shaped steel products, in order to prevent the mixing of steel products of different properties (different materials) into the shipment, a spark test is conducted in which an operator checks for the presence of different materials by observing the sparks that are generated when a grinder comes into contact with the end of the steel product.

The spark test involves visually checking for sparks that are generated when the edge of the steel material is ground. This is because it is empirically known that the shape and quantity of sparks that fly, such as the number of spark explosions and the number of streamlines, change depending on the chemical composition of the steel material.

However, this spark test is a sensory inspection that depends on the operator's level of skill, and the skills must be passed on to ensure that the test can continue to be performed. Furthermore, some steel product suppliers may require that spark tests be performed on all steel products they deliver.

As a method for automatically conducting spark testing, for example, the technology described in Patent Document 1 below is disclosed. Patent Document 1 discloses a device that estimates the composition of steel material from image data using a trained model for estimating the composition of steel material.

Japanese Patent Application Laid-Open No. 2020-009435

However, in the device for estimating the composition of steel materials described in Patent Document 1, the image data on which the estimation is based is the data generated by the camera itself, and no image processing has been performed. Therefore, there is a possibility of disturbances such as differences in the brightness of lighting at the shooting location, background equipment being reflected in the image, or background equipment reflecting light from lighting or sparks, which may be mistakenly recognized as part of the sparks.

Using uncertain data due to such disturbances is not only inappropriate for use in estimating the composition of steel, but is also likely to result in reduced accuracy in judgments. Furthermore, even if a trained model is constructed using such data, the accuracy of the trained model itself is low, so the accuracy of the judgments will inevitably be low as well.

The present invention has been made with the above-mentioned points in mind, and aims to provide a steel material determination device, a method for generating a trained model, a steel material determination method, and a method for manufacturing steel material that can accurately determine the characteristics of steel material by performing image processing in advance on spark images used when generating a trained model used to determine the characteristics of steel material or when actually using the trained model to determine steel material, thereby generating and using a trained model that appropriately eliminates disturbances.

The steel material determination device in an embodiment of the present invention comprises a determination unit that performs image processing to separate sparks from other components in the image data using a gradation-based threshold value that is set in advance for each color, on a color that is determined for each pixel of image data that includes sparks generated when grinding steel material obtained by an imaging device, and determines the steel material indicated by the spark image by using the spark image after image processing as inference data and applying it to a trained model that is prepared in advance.

The steel material determination device according to an embodiment of the present invention further includes a learning unit that uses the processed spark image as input data and the characteristics of the steel material as training data to generate a trained model through machine learning that determines the steel material indicated by the spark image.

A method for generating a trained model in an embodiment of the present invention includes the steps of: acquiring image data including sparks generated when grinding steel material acquired by an imaging device; performing image processing on the image data to separate sparks from other components using a gradation-based threshold value that is set in advance for each color for a color determined for each pixel of the image data, thereby obtaining a spark image after the image processing; and using the spark image after the image processing as input data and the characteristics of the steel material as training data, generating a trained model by machine learning that determines the steel material indicated by the spark image.

In the method for generating a trained model according to an embodiment of the present invention, the step of performing image processing involves extracting only sparks from image data and generating a spark image after image processing.

A steel material determination method in an embodiment of the present invention includes the steps of: acquiring image data including sparks generated when grinding steel material acquired by an imaging device; performing image processing on the image data using a gradation-based threshold value that is set in advance for each color for a color determined for each pixel of the image data to separate the sparks from other components in the image data, thereby obtaining a spark image after the image processing; and using the spark image after the image processing as input data and the characteristics of the steel material as training data, and using a trained model generated by machine learning to determine the steel material indicated by the spark image.

The step of performing image processing in the steel material determination method according to an embodiment of the present invention involves extracting only sparks from the image data through image processing to produce a spark image after image processing.

A method for manufacturing steel material in an embodiment of the present invention includes a steel material determination method that includes the steps of: acquiring image data including sparks generated when grinding steel material acquired by an imaging device; performing image processing on the image data to separate sparks from other components using a gradation-based threshold value that is set in advance for each color for a color that is determined for each pixel of the image data, thereby obtaining a spark image after the image processing; and using the spark image after the image processing as input data and the characteristics of the steel material as training data, and using a trained model generated by machine learning to determine the steel material indicated by the spark image.

With the steel material determination device, trained model generation method, steel material determination method, and steel material manufacturing method according to the embodiments of the present invention, by performing image processing in advance on the spark images used when generating the trained model used to determine the characteristics of steel material or when actually using the trained model to determine steel material, it is possible to generate and use a trained model that appropriately eliminates disturbances, thereby enabling the characteristics of steel material to be determined with high accuracy.

1 is an explanatory diagram showing an entire apparatus for discriminating steel materials from the characteristics of the steel materials, including a steel material determination device according to an embodiment of the present invention; 2 is a block diagram showing the internal configuration of a control device provided in the steel material determination device according to the embodiment of the present invention. FIG. 1 is a spark image used in an embodiment of the present invention, showing the spark image before image processing in the image processing unit. 10 is a spark image used in an embodiment of the present invention, showing the spark image after image processing in an image processing unit. 1 is a flowchart showing the flow of machine learning to generate a trained model used by a steel material determination device to determine steel material in an embodiment of the present invention. 3 is a flowchart showing an inference flow for determining a steel material by the steel material determination device according to the embodiment of the present invention.

Embodiments of the present invention will now be described in detail with reference to the drawings. Figure 1 is an explanatory diagram showing the entire apparatus for identifying steel material S based on the characteristics of the steel material S, including a steel material identification device 1 according to an embodiment of the present invention.

Identification of steel material S is performed based on spark images obtained by capturing with an imaging device C the shape of sparks F generated when the end of steel material S is ground with a grinder G. As mentioned above, the shape of the sparks changes depending on the component composition; for example, the number of spark explosions and the number of streamlines increases as the carbon content increases. Therefore, it is possible to identify steel material S by looking at the shape of the sparks.

The location where the steel material S is ground by the grinder G, i.e., the location where the spark image is captured by the imaging device C, may be, for example, in the middle of the production line, or may be a location specifically set up for identifying the steel material S.

The imaging device C is, for example, a camera that captures spark images. The imaging device C used in the embodiment of the present invention is configured to be able to capture monochrome images or color images. Therefore, the image data transmitted to the control device 2 of the steel product identification device 1, which will be described later, is either a monochrome image or a color image of the sparks. It is possible to arbitrarily select whether to use a monochrome image or a color image. Furthermore, although not shown, auxiliary equipment, such as a strobe, may be combined to capture spark images more clearly _.

In Figure 1, the imaging device C is located below the grinder G, in a position that allows it to capture images of sparks F from below, but it may be located in any position as long as it can capture images of sparks that are sufficient to be used to determine the quality of the steel material S.

The data of the spark images captured by the imaging device C is transmitted to the steel material determination device 1 connected to the imaging device C. The steel material determination device 1 controls the imaging device C, and uses the spark image data acquired by the imaging device C to generate a trained model for identifying steel material S, and then uses the generated trained model to identify steel material S.

Figure 2 is a block diagram showing the internal configuration of the control device 2 provided in the steel material determination device 1 in an embodiment of the present invention. Note that the control device 2 shown in Figure 2 only shows functions related to generating a trained model used when determining the steel material S, as described below, and determining the steel material S, and various other functions are not shown.

The control device 2 may therefore have various configurations not shown in Figure 2, such as a CPU (Central Processing Unit), ROM (Read Only Memory), RAM (Random Access Memory), and an input/output interface connected via a bus.

That is, the input/output interface may be connected to, for example, an input unit for inputting various parameters required to generate a trained model used to identify steel material S, a display unit that shows the operator the results of the judgment, such as the steel type of the identified steel material S, or a communication control unit that controls communication with other devices.

The control device 2 includes an information acquisition unit 21, an image processing unit 22, a learning unit 23, a memory unit 24, a judgment unit 25, and a judgment result notification unit 26. The information acquisition unit 21 acquires data on spark images generated when grinding steel material S from the imaging device C.

The spark image data may be acquired directly from the imaging device C, or may be temporarily stored in a database (not shown in Figure 2) and then acquired at a preset timing.

The image processing unit 22 performs image processing on the spark image data acquired via the information acquisition unit 21. The reason for performing image processing on the spark image data here is to remove any disturbances that may be contained within the data that are unnecessary for identifying the steel material S. If the spark image contains disturbances, this will reduce the accuracy of the trained model generated using the spark image data as input data, and will also reduce the accuracy of steel material identification when an identification process is actually performed using the trained model. Therefore, it is necessary to perform image processing on the spark image data to remove the disturbances.

Specifically, the image processing unit 22 can perform processing using a variety of methods, but in an embodiment of the present invention, for example, the following processing is performed. First, binarization processing is performed based on the acquired spark image data. Binarization processing is an image processing method that converts the target image into only two colors, white and black, for each pixel, using a certain threshold as a reference. By performing this processing, it is possible to separate the target spark from the rest of the background and eliminate background disturbances.

In image data, a color is determined for each pixel. If the spark image data is grayscale, each pixel contains only one type of information, luminosity, and is therefore represented, for example, in 256 shades from white to black. On the other hand, as mentioned above, if imaging device C captures a color image, each pixel can be represented in 256 shades of RGB.

Therefore, when processing an image in which the acquired image is a color image, for example, upper and lower thresholds are set in advance for each RGB of each pixel. Pixels that fall within the threshold range are then marked white, and pixels that fall outside the threshold range are marked black.

For example, when we look at the color of sparks, they are mainly represented in colors such as red (R: 255, G: 0, B: 0), orange (R: 255, G: 60-170, B: 0), and yellow (R: 255, G: 255, B: 0). On the other hand, reflected light, which may be reflected in things other than sparks, is often white (R: 255, G: 255, B: 255).

So, for example, the upper threshold for "B" among RGB is changed to set the threshold so that white or near-white colors are not extracted. By setting it in this way, it is possible to eliminate disturbances other than sparks, such as background equipment, that may be reflected in the spark image data, making it easier to extract only sparks from the resulting spark image data.

Figure 3 shows the spark image BF before image processing in the image processing unit 22, and Figure 4 shows the spark image AF after image processing, regarding the spark image used in an embodiment of the present invention.

In the spark image BF shown in Figure 3, in addition to the spark F, background equipment B is also visible at the bottom of the image. By performing the image processing described above on this spark image BF in the image processing unit 22, the spark image AF shown in Figure 4 is obtained.

As shown in spark image AF in Figure 4, the lower area of spark image BF, where background equipment B was shown, is shown in black, and background equipment B is not shown. Therefore, only the spark image has been extracted.

Note that, as mentioned above, there is an image processing method that extracts only sparks by performing binarization processing in the image processing unit 22, but as mentioned above, the image processing methods that can be used are not limited to this method.

For example, when using a threshold to distinguish between sparks and non-sparks, it is possible to leave the extracted spark image in its original color, and change the non-spark parts to a color that is distinguishable from sparks, such as black. In other words, particularly when the image captured by the imaging device C is a color image, it is possible to leave the color as is, rather than performing processing such as binarization, which would display the spark parts in white, and perform color conversion processing only on the non-spark disturbance parts. In this case, the color of the sparks is also trained into the trained model as data indicating the characteristics of the steel material S, making it possible to generate a trained model with higher accuracy.

By performing this type of processing, it is possible to extract only the spark portion with the same degree of accuracy as when performing binarization processing, and the burden on the image processing unit 22 when performing image processing is reduced, allowing for faster processing.

The learning unit 23 generates a trained model by machine learning to be used when distinguishing the steel material S. That is, when generating the trained model, the learning unit 23 uses, as input data, the spark image data acquired by the information acquisition unit 21 and after image processing by the image processing unit 22.

On the other hand, training data is data that indicates the characteristics of the steel material S to be identified. Examples of data that indicates the characteristics of the steel material S that serve as training data include the steel type (standard) and components of the steel material S. Furthermore, examples of components that indicate the characteristics of the steel material S include the carbon content and alloy elements.

The training data may be obtained from the memory unit 24 of the steel material determination device 1, which will be described later, or from a database or the like to which the steel material determination device 1 is separately connected.

The learning unit 23 uses the spark image data after image processing as input data and data indicating the characteristics of the steel material S as training data to generate a trained model through machine learning. A machine learning algorithm is then used to generate the trained model. Examples of machine learning algorithms that can be used include neural networks, decision tree learning, random forests, and support vector regression.

The trained model generated (constructed) by the learning unit 23 is then stored, for example, in the memory unit 24 and used when performing the steel material S discrimination process in the judgment unit 25, which will be described next.

Here, the memory unit 24 is composed of, for example, a semiconductor or a magnetic disk, and stores the trained model generated above, programs for controlling the steel material determination device 1, etc.

As shown in Figure 3, the control device 2 in this embodiment of the present invention is described assuming that it has a memory unit 24 installed inside it. However, the memory unit 24 may not be installed inside the control device 2, and for example, an external database connected to the steel material determination device 1 may perform its function, and the configuration can be freely selected.

The determination unit 25 uses the trained model to identify the steel material S. That is, the spark image data acquired from the imaging device C is used as inference data, and inference is performed using the trained model stored in the memory unit 24. Note that the spark image used as inference data is the spark image data after image processing by the image processing unit 22.

When inference is performed in the judgment unit 25, a judgment is made for each steel material S regarding characteristics such as the steel type (standard) and components of the steel material S. The judgment results are sent to the judgment result notification unit 26. The judgment result notification unit 26 notifies the operator of the results of the judgment made by the judgment unit 25, for example, via a display unit, etc.

[Operation]
Next, the flow of generating a trained model in the control device 2 and the flow of determining the steel material S using the generated trained model will be described with reference to Figures 5 and 6.

First, let us look at the flow for generating a trained model. Figure 5 is a flowchart showing the flow of machine learning used to generate a trained model used by the steel material determination device 1 when determining steel material S in an embodiment of the present invention.

The information acquisition unit 21 of the control device 2 acquires data on the spark image captured by the imaging device C, which is generated when the grinder G comes into contact with the end of the steel material S (ST1). The acquired spark image data includes not only the spark image but also data on disturbances such as background equipment.

Such spark image data is sent from the information acquisition unit 21 to the image processing unit 22, where image processing is performed (ST2). The image processing unit 22 performs image processing such as binarization, as described above, to remove disturbances and generate an image containing only the spark image. The spark image after image processing is used as input data in the learning unit 23 when a trained model is generated.

The information acquisition unit 21 also acquires data indicating the characteristics of the steel material S, which serves as training data (ST3). As described above, this training data may be acquired, for example, from a database (not shown) connected to the control device 2.

The control device 2 now has the data necessary to generate a trained model. The learning unit 23 then performs machine learning using the processed spark image as input data and data indicating the characteristics of the steel material S as training data (ST4). The model obtained as a result of the machine learning is then stored in the memory unit 24 as a trained model (ST5).

This concludes the learning stage in the control device 2. Next, we will explain the flow of the inference process (determination process) for steel material S using the generated trained model. Figure 6 is a flowchart showing the inference flow for determining steel material by the steel material determination device 1 in an embodiment of the present invention.

The control device 2 first acquires, via the information acquisition unit 21, data that will form the basis of the inference data used to determine the steel material S (ST11). This data is the spark image data acquired by the imaging device C itself, and has not been subjected to any image processing.

Next, the information acquisition unit 21 sends the acquired spark image data to the image processing unit 22. The image processing unit 22 performs image processing on the spark image data (ST12). The processed spark image in the image processing unit 22 is used as inference data.

The determination unit 25 uses the spark image data after image processing as inference data and performs a determination process on the steel material S that is the subject of the spark image using the trained model acquired by accessing the memory unit 24 (ST13, ST14). The determination result is transmitted from the determination unit 25 to the determination result notification unit 26, and is notified, for example, to an operator (ST15).

As explained above, in both the process of generating a trained model and the process of determining steel using that trained model, the spark image data acquired by imaging device C is not used as is, but is image processed to remove any information that could be a disturbance, leaving only the spark image data. By using such image-processed spark images as input data for machine learning or as inference data for inference processing, it is possible to generate and use a trained model that appropriately removes disturbances, thereby enabling the characteristics of steel to be determined with high accuracy.

The embodiments of the present invention described so far are merely examples of the present invention. Various modifications and improvements can be made to these embodiments, and such modifications and improvements can also be included in the present invention. These embodiments and their variations are included within the scope and spirit of the invention, and are also included in the scope of the inventions and their equivalents set forth in the claims.

REFERENCE SIGNS LIST 1 Steel material determination device 2 Control device 21 Information acquisition unit 22 Image processing unit 23 Learning unit 24 Memory unit 25 Determination unit 26 Determination result notification unit C Imaging device G Grinder S Steel material

Claims (9)

an image processing unit that performs image processing for separating the sparks from other components in the image data by using a threshold value based on gradation that is set in advance for each color, the threshold value being determined for each pixel of image data including sparks generated when grinding steel material and that is acquired by an imaging device, and extracts a spark image after image processing;
a determination unit that determines the steel material indicated by the spark image by using the spark image after image processing by the image processing unit as inference data and applying it to a trained model prepared in advance;
A steel material determination device equipped with: a learning unit that uses the spark image after image processing as input data and the characteristics of the steel material as training data to generate a trained model for determining the steel material indicated by the spark image by machine learning;
The steel material determination device according to claim 1, further comprising: The steel product identification device described in claim 2, characterized in that the learning unit performs machine learning using at least one machine learning algorithm from among neural networks, decision tree learning, random forests, and support vector regression. acquiring image data including sparks generated when grinding the steel material, the image data being acquired by an imaging device;
a step of executing image processing on the image data to separate the sparks from other components in the image data using a threshold value based on gradation that is set in advance for each color determined for each pixel of the image data, and acquiring a spark image after image processing;
a step of generating a trained model by machine learning that determines the steel material indicated by the spark image using the spark image after image processing as input data and the characteristics of the steel material as training data;
A method for generating a trained model, comprising: The step of performing image processing includes:
The method for generating a trained model according to claim 4, characterized in that only sparks are extracted from the image data by image processing to obtain the spark image after image processing. The method for generating a trained model described in claim 4 or claim 5, characterized in that the machine learning uses at least one machine learning algorithm from among neural networks, decision tree learning, random forests, and support vector regression. acquiring image data including sparks generated when grinding the steel material, the image data being acquired by an imaging device;
a step of executing image processing on the image data to separate the sparks from other components in the image data using a threshold value based on gradation that is set in advance for each color determined for each pixel of the image data, and acquiring a spark image after image processing;
a step of determining the steel material indicated by the spark image using a trained model generated by machine learning, with the spark image after image processing as input data and the characteristics of the steel material as training data;
A steel material determination method comprising: The step of performing image processing includes:
8. The steel material determination method according to claim 7, wherein only sparks are extracted from the image data by image processing, and the spark image after image processing is used. A steel manufacturing method comprising the steel product determination method described in claim 7 or claim 8.