JP7540235B2 - METHOD FOR MANUFACTURING LEARNING MODEL USED IN QUALITY ASSESSMENT, LEARNING SYSTEM, QUALITY ASSESSMENT SYSTEM, AND QUALITY ASSESSMENT METHOD - Google Patents

Description

The present invention relates to a method for manufacturing a learning model used for quality evaluation of components made of heat-treated steel, a learning system for the manufacturing of the model, a quality evaluation system using the learning model, and a quality evaluation method.

Heat treatments such as carburizing and quenching are carried out on components made of carbon steel or alloy steel in order to improve their wear resistance and fatigue resistance. Factors that have a significant effect on the quality of components made of heat-treated steel include carbon concentration, amount of retained austenite, and hardness. Therefore, the quality of heat-treated components is sometimes evaluated based on physical property values such as carbon concentration, amount of retained austenite, and hardness.

For example, X-ray analysis is used to measure the amount of retained austenite. Patent Document 1 discloses a technology for determining the amount of retained austenite in steel materials from the results of X-ray analysis, although this is for the purpose of life assessment.

JP 2018-040770 A

In the quality evaluation of steel members that have been heat-treated by carburizing and quenching, the carbon concentration in the carburized layer is the most important factor and has a large effect on the accuracy of the quality evaluation.
The carbon concentration may be measured using an electron probe microanalyzer (EPMA).
However, in measuring the carbon concentration using EPMA, the specimen needs to be placed in a vacuum chamber, which takes a relatively long time to measure, and also requires high equipment costs.
This has had an impact on the time and cost required to carry out quality evaluations on components made of heat-treated steel.

The present invention aims to provide a new technical means that enables highly accurate quality evaluation of heat-treated products at low cost and in a short time.

(1) The present disclosure relates to a method for manufacturing a learning model used for evaluating the quality of a component made of heat-treated carbon steel or alloy steel, comprising:
an image acquisition step of acquiring a microscopic image of a target region at a predetermined depth from a surface of a target component made of heat-treated carbon steel or alloy steel and having a carbon concentration actually measured;
an image processing step of generating a bright side image in which the grains of the tissue of the target material are shown as bright objects and other areas are shown as dark areas from a grayscale image based on the microscope image, and a dark side image in which the grains of the tissue of the target material are shown as dark objects and other areas are shown as bright areas;
a feature amount calculation step of calculating bright side feature amounts of the plurality of objects included in the bright side image and dark side feature amounts of the plurality of objects included in the dark side image;
a model generating step of generating a learning model that outputs an estimated value of the carbon concentration from the bright side feature amount and the dark side feature amount of the component to be evaluated, by using the bright side feature amount and the dark side feature amount and an actual measured value of the carbon concentration of the target component as teacher data;
Including,
The bright-side feature amount and the dark-side feature amount calculated in the feature amount calculation step include at least one of a circularity of an object included in the bright-side image and a representative value of the brightness obtained from a frequency distribution of the brightness of pixels constituting the object included in the dark-side image.

In a microscopic image of heat-treated (hardened) steel, when the carbon concentration is relatively low, many of the relatively bright areas are needle-shaped, and as the carbon concentration increases, the needle-shaped areas in the relatively bright areas decrease and the lumpy areas increase. In other words, it can be said that the shape of the relatively bright areas (objects included in the bright-side image) in a microscopic image of hardened steel correlates with the carbon concentration.
Furthermore, in a microscopic image of heat-treated (hardened) steel, the relatively dark areas become darker when the carbon concentration is high compared to when the carbon concentration is low. In other words, it can be said that the brightness of the relatively dark areas (objects included in the dark side image) in a microscopic image of hardened steel correlates with the carbon concentration.
In this regard, in the present invention, the bright side feature and the dark side feature calculated in the feature calculation process include at least one of the circularity of the object included in the bright side image and a representative value of the brightness obtained from a frequency distribution of the brightness of the pixels constituting the object included in the dark side image, so that it is possible to obtain a learning model in which at least one of these is used as a feature.
By using this learning model, it is possible to improve the accuracy of estimating the carbon concentration of a target component.
If an estimated value of carbon concentration is obtained using a learning model obtained by this manufacturing method, it is not necessary to use an apparatus such as an EPMA, which makes it possible to reduce costs and measurement time.
As a result, quality evaluation of the heat-treated product can be performed with high accuracy at low cost and in a short time.

(2) In the above-mentioned method for manufacturing a learning model, the representative value may be a median.
In this case, the change in brightness caused by the difference in carbon concentration can be appropriately represented, and a more preferable learning model can be generated.

(3) The present disclosure also provides a learning system for producing a learning model used for quality evaluation of a member made of heat-treated carbon steel or alloy steel by machine learning, comprising:
an image acquisition device for acquiring a microscopic image of a target region at a predetermined depth from a surface of a target component made of heat-treated carbon steel or alloy steel and having a carbon concentration actually measured;
an image processing unit that generates, from a grayscale image based on the microscope image, a bright side image in which the grains of the tissue of the target material are shown as bright objects and other areas are shown as dark regions, and a dark side image in which the grains of the tissue of the target material are shown as dark objects and other areas are shown as bright regions;
a feature amount calculation unit that calculates bright side feature amounts of the plurality of objects included in the bright side image and dark side feature amounts of the plurality of objects included in the dark side image;
a learning unit that uses the bright side feature amount and the dark side feature amount and an actual measured value of the carbon concentration of the target component as teacher data to train a learning model so as to output an estimated value of the carbon concentration from the bright side feature amount and the dark side feature amount of the component to be evaluated;
Including,
The bright-side feature and the dark-side feature calculated by the feature calculation unit include at least one of a circularity of an object included in the bright-side image and a representative value of the brightness obtained from a frequency distribution of the brightness of a plurality of pixels constituting the object included in the dark-side image.

According to this learning system, the bright side features and dark side features calculated by the feature calculation unit include at least one of the circularity of an object included in the bright side image and a representative value of the brightness obtained from a frequency distribution of the brightness of the pixels constituting the object included in the dark side image, so that it is possible to obtain a learning model that uses at least one of these features as a feature.
By using this learning model, it is possible to improve the accuracy of estimating the carbon concentration of a target component.
Furthermore, if the carbon concentration is estimated using this learning model, there is no need to use an apparatus such as an EPMA, which reduces costs and shortens the measurement time.
As a result, quality evaluation of the heat-treated product can be performed with high accuracy at low cost and in a short time.

(4) The present disclosure also provides a quality evaluation system for evaluating the quality of an evaluation target component made of heat-treated carbon steel or alloy steel, comprising:
an image acquisition device for acquiring a microscopic image of a target region at a predetermined depth from a surface of the evaluation target component;
an image processing unit that generates, from a grayscale image based on the microscope image, a bright side image in which grains of the texture of the evaluation target component are shown as bright objects and other areas are shown as dark regions, and a dark side image in which grains of the texture of the evaluation target component are shown as dark objects and other areas are shown as bright regions;
a feature amount calculation unit that calculates bright side feature amounts of the plurality of objects included in the bright side image and dark side feature amounts of the plurality of objects included in the dark side image;
an evaluation calculation unit that uses the bright side feature amount and the dark side feature amount as a data set, and outputs an estimated value of the carbon concentration of the evaluation target component by a learning model obtained by learning with the learning system described in (3) above;
Equipped with.

By using the learning model obtained by learning using the learning system in (3) above, it is possible to improve the accuracy of estimating the carbon concentration of the target component. In addition, if the estimated value of the carbon concentration is obtained using this learning model, it is not necessary to use an apparatus such as an EPMA, which makes it possible to reduce costs and measurement time.
As a result, quality evaluation of the heat-treated product can be performed with high accuracy at low cost and in a short time.

(5) A quality evaluation method for evaluating the quality of an evaluation target component made of heat-treated carbon steel or alloy steel, comprising the steps of:
an image acquisition step of acquiring a microscope image of a target region at a predetermined depth from a surface of the evaluation target component;
an image processing step of generating, from a grayscale image based on the microscope image, a bright side image in which the grains of the texture of the evaluation target component are shown as bright objects and other areas are shown as dark areas, and a dark side image in which the grains of the texture of the evaluation target component are shown as dark objects and other areas are shown as bright areas;
a feature amount calculation step of calculating bright side feature amounts of the plurality of objects included in the bright side image and dark side feature amounts of the plurality of objects included in the dark side image;
an evaluation calculation step of outputting an estimated value of the carbon concentration of the evaluation target component by a learning model generated by the manufacturing method described in (1) or (2) above, using the bright side feature amount and the dark side feature amount as a data set;
Includes.

By using the learning model generated by the manufacturing method (1) or (2) above, it is possible to improve the accuracy of estimating the carbon concentration of the target component. In addition, if the estimated value of the carbon concentration is obtained using this learning model, it is not necessary to use an apparatus such as an EPMA, so that it is possible to reduce costs and shorten the measurement time.
As a result, quality evaluation of the heat-treated product can be performed with high accuracy at low cost and in a short time.

The disclosed invention makes it possible to perform quality evaluation of heat-treated products with high accuracy at low cost and in a short time.

1 is a flowchart illustrating an example of a method for producing a learning model of the present invention. FIG. 1 is a block diagram showing a configuration of a learning system. FIG. 2 is a cross-sectional view of a raceway of a rolling bearing. 1 is an example of a microscope image of each of the target members. FIG. 11 is an explanatory diagram of a process for joining two partition images into one. FIG. 13 is an explanatory diagram of a microscope image in which section images are stitched together. FIG. 2 is an explanatory diagram of a microscope image of a target area. FIG. 11 is an explanatory diagram of a noise removal process. FIG. 13 is an explanatory diagram of an image before a histogram equalization process. FIG. 13 is an explanatory diagram after a histogram equalization process. FIG. 2 is an explanatory diagram of a dark side image and a bright side image. FIG. 1 is an illustration of a histogram generated from a grayscale image. FIG. 2 is an explanatory diagram showing one of the dark objects included in the dark side image. FIG. 11 is an explanatory diagram showing one of the bright objects included in the bright side image. FIG. 15( a ) is a diagram showing a schematic diagram of the metal structure of a target component after heat treatment in which the carbon concentration is 0.2%, and FIG. 15 ( b ) is a diagram showing a schematic diagram of the metal structure of a target component after heat treatment in which the carbon concentration is 0.9%. FIG. 2A is a diagram showing an example of a bright side image of a target member having a measured carbon concentration of 0.2%, and FIG. 2B is a partially enlarged view of FIG. FIG. 2A is a diagram showing an example of a bright side image of a target member having a measured carbon concentration of 0.9%, and FIG. 2B is a partially enlarged view of FIG. FIG. 13 is a diagram showing an example of a histogram for determining a median value of brightness of an object included in a dark side image. FIG. 2A is a diagram showing an example of a dark side image of a target member having a measured carbon concentration of 0.2%, and FIG. 2B is a partially enlarged view of FIG. FIG. 2A is a diagram showing an example of a dark side image of a target member having a measured carbon concentration of 0.9%, and FIG. 2B is a partially enlarged view of FIG. 13 is a graph showing estimated values output by a learning model and actual measured values. FIG. 1 is a flow diagram illustrating a quality evaluation method. 1A is a graph showing an example of the result of calculating the coefficient of determination for the estimated carbon concentration values obtained using the learning models of Examples 1 to 3 and the Comparative Example, and FIG. 1B is a graph showing an example of the result of calculating the RMSE for the estimated carbon concentration values obtained using the learning models of Examples 1 to 3 and the Comparative Example.

[Introduction]
FIG. 1 is a flowchart showing an example of a method for manufacturing a learning model of the present invention. The manufacturing method shown in FIG. 1 can also be said to be a method for constructing a learning model (a method for modeling a learning model). The learning model is constructed by machine learning. The constructed learning model is an arithmetic expression used for quality evaluation of a member made of heat-treated steel (carbon steel or alloy steel). FIG. 2 is a block diagram showing the configuration of a learning system for manufacturing a learning model by machine learning. The learning system includes an image acquisition device 11 including a microscope and a camera, and a computer device 12.

First, a method for constructing a learning model will be described. Next, a quality evaluation system and a quality evaluation method for performing quality evaluation using the constructed learning model will be described.
A component made of heat-treated carbon steel or alloy steel used to construct a learning model is referred to as a "subject component." In contrast, a component for which the quality evaluation is performed using the learning model is referred to as an "evaluation target component." The evaluation target component is made of the same heat-treated steel (carbon steel or alloy steel) as the target component.

The target component (evaluation target component) is a heat-treated carbon steel or alloy steel, such as chromium steel, chromium molybdenum steel, or chromium molybdenum nickel steel. Examples of carbon steel are S10C, C25C, S40C, etc. Examples of alloy steel are SCM415, SCr415, SNCM420, SAE5120, etc. The alloy steel contains at least one of nickel, chromium, manganese, and molybdenum in a specified content.

Whether it is carbon steel or alloy steel, steel with a carbon concentration of 0.4% or less, and especially low carbon steel, is used as the target component and evaluation target component in the presently disclosed invention. In addition, steel with a retained austenite amount in the range of 0 to 50% is used as the target component and evaluation target component in the presently disclosed invention. In addition, steel with a hardness in the range of 400 to 800 HV is used as the target component and evaluation target component in the presently disclosed invention.

A specific example of the target member and the evaluation target member is, for example, a raceway of a rolling bearing. Figure 3 is a cross-sectional view of a raceway of a rolling bearing. The raceway shown in Figure 3 is an inner ring 7 of a tapered rolling bearing. When the target member and the evaluation target member are the inner ring 7, the learning model is a model for performing a quality evaluation of the raceway 8 of the inner ring 7 where the rolling elements (tapered rollers) roll and contact. The raceway 8 of the inner ring 7 is the subject of the quality evaluation. To be more specific, in the case of the inner ring 7, the position about 100 μm from the surface of the raceway 8 is the position where the shear stress is maximum. Therefore, the range including that position is defined as the "target region", and a learning model is constructed for performing a quality evaluation of the target region. Furthermore, the learning model is used to perform a quality evaluation of the target region.

The heat treatment includes carburizing and quenching, carbonitriding and quenching, etc. In this disclosure, the case where carburizing and quenching is performed will be described. Carburizing and quenching is performed for the purpose of improving the wear resistance and fatigue resistance of, for example, the raceway of a rolling bearing (inner ring 7 shown in FIG. 3). Factors that have a significant effect on the quality of steel that has been carburized and quenched include the carbon concentration, the amount of retained austenite, and hardness.

In FIG. 2, the computer device 12 includes a processing device 13 including a CPU (Central Processing Unit), a storage device 14 consisting of a hard disk or the like, and an input/output interface 15. The processing device 13 realizes various functional units by executing a computer program stored in the storage device 14. As the functional units, the processing device 13 includes an image processing unit 16, a feature calculation unit 17, a learning unit 18, and an evaluation calculation unit 19. The storage device 14 can store data processed by the processing device 13 (image data, feature values, learning models, various information for learning).

[Method of manufacturing learning model]
1 includes a preparatory step S1, an image acquisition step S2, an image processing step S3, a feature amount calculation step S4, and a model generation step S5. Each step will be described below.

[Preliminary step S1]
Heat treatment (carburizing and quenching) is performed on a plurality of target members (intermediate products that will become the inner ring 7 shown in FIG. 3). In this heat treatment, the carbon potential in the heat treatment furnace is made different for each target member. In other words, the carbon potential, which is a heat treatment condition, is made different for each target member. More specifically, the carbon potential is set to 0.2% for the target member TP1 designated as test piece number 1, 0.5% for the target member TP2 designated as test piece number 2, 0.75% for the target member TP3 designated as test piece number 3, 0.8% for the target member TP4 designated as test piece number 4, and 0.9% for the target member TP5 designated as test piece number 5. The heat treatment conditions other than the carbon potential are the same.

A plurality of target members TP1 to TP5 are prepared, not just one each. In other words, a plurality of target members TP1, a plurality of target members TP2, a plurality of target members TP3, etc. are prepared. Each of the target members TP1 to TP5 is cut, and the physical properties of the target regions are measured.

The carbon concentration of each of the heat-treated target materials TP1 to TP5 is measured using an EPMA to obtain an actual carbon concentration value. That is, the carbon concentration is obtained as a physical property value in the target region of each of the target materials TP1 to TP5.
For each of the heat-treated target parts TP1 to TP5, the amount of retained austenite is measured by X-ray analysis to obtain an actual value of the amount of retained austenite. That is, the amount of retained austenite is obtained as a physical property value in the target region of each of the target parts TP1 to TP5.
The hardness of each of the heat-treated target parts TP1 to TP5 is measured by a hardness tester (Vickers hardness tester) to obtain an actual hardness value. In other words, the hardness is obtained as a physical property value in the target region of each of the target parts TP1 to TP5.
Further, an inspector (expert) performs a microscope observation of each of the target parts TP1 to TP5 after the heat treatment, and determines a quality level value (actual measurement value). The quality level value is, for example, a score value on a 5-point scale. The quality level value may be determined by taking into consideration the results of the microscope observation, including each of the actual measurement values (physical property values).

As the physical property values of each of the target parts TP1 to TP5 after the heat treatment, at least the carbon concentration among the carbon concentration, the retained austenite, and the hardness needs to be obtained, but in the present disclosure, all of them are obtained.
Furthermore, if the physical property values of each of the target parts TP1 to TP5 after heat treatment are obtained, it is not necessary to obtain the quality level values of each of the target parts TP1 to TP5; however, in the present disclosure, both are obtained.
The physical property values and the scores of the target materials TP1 to TP5 are stored in the storage device 14 in association with the target materials TP1 to TP5 (information on the identification numbers).
In this embodiment, it is assumed that the measured values of the carbon concentrations of the target members TP1 to TP5 are the same as the carbon potentials set during the heat treatment.
Furthermore, the carbon concentrations of the target members TP1 to TP5 do not have to be measured using EPMA, and a carbon analyzer or the like may also be used.

[Image acquisition step S2]
The cut surfaces of the target members TP1 to TP5 are mirror-polished and etched with a predetermined etching solution, and then a microscopic image of the metal structure is acquired by the image acquisition device 11. In the image acquisition step S2, 4 shows examples of the microscope images of the target areas TP1 to TP5.

In this way, in the image acquisition process S2, microscopic images of the target area at a predetermined depth from the surface of each of the target parts TP1 to TP5 whose physical property values and quality level values have been confirmed are acquired. The acquired images are transmitted to the computer device 12 and stored in the storage device 14.

The range of one microscope image is narrow. Therefore, for each of the target members TP1 to TP5, microscope images of different locations along a direction perpendicular to the depth direction from the surface of the raceway 8 of the inner ring 7 shown in Figure 3 are obtained. In Figure 3, the direction perpendicular to the depth direction from the surface of the raceway 8 is indicated by a "dashed line." Multiple microscope images of the target area are obtained along this dashed line. In this case, the target area is divided into multiple sections, and images are taken so that each section partially overlaps. The direction in which the area is divided into multiple sections is along the dashed line, and images are taken so that the ends in the direction along the dashed line overlap. An image of one section is called a "section image."

In each of the partition images, a characteristic part included in the end (the part) is extracted. A characteristic part is, for example, a part that has a different brightness (higher part) compared to the surroundings, and multiple characteristic parts are extracted. FIG. 5 is an explanatory diagram of the process of joining two partition images into one. In FIG. 5, a common characteristic part is shown by a circle in two partition images P1 and P2 adjacent to each other along the dashed line. Using the characteristic part included in the partition image as a reference, a single microscope image of the target area is obtained by joining multiple partition images along the dashed line. Note that FIG. 5 is an explanatory diagram of the case where two partition images P1 and P2 adjacent to each other along the dashed line are joined. FIG. 6 is an explanatory diagram of a microscope image Pj in which multiple partition images (five) more than two are joined. The direction in which the partition images are joined, that is, the direction along the dashed line, is referred to as the "width direction". The process of joining the partition images may be performed by the image processing unit 16 of the computer device 12.

In this way, in the image acquisition process S2, the target area is divided into multiple sections, and multiple section images (P1, P2) are acquired by photographing each section so that a portion (edge) of each section overlaps. Furthermore, the multiple section images (P1, P2) are connected based on the characteristic parts contained in the section images (P1, P2). This results in one microscope image Pj (see Figure 6) of the target area being acquired.

Furthermore, in the image acquisition step S2, the microscopic image Pj shown in FIG. 6 is processed. More specifically, the image is processed into a microscopic image (see FIG. 7) of a predetermined width in the depth direction, with a position at a predetermined depth from the surface of the track 8 (see FIG. 3) as the center. The predetermined depth is a position 100 μm from the surface of the track 8, and the predetermined width is, for example, 100 μm. In other words, the image is processed into a microscopic image of a predetermined width (100 μm) in the depth direction including the position where the shear stress is maximum. The processing may be performed by the image processing unit 16 of the computer device 12.

As a result of the above, microscopic images such as those shown in Figure 7 are obtained for each of the target parts TP1 to TP5.

[Image processing step S3]
In the image processing step S3, each of the microscopic images acquired in the image acquisition step S2 is processed by the image processing unit 16 of the computer device 12. The image processing step S3 includes a noise removal process S3-1, a histogram equalization process S3-2, and a process S3-3 for generating a bright side image and a dark side image. In the computer device 12, the microscopic images are processed as grayscale images.

[Noise Removal Processing S3-1]
(A) in Fig. 8 is an explanatory diagram of a part of the microscope image (grayscale image) shown in Fig. 7. (B) in Fig. 8 is a graph showing the relationship between the position (pixel) along one line in the width direction in the image of (A) and the brightness at that position. The horizontal direction of the graph of (B) represents the position in the width direction, and the vertical direction of the graph of (B) represents the brightness.

The partition image (see FIG. 5) acquired in the image acquisition step S2 is affected by the lens of the camera of the image acquisition device 11, and has a relatively high brightness in the center of the partition image and a relatively low brightness at the edge of the partition image. For this reason, as shown in FIG. 8B, when multiple partition images are stitched together as described above, the brightness fluctuates greatly and periodically along the width direction. Therefore, in the image processing step S3, noise caused by such fluctuations is first removed. This removal process will be explained using the image in FIG. 7 and parts of it shown in FIG. 8A to FIG. 8F.

The grayscale image shown in FIG. 7 is copied to generate a copy image. From the copy image, the following formula (1) is used to generate a background image. In other words, the pixel of interest in the copy image is set as the origin, weighting is performed using the following formula (1), and a process is executed to calculate the average brightness.

In formula (1), "x" is the coordinate of the pixel in the grayscale image in the width direction, and "y" is the coordinate of the pixel in the grayscale image in the direction perpendicular to the width direction. "σ" is a set value, which is the maximum difference between pixel values that are determined to have the same brightness. This process is a blurring process known as Gaussian blurring.

(C) in Figure 8 is the grayscale image obtained by the blurring process. (D) in Figure 8 is an explanatory diagram showing the relationship between positions (pixels) along one row in the width direction in the image of (C) and the brightness at those positions. The grayscale image obtained by the blurring process corresponds to the background image of the original grayscale image. In this way, by performing a process of blurring the original grayscale image, the background image shown in (C) is generated as the background image.

Noise is removed from the original grayscale image shown in (A) in Fig. 8 by using the background image shown in (C). (E) in Fig. 8 is the grayscale image from which noise has been removed. (F) in Fig. 8 is an explanatory diagram showing the relationship between positions (pixels) along one line in the width direction in the image of (E) and the brightness at those positions.
By performing the calculation according to the following equation (2), a grayscale image from which noise has been removed is generated, as shown in (E) in FIG.

The calculation shown in formula (1) may be performed on the original grayscale image to generate a background image, and the copied image may be edited with the background image to generate a grayscale image with noise removed.

As described above, in the image processing step S3, a noise removal process S3-1 is performed. This process generates a copy image by copying the original grayscale image shown in FIG. 7. A process for blurring one of the original grayscale image and the copy image is performed to generate a background image (see (C) in FIG. 8). This background image is used to generate a grayscale image (see (E) in FIG. 8) from which noise has been removed from the other of the original grayscale image and the copy image. In the present disclosure, the equalization process S3-2 described below is performed on the grayscale image from which noise has been removed, and a bright side image and a dark side image, which will be described later, are generated separately based on the image data that has been subjected to this process.

[Histogram equalization process S3-2]
A histogram of brightness is generated from each grayscale image from which noise has been removed by the noise removal process S3-1, and the histogram is then subjected to equalization process S3-2. In the case of the present disclosure, the histogram is a cumulative sum of the number of pixels for each brightness (tone) from brightness 0 to brightness 255. The histogram equalization process S3-2 is a process for converting the original data so that the slope of the graph of the cumulative frequency of the histogram becomes constant. More specifically, a conversion process according to the following formula (3) is performed on each pixel of the grayscale image from which noise has been removed.

Fig. 9A shows a part of the grayscale image from which noise has been removed, and Fig. 9B shows a histogram of Fig. 9A. The horizontal axis of Fig. 9B represents brightness (0 to 255), and the vertical axis represents the number of pixels.
Fig. 10A shows a part of a grayscale image that has been subjected to uniformization processing, and Fig. 10B shows a histogram of Fig. 10A. The horizontal axis of Fig. 10B represents brightness (0 to 255), and the vertical axis represents the number of pixels.
This uniformization process S3-2 clarifies the contours of the grains contained in the image.

From the grayscale image that has been subjected to the equalization process S3-2 as described above, a dark side image and a bright side image, which will be described later, are generated.

[Process S3-3 for generating bright side image and dark side image]
Fig. 11A shows a portion of a grayscale image that has been subjected to equalization processing, Fig. 11B shows a dark side image generated from the grayscale image, and Fig. 11C shows a bright side image generated from the grayscale image.
The dark side images are images in which the grains of the tissue of the target members (TP1 to TP5, respectively) are shown as dark objects (black objects) and the rest are shown as bright areas (white areas).
The bright side images are images in which the tissue grains of the target members (TP1 to TP5, respectively) are shown as bright objects (white objects) and the rest are shown as dark areas (black areas).

The grains of the structure correspond to “crystal grains.” The grains shown as dark objects in the dark-side image (B) and the grains shown as bright objects in the bright-side image (C) are different and have different structures.
Dark objects (dark regions) are areas that have been corroded by the etching, and light objects (light regions) are areas that have not been corroded by the etching.
The bright side image and the dark side image are generated as follows. This generation is performed by the image processing unit 16 of the computer device 12.

First, a histogram is generated from the grayscale image ((A) in Figure 11) that has been subjected to equalization processing. This histogram accumulates the number of pixels for each brightness level from 0 to 255. Figure 12 is an explanatory diagram of this histogram, with the horizontal axis (class) representing brightness (0 to 255) and the vertical axis (frequency) representing the number of pixels. Figure 12 also shows a curve L indicating the cumulative frequency for each brightness level (each tone).

A first threshold value and a second threshold value used to generate a bright side image and a dark side image are set in advance.
In the histogram shown in Fig. 12, the brightness at which the cumulative ratio from the dark side (brightness 0) is a first set value is set as the first threshold value. The first set value may be, for example, a value between 30% and 50%. In the present disclosure, 40% is used as the first set value. Therefore, in the case of the histogram shown in Fig. 12, the brightness at which the cumulative ratio from the dark side (brightness 0) is 40%, that is, the brightness "100", is set as the first threshold value.
In addition, in the histogram shown in FIG. 12, the brightness at which the cumulative ratio from the bright side (brightness 255) is the second set value is set as the second threshold value. For example, a value of 10% or more and 30% or less is adopted as the second set value. In the invention of the present disclosure, 20% is adopted as the second set value. Therefore, in the case of the histogram shown in FIG. 12, the brightness at which the cumulative ratio from the bright side (brightness 255) is 20%, that is, the brightness "180", is set as the second threshold value. The second threshold value is a value larger than the first threshold value.

Then, from the grayscale image (A in FIG. 11) that has been subjected to the equalization process, a dark side image (B in FIG. 11) and a bright side image (C in FIG. 11) are generated as follows.
That is, among the pixels of the grayscale image (A) that has been subjected to the equalization process, pixels whose brightness is equal to or less than a first threshold (brightness equal to or less than 100) are regarded as elements that constitute a dark object. In contrast, pixels whose brightness exceeds a second threshold (brightness equal to or greater than 180) are regarded as elements that constitute a bright object. In this way, a dark-side image (B) is generated from the grayscale image (A) that has been subjected to the equalization process, in which pixels that are equal to or less than the first threshold are black and other areas are white. Also, a bright-side image (C) is generated from the grayscale image (A) that has been subjected to the equalization process, in which areas that exceed the second threshold are white and other areas are black.

As described above, a set of images, consisting of a bright side image and a dark side image, is generated based on the grayscale images of each of the target members TP1 to TP5. As described above, since multiple target members (e.g., target member TP1) are prepared, a set of images, consisting of a bright side image and a dark side image, is generated for each of the multiple target members (TP1). The same is true for the remaining target members TP2 to TP5, and a set of images, consisting of a bright side image and a dark side image, is generated for each target member.

[Feature amount calculation step S4]
11B, the dark side image is an image in which tissue grains are shown as dark objects (black objects). Therefore, dark side feature amounts of a plurality of objects included in the dark side image (B) are obtained.
11C, the bright side image is an image in which tissue grains are shown as bright objects (white objects). Therefore, bright side feature amounts of the multiple objects included in the bright side image (C) are obtained.
The process of obtaining the dark side feature amount and the bright side feature amount is executed by the image processing function of the feature amount calculation unit 17 of the computer device 12 .

Figure 13 is an explanatory diagram showing one of the dark objects contained in the dark side image ((B) in Figure 11). Figure 14 is an explanatory diagram showing one of the bright objects contained in the bright side image ((C) in Figure 11). Multiple objects are extracted from each of the bright side image and the dark side image. The extracted object is a mass portion having a predetermined area. Note that objects whose area is smaller than a threshold value and objects whose area is larger than a threshold value are excluded from the objects to be extracted. Dark side features of each object extracted from the dark side image are calculated. Bright side features of each object extracted from the bright side image are calculated.

Examples of the dark side feature amount include the major axis, minor axis, angle, circularity, representative value of brightness, skewness, number of pixels, aspect ratio, tone density, etc. of a dark object.
Examples of bright side features include the major axis, angle, circularity, representative value of brightness, kurtosis, skewness, number of pixels, and aspect ratio of a bright object.
As shown in Figures 13 and 14, the long axis is the maximum dimension of the object, and the short axis is the maximum dimension in the direction perpendicular to the long axis. The angle is the inclination angle of the long axis with respect to the width direction. The circularity is the calculated value of "4π x (area of object) / (perimeter of object) ^ 2". The aspect ratio is the calculated value of "long axis / short axis". The concentricity is the calculated value of "(area of object) / (area of the smallest rectangle that encloses the object)".
Representative values of brightness include the median, maximum, and mode of brightness. The median brightness is as follows: "The median of the class values when a histogram is created for an object in a grayscale image ((A) in FIG. 11) whose features are to be calculated, with the horizontal axis (class) being brightness and the vertical axis (frequency) being the number of pixels." The maximum brightness is as follows: "The maximum class value when said histogram is created." The mode of brightness is as follows: "The class value with the highest frequency in said histogram." Skewness and kurtosis are values calculated from the shape of said histogram.

Here, the bright side feature amount and the dark side feature amount calculated in the feature amount calculation step of this embodiment include at least one of the circularity of the object in the bright side image and a representative value of the brightness of the object in the dark side image.
In other words, when the bright side feature calculated in the feature calculation process includes the circularity of the object, the dark side feature calculated in the feature calculation process may or may not include a representative value of the brightness of the object.
In addition, when the dark side feature calculated in the feature calculation process includes a representative value of the brightness of the object, the bright side feature calculated in the feature calculation process may or may not include the circularity of the object.
In this embodiment, the bright side feature amount and the dark side feature amount calculated in the feature amount calculation step include both the circularity of the object in the bright side image and a representative value of the brightness of the object in the dark side image.

The circularity of an object in the bright side image, which is included in the bright side feature, correlates with the carbon concentration of the target material. Below, we explain the correlation between the circularity of an object in the bright side image and the carbon concentration of the target material.

When observing the metal structure of heat-treated (hardened) steel with a metallurgical microscope, if the carbon concentration of the steel is relatively low, the needle-shaped blocks that make up the martensite structure are observed as relatively bright areas. On the other hand, if the carbon concentration of the steel is relatively high, the chunky retained austenite is observed as relatively bright areas.

FIG. 15( a ) is a diagram showing a schematic diagram of the metal structure of a target component after heat treatment in which the carbon concentration is 0.2%, and FIG. 15 ( b ) is a diagram showing a schematic diagram of the metal structure of a target component after heat treatment in which the carbon concentration is 0.9%.
15(a), a plurality of needle-shaped blocks B are generally observed in the metal structure of a target member having a carbon concentration of 0.2%. Each block B is composed of lath martensite. Each block B is composed of lath martensite with a nearly uniform crystal orientation. For this reason, the degree of light reflection and unevenness vary from block B to block B, and in a microscope image, each block includes a portion observed as a relatively bright portion (block B1) and a portion observed as a dark portion (block B2).

Furthermore, as shown in FIG. 15( b ), in the metal structure of a target member having a carbon concentration of 0.9%, lenticular martensite R, retained austenite A, and a block B (B2) composed of lath martensite are generally observed.
The retained austenite A is surrounded by lenticular martensite R and blocks B, forming a block shape, and is observed as a relatively bright part. When the carbon concentration is relatively low, the retained austenite and lenticular martensite hardly appear, but increase as the carbon concentration increases.

Therefore, when the carbon concentration is relatively low in the microscope image, most of the relatively bright parts are the blocks B1 shown in Fig. 15(a) and are needle-shaped. When the carbon concentration is high, the needle-shaped parts in the relatively bright parts decrease and the lumpy parts (retained austenite A in Fig. 15(b)) increase. In other words, it can be said that the shape of the relatively bright parts (objects included in the bright side image) in the microscope image of the hardened steel has a correlation with the carbon concentration.
From this, it can be said that the circularity of an object included in a bright side image also has a correlation with the carbon concentration.

Fig. 16(a) is a diagram showing an example of a bright side image of a target member TP1 having a measured carbon concentration of 0.2%, and Fig. 16(b) is a partially enlarged view of Fig. 16(a). Fig. 16(b) is an enlarged view of the area inside a square mark F1 in Fig. 16(a).
16B, an object O1 represents a block and has a needle shape. The circularity of this object O1 is 0.1.

Fig. 17(a) is a diagram showing an example of a bright side image of a target member TP5 having a measured carbon concentration of 0.9%, and Fig. 17(b) is a partially enlarged view of Fig. 17(a). Fig. 17(b) is an enlarged view of the area inside the square mark F2 in Fig. 17(a).
17B, an object O2 shows retained austenite and has a nearly rectangular, lumpy shape. The circularity of this object O2 is 0.5.

16 and 17, the object O1 of the target member TP1 has a relatively low circularity because it is needle-like, while the object O2 of the target member TP5 has a relatively high circularity because it is chunk-like.
As such, the circularity of the target part TP5, whose actual carbon concentration is 0.9%, is greater than the circularity of the target part TP1, whose actual carbon concentration is 0.2%, and it can be seen that the circularity of an object contained in the bright side image can be suitably used as a feature related to carbon concentration.

The representative value of the object's brightness, which is included in the dark feature, also correlates with the carbon concentration of the target material. Below, we will explain the correlation between the representative value of the object's brightness and the carbon concentration of the target material.

As shown in Figures 15(a) and 15(b), block B seen in the metal structure of heat-treated (hardened) steel contains, as described above, a portion observed as a relatively bright portion (block B1 in Figure 15(a)) and a portion observed as a relatively dark portion (block B2 in Figures 15(a) and (b)). As the carbon concentration of block B increases, the structure becomes finer, and the reflectance changes accordingly, resulting in differences in brightness when observed. More specifically, block B changes to become darker as the carbon concentration increases.

For this reason, in a microscope image where the carbon concentration is relatively low, many of the relatively dark parts are block B2 shown in Fig. 15(a). When the carbon concentration is high, the relatively dark part block B (B2) becomes darker than when the carbon concentration is low. In other words, it can be said that the brightness of the relatively dark parts (objects included in the dark side image) in a microscope image of a hardened steel material has a correlation with the carbon concentration.
From this, it can be said that the representative value of the brightness of the object included in the dark side image also has a correlation with the carbon concentration.

In this embodiment, the median value is used as the representative value of the brightness of the object included in the dark side image.
FIG. 18 is a diagram showing an example of a histogram for determining the median value of the brightness of an object included in a dark side image.
18, in order to find the median brightness of an object included in the dark side image, a histogram is created for the pixels that make up the object, with the horizontal axis representing pixel brightness (class value) and the vertical axis representing the number of pixels (frequency). The median brightness is found based on this histogram.
The median in a histogram refers to a class value (brightness) corresponding to a cumulative frequency that is half the total frequency when the cumulative frequency is taken from the bottom of the frequency distribution.

Fig. 19(a) is a diagram showing an example of a dark side image of a target member TP1 having a measured carbon concentration of 0.2%, and Fig. 19(b) is a partially enlarged diagram of Fig. 19(a). Fig. 19(b) is an enlarged view of the area inside a square mark F3 in Fig. 19(a).
Fig. 20(a) is a diagram showing an example of a dark side image of a target member TP5 having a measured carbon concentration of 0.9%, and Fig. 20(b) is a partially enlarged diagram of Fig. 20(a). Fig. 20(b) is an enlarged view of the area inside the square mark F4 in Fig. 20(a).

Comparing the median brightness value of object O3 in FIG. 19B with the median brightness value of object O4 in FIG. 20B, it was confirmed that the median brightness value of object O4 was smaller.
Thus, in the examples shown in Figures 19 and 20, the median brightness of the objects in the target part TP5, which has an actual measured carbon concentration of 0.9%, is smaller than the median brightness of the objects in the target part TP1, which has an actual measured carbon concentration of 0.2%, and it can be seen that the median brightness of the objects included in the dark side image can be suitably used as a feature related to carbon concentration.

The above-mentioned examples of dark side features and bright side features are the features that are ultimately used for machine learning as a result of the deletion process described below. The bright side features and dark side features used may be the same, but as described above, they are different.

In the feature amount calculation step S4, the following reduction process (1) or (2) is performed on the bright side feature amount.
Reduction process (1): A process for reducing one of two different bright side feature amounts when the correlation between these two bright side feature amounts is strong.
Reduction process (2): A process of reducing some of the bright side feature amounts by performing principal component analysis on the bright side feature amounts.

In the feature amount calculation step S4, the following reduction process (3) or (4) is performed on the dark side feature amount.
Reduction process (3): A process for reducing one of two different dark side feature amounts when the correlation between these two dark side feature amounts is strong.
Reduction process (4): A process of reducing some of the dark side feature amounts by performing principal component analysis on the dark side feature amounts.

In this way, the light feature amount and the dark feature amount are reduced, and machine learning is performed using the reduced feature amount data. For example, if the correlation coefficient between two feature amounts is 0.8 or more, it is determined that the correlation is strong.

The bright side feature amount and the dark side feature amount thus obtained are further normalized by the following formula (4).

In this way, in the feature calculation step S4, bright side features of multiple objects included in the bright side image ((C) in FIG. 11) and dark side features of multiple objects included in the dark side image ((B) in FIG. 11) are found. The bright side features and dark side features of each of the target members TP1 to TP5 are found as numerical values and stored in the storage device 14. In this way, the bright side features and dark side features obtained from the microscope image of one target member (e.g., TP1) are associated with information on the identification number of one target member (e.g., TP1) and stored in the storage device 14.

The type of steel may be included as a light side feature and a dark side feature. In this case, a dummy variable that quantifies the type of steel is used. For example, the dummy variable for S20C is "0" and the dummy variable for S55C is "1".

[Model generation step S5]
In the model generation step S5, a learning model is generated using the light side feature amounts and dark side feature amounts of each of the target members TP1 to TP5 stored in the storage device 14.
Each process in the model generation step S5 is performed by the learning unit 18 of the computer device 12. The machine learning performed by the learning unit 18 is performed based on a machine learning algorithm including any one of linear regression, ridge regression, and lasso regression. In the method of the present disclosure, the use of lasso regression is preferable because it increases accuracy. The machine learning performed here is supervised learning.

As described above, one target component (e.g., TP1) is associated with the light side feature amount and the dark side feature amount of that target component (e.g., TP1). In this embodiment, the learning models that can be generated are as follows:

In the pre-process S1, the carbon concentrations of the target materials TP1 to TP5 are actually measured as physical property values of each of the target materials TP1 to TP5, and the measured values of the carbon concentrations are stored in the storage device 14 in association with the target materials TP1 to TP5.
Therefore, a learning model is generated using the bright side feature amount and the dark side feature amount calculated in the feature amount calculation step S4 for one target component and the actual measured value of the carbon concentration of the one target component as training data. The generated learning model is a model that outputs an estimated value of the carbon concentration as a physical property value of the target component from the bright side feature amount and the dark side feature amount.

In the preliminary process S1, the amount of retained austenite is actually measured as a physical property value for each of the target parts TP1 to TP5, and the actual measured values of the amount of retained austenite are stored in the memory device 14 in correspondence with the target parts TP1 to TP5.
Therefore, a learning model is generated using the bright side feature amount and the dark side feature amount calculated in the feature amount calculation step S4 for one target component and the actual measured value of the amount of retained austenite for the one target component as training data. The generated learning model is a model that outputs an estimated value of the amount of retained austenite as a physical property value of the target component from the bright side feature amount and the dark side feature amount.

In the pre-process S1, the hardness of each of the target members TP1 to TP5 is actually measured as a physical property value, and the measured hardness values are stored in the storage device 14 in association with the target members TP1 to TP5.
Therefore, a learning model is generated using the bright side feature amount and the dark side feature amount calculated in the feature amount calculation step S4 for one target component and the actual measured value of the hardness of the one target component as teacher data. The generated learning model is a model that outputs an estimated value of hardness as a physical property value of the target component from the bright side feature amount and the dark side feature amount.

In the preliminary process S1, the quality level values (scoring values on a 5-point scale) of the target parts TP1 to TP5 are scored by workers, and the actually scored quality level values (actual measured values of the quality level values) are stored in the memory device 14 in correspondence with the target parts TP1 to TP5.
Therefore, a learning model is generated using the bright side feature amount and the dark side feature amount calculated in the feature amount calculation step S4 for one target component and the actual measured value of the quality level value of that one target component as training data. The generated learning model is a model that outputs an estimate of the quality level value of the target component from the bright side feature amount and the dark side feature amount.

As described above, bright-side images and dark-side images are generated from the microscopic images of each of the target parts TP1 to TP5, and once bright-side feature values and dark-side feature values are obtained, a learning model is generated that outputs an estimate of a physical property value (an estimate of carbon concentration, an estimate of the amount of retained austenite, or an estimate of hardness) from these bright-side feature values and dark-side feature values.
Furthermore, in the invention of the present disclosure, bright-side images and dark-side images are generated from the microscopic images of each of the target parts TP1 to TP5, and when bright-side feature quantities and dark-side feature quantities are obtained, a learning model that outputs an estimated value of a quality level value from these bright-side feature quantities and dark-side feature quantities is also generated.

In this manner, we have described a case in which both a learning model that outputs each of the three types of physical property values and a learning model that outputs a quality level value are generated, but it is sufficient that at least one of these learning models that outputs an estimated value of carbon concentration is generated. Even in this case, it is possible to use that learning model to perform quality evaluation of the evaluation target component, which will be described later.

[Regarding the manufacturing method of the learning model of the present disclosure]
As described above, the method for manufacturing a learning model according to the present disclosure is a method for manufacturing (constructing) a learning model used for quality evaluation of a member made of heat-treated carbon steel or alloy steel, which includes an image acquisition step S2, an image processing step S3, a feature amount calculation step S4, and a model generation step S5, as described above.

In the image acquisition process S2, a microscopic image of the target area of each of the target parts TP1 to TP5 is acquired. At least one physical property value of the carbon concentration, hardness, and amount of retained austenite, and a quality level value (a score on a five-point scale) of each of the target parts TP1 to TP5 are actually measured. This image acquisition process S2 is performed by the image acquisition device 11.

In the image processing step S3, a bright side image ((C) in FIG. 11) and a dark side image ((B) in FIG. 11) are generated separately from the grayscale image based on the microscopic image acquired in the image acquisition step S2. The image processing step S3 is performed by the image processing unit 16, which is one of the functional parts of the computer device 12.

In the feature amount calculation step S4, bright side feature amounts of a plurality of objects included in the bright side image and dark side feature amounts of a plurality of objects included in the dark side image are calculated. The feature amount calculation step S4 is performed by the feature amount calculation unit 17, which is one of the functional parts of the computer device 12.
The bright side feature amount and the dark side feature amount calculated in the feature amount calculation step S4 include both the circularity of the object included in the bright side image and the median brightness of the object included in the dark side image.

In the model generation step S5, a learning model that outputs an estimated value of a physical property value is generated from the bright side feature amount and the dark side feature amount. In the invention of the present disclosure, in the model generation step S5, a learning model that outputs an estimated value of a quality level value is also generated from the bright side feature amount and the dark side feature amount.
The learning model is generated by machine learning. When generating a learning model that outputs estimated values of physical properties, the bright side feature values and dark side feature values obtained in the feature amount calculation process S4 and the actual measured values of the physical properties of the target members TP1 to TP5 are used as teacher data. When generating a learning model that outputs estimated values of quality level values, the bright side feature values and dark side feature values obtained in the feature amount calculation process S4 and the actual measured values of the quality level values of the target members TP1 to TP5 are used as teacher data. The model generation process S5 is performed by the learning unit 18, which is one of the functional units of the computer device 12.

In the image acquisition process S2, it is sufficient that only the carbon concentration among the physical property values of each of the target materials TP1 to TP5 is actually measured. In this case, only a learning model that outputs an estimated value of the carbon concentration is generated.

According to the above method, a learning model is obtained in which the bright area feature amount and the dark area feature amount are explanatory variables and the physical property value is the objective variable. Also, a learning model is obtained in which the bright area feature amount and the dark area feature amount are explanatory variables and the quality level value is the objective variable. The learning model is produced using not only the bright side feature amount or not only the dark side feature amount, but both the bright side feature amount and the dark side feature amount as teacher data. By performing quality evaluation using the learning model produced in this way, the physical property value or the quality level value can be estimated with high accuracy, and the accuracy of the quality evaluation can be improved.

Furthermore, the learning model using carbon concentration as the objective variable includes the circularity of an object included in the bright side image and the median brightness of an object included in the dark side image as bright side features and dark side features.
As described above, the circularity of an object included in the bright-side image and the median brightness of an object included in the dark-side image are correlated with the carbon concentration.
In other words, the learning model that uses carbon concentration as the objective variable includes, as bright-side feature and dark-side feature, the circularity of an object included in the bright-side image, which is correlated with carbon concentration, and the median brightness of an object included in the dark-side image. Therefore, by using this learning model, it is possible to improve the accuracy when estimating the carbon concentration of a target component.
If the carbon concentration is estimated using this learning model, it is not necessary to use an apparatus such as an EPMA, which reduces costs and shortens the measurement time.
As a result, quality evaluation of the heat-treated product can be performed with high accuracy at low cost and in a short time.

The bright side feature amount and the dark side feature amount calculated in the feature amount calculation step S4 need only include at least one of the circularity of the object included in the bright side image and the median brightness of the object included in the dark side image. If at least one of these is included, the accuracy of estimating the carbon concentration of the target component can be improved.

Furthermore, in order to generate the bright side image ((C) in FIG. 11) and the dark side image ((C) in FIG. 11), in the image processing step S3 included in the method of the present disclosure, among the pixels of the grayscale image ((A) in FIG. 11), pixels whose brightness is equal to or less than a first threshold value are regarded as elements constituting a dark object, and pixels whose brightness exceeds a second threshold value that is greater than the first threshold value are regarded as elements constituting a bright object. In this way, from the grayscale image, a dark side image in which pixels equal to or less than the first threshold value are black and other areas are white, and a bright side image in which areas exceeding the second threshold value are white and other areas are black are generated.
In this method, regions in the grayscale image where the brightness is at an intermediate level between the first threshold value and the second threshold value are not treated as tissue grains that are effective for determining bright side feature amounts and dark side feature amounts.

In addition, in the image processing step S3 of the method disclosed herein, a brightness histogram ((B) in FIG. 9) is generated from the grayscale image ((A) in FIG. 9), and the histogram is equalized. A dark side image and a bright side image are generated from the grayscale image ((A) in FIG. 10) that has been subjected to the equalization process. In this case, for example, if the brightness of the grayscale image is biased, the overall balance is improved. Also, the outlines of grains in the grayscale image are clarified.

FIG. 21A is a graph showing the estimated carbon concentration value obtained using the learning model for evaluation of carbon concentration constructed by carrying out the homogenization process, and the actual measured carbon concentration value.
In contrast, (B) in Figure 21 is a graph showing the estimated carbon concentration obtained using an evaluation learning model for carbon concentration constructed without performing the homogenization process, and the actual measured carbon concentration.

In (A) and (B) in Figure 21, an estimate of the carbon concentration was obtained using an evaluation learning model that outputs an estimate of the carbon concentration, which was generated using the light-side feature values and dark-side feature values of each of the target materials TP1 to TP5.
The estimated carbon concentration was obtained by actually measuring the carbon concentration and providing the bright side feature amount and the dark side feature amount of the target component, the actual measured value of which was known, to the evaluation learning model.
In (A) and (B) of FIG. 21, estimated values versus actual measured values of a target member are plotted on a graph.

The bright side image and dark side image used to generate the evaluation learning model to obtain the estimated value of (A) in Figure 21 were generated by performing the necessary processing, including histogram equalization processing, on the microscopic images of each of the target members TP1 to TP5.
The bright side image and dark side image used to generate the evaluation learning model to obtain the estimated value of (B) in Figure 21 were generated by performing other necessary processing except for histogram equalization processing on the microscopic images of each of the target members TP1 to TP5.

The coefficient of determination (contribution rate) between the estimated value and the actual value when the equalization process is performed ((A) in Figure 21) is closer to 1 than when the equalization process is not performed ((B) in Figure 21). In other words, it can be seen that the prediction accuracy of the learning model is improved by performing the equalization process.

In the image acquisition step S2 included in the method of the present disclosure, the target area of each of the target members TP1 to TP5 is divided into multiple sections, and multiple section images are acquired by photographing each section so that a portion of each section overlaps. Then, a single microscope image of the target area is acquired by connecting the multiple section images based on the characteristic parts contained in each section image. With this method, the target area can be set to be wide. Also, many objects, their bright side features, and their dark side features can be obtained. This eliminates the need to perform the next image processing step S3 for each image of the multiple sections.

When images are stitched together as described above, it is preferable to perform the noise removal process S3-1 to remove the background of the images. That is, in the image processing step S3 (noise removal process S3-1), a background image ((C) in FIG. 8) is generated by blurring a copy image ((A) in FIG. 8) obtained by copying the acquired grayscale image. The background image is used to generate a grayscale image ((E) in FIG. 8) in which noise has been removed from the original grayscale image. Then, a bright side image and a dark side image are generated from the grayscale image from which noise has been removed. This method makes it possible to reduce noise that has entered the image due to the influence of the microscope (lens) and noise that has been generated by stitching together the section images as described above.

Furthermore, in the feature calculation step S4 of the present disclosure, the deletion process (1) or the reduction process (2) is performed on the bright side feature. The deletion process (3) or the reduction process (4) is performed on the dark side feature. As a result, some of the bright side feature and some of the dark side feature are deleted, and then a learning model is generated. By using the learning model produced in this manner, the accuracy of estimating the physical property values and the quality level values is further improved.

[Quality evaluation system and quality evaluation method]
A quality evaluation system and a quality evaluation method for evaluating the quality of an evaluation target component using the learning model manufactured (constructed) as described above will be described. The evaluation target component is made of the same carbon steel or alloy steel as the target component used to generate the learning model, and has been subjected to the same heat treatment as the target component.

In the present disclosure, the "quality evaluation" can be performed as follows.
That is, at least an estimated value of the carbon concentration in the evaluation target component is obtained, and quality evaluation can be performed based on the estimated value of the carbon concentration.
The estimated hardness, the estimated amount of retained austenite, and the estimated quality level value can be optionally added as quality evaluation items.

The quality evaluation method is outlined below.
That is, a microscope image of the evaluation target component is acquired, and dark side feature quantities and light side feature quantities related to the evaluation target component are acquired by the same means as in the case of constructing the learning model. These dark side feature quantities and light side feature quantities are used as input data to perform calculations using the learning model, and estimates of physical property values and quality level values are obtained. In this manner, quality evaluation of the evaluation target component is performed. The quality evaluation method will be specifically described below.

Fig. 22 is a flow diagram illustrating the quality evaluation method. As shown in Fig. 22, the quality evaluation method includes an image acquisition step S11, an image processing step S12, a feature amount calculation step S13, and an evaluation calculation step S14.
A quality evaluation system that performs the quality evaluation method shown in Fig. 22 can be configured with the image acquisition device 11 and computer device 12 provided in the learning system shown in Fig. 2. In this case, the quality evaluation system includes the image acquisition device 11, an image processing unit 16, a feature calculation unit 17, and an evaluation calculation unit 19.

In the image acquisition process S11, the evaluation target part TP0 is cut, the cut surface is mirror-polished, and etched with a specified corrosive liquid, after which a microscopic image of the metal structure is acquired by an image acquisition device. The image acquisition device is the same as the image acquisition device 11 (see FIG. 2) used in the learning model manufacturing method. The processing performed in the image acquisition process S11 is the same as the image acquisition process S2 performed in the learning model manufacturing method (see FIG. 1), and the same processing as that in the image processing process S2 is also performed in the image acquisition process S11 of the quality evaluation method. This allows a microscopic image of a target area at a specified depth from the surface of the evaluation target part TP0 to be acquired.

In the image processing step S12, a bright side image in which the grains of the tissue of the evaluation target part TP0 are shown as bright objects and the rest are shown as dark areas, and a dark side image in which the grains of the tissue of the evaluation target part TP0 are shown as dark objects and the rest are shown as bright areas are generated from a grayscale image based on the microscope image. The processing performed in the image processing step S12 is the same as the image processing step S3 performed in the manufacturing method for the learning model (see FIG. 1), and the same processing as that in the image processing step S3 is also performed in the image processing step S12 of the quality evaluation method. The generation of the bright side image and the dark side image is performed by an image processing unit, which is one of the functional parts of the computer device 12, and this image processing unit is the same as the image processing unit 16 (see FIG. 2) used in the manufacturing method for the learning model.

In the feature amount calculation step S13, bright side features of the objects included in the bright side image and dark side features of the objects included in the dark side image are calculated. The process performed in the feature amount calculation step S13 is the same as the feature amount calculation step S4 performed in the learning model manufacturing method (see FIG. 1), and the same process as the feature amount calculation step S4 is also performed in the feature amount calculation step S13 of the quality evaluation method. The process of calculating the bright side features and the dark side features is performed by a feature amount calculation unit, which is one of the functional units of the computer device 12, and the feature amount calculation unit is the same as the feature amount calculation unit 17 (see FIG. 2) used in the learning model manufacturing method.

As described above, in the present disclosure, the learning models constructed are a learning model for estimating carbon concentration, a learning model for estimating hardness, a learning model for estimating the amount of retained austenite, and a learning model for estimating quality level values. These learning models are stored in the storage device 14 of the computer device 12.

In the evaluation calculation process S14, the bright side feature values and the dark side feature values obtained in the feature calculation process S13 are used as a data set, and an estimated value of the physical property value of the evaluation target part TP0 is output by the learning model generated by the learning model manufacturing method (see FIG. 1).
In this case, it is not necessary to explain, but when a learning model for estimating the carbon concentration is used, an estimated value of the carbon concentration is obtained. The same applies to the retained austenite and hardness.
In the evaluation calculation step S14, the bright side feature amount and the dark side feature amount obtained in the feature amount calculation step S13 are used as a data set, and an estimated value of the quality level value (e.g., a score value on a 5-point scale) of the evaluation target part TP0 is output by the learning model generated by the learning model manufacturing method (see FIG. 1). At this time, the learning model for estimating the quality level value is used.
The evaluation calculation step S14 is performed by an evaluation calculation unit 19 which is one of the functional units of the computer device 12.

As described above, according to the present disclosure, by using the learning model constructed by the learning system shown in Fig. 2, the accuracy of estimating the physical property values of the evaluation target part TP0 is improved, and the accuracy of estimating the quality level value of the evaluation target part TP0 is also improved. As a result, it is possible to improve the accuracy of the quality evaluation of the evaluation target part TP0.
Furthermore, by using a learning model for estimating carbon concentration, it is possible to improve the accuracy in estimating the carbon concentration of the evaluation target component.
Furthermore, if the carbon concentration is estimated using this learning model, there is no need to use an apparatus such as an EPMA, which reduces costs and shortens the measurement time.
As a result, quality evaluation of the heat-treated product can be performed with high accuracy at low cost and in a short time.

[About the accuracy of carbon concentration estimation]
Next, a verification test on the estimation accuracy of the carbon concentration conducted by the inventors of the present application will be described.
The following learning model was prepared as a learning model for estimating carbon concentration to be used in the validation test.
Example 1: The bright side feature amount includes the circularity of the object, and the dark side feature amount does not include the median brightness of the object.
Example 2: The bright side feature does not include the circularity of the object, and the dark side feature includes the median brightness of the object.
Example 3: The bright side feature amount includes the circularity of the object, and the dark side feature amount includes the median brightness of the object.
Comparative example: The bright side feature does not include the circularity of the object, and the dark side feature does not include the median brightness of the object.

Among the features in the learning models of Examples 1 to 3 and the comparative example, the features included other than the features described above were all the same.
Using the learning models of Examples 1 to 3 and the comparative example, the carbon concentrations of the evaluation target components (target components TP1 to TP5) whose carbon concentrations were known by actual measurements were estimated, and the estimation accuracy was evaluated by comparing the actual carbon concentration values with the estimated values.

FIG. 23(a) is a graph showing an example of the results of determining the coefficient of determination for the estimated values of carbon concentrations obtained using the learning models of Examples 1 to 3 and the comparative example.
FIG. 23(a) shows the coefficients of determination for each of Examples 1 to 3 and the comparative example.
The coefficient of determination is a coefficient that indicates how closely the predicted value of the dependent variable (estimated value of carbon concentration) matches the actual value of the dependent variable (actual measured value of carbon concentration), and the closer the value is to 1, the closer the predicted value and the value of the dependent variable match.
As shown in FIG. 23(a), it can be seen that the coefficients of determination in Examples 1 to 3 are all larger than the coefficient of determination in the comparative example.

FIG. 23(b) is a graph showing an example of the results of calculating RMSE (Root Mean Square Error) for the estimated values of carbon concentrations obtained using the learning models of Examples 1 to 3 and the Comparative Example.
FIG. 23(b) shows the RMSE of each of Examples 1 to 3 and the comparative example.
RMSE is an index showing the deviation between the predicted value of the dependent variable (estimated value of carbon concentration) and the actual value of the dependent variable (actual measured value of carbon concentration), and the smaller the RMSE, the smaller the deviation.
As shown in FIG. 23(b), it can be seen that the RMSE of each of Examples 1 to 3 is smaller than the RMSE of the comparative example.

The results of the above verification tests clearly show that the estimation accuracy of carbon concentration in Examples 1 to 3 is higher than that of the comparative example.

The embodiments disclosed herein are illustrative in all respects and are not restrictive.
For example, the above-mentioned method for manufacturing a learning model has been exemplified as obtaining a learning model for estimating physical property values (carbon concentration, amount of retained austenite, and hardness) and a learning model for estimating quality level values, but the above method can be configured to obtain at least a learning model for estimating carbon concentration.

The scope of the present invention is not limited to the above-described embodiment, but includes all modifications within the scope of the claims.

16: Image processing unit 17: Feature amount calculation unit 18: Learning unit 19: Evaluation calculation unit S1: Preliminary process S2: Image acquisition process S3: Image processing process S4: Feature amount calculation process S5: Model generation process S11: Image acquisition process S12: Image processing process S13: Feature amount calculation process S14: Evaluation calculation process TP1 to TP5: Target member

Claims (5)

A manufacturing method of a learning model used for quality evaluation of a component made of heat-treated carbon steel or alloy steel, comprising:
an image acquisition step of acquiring a microscopic image of a target region at a predetermined depth from a surface of a target component made of heat-treated carbon steel or alloy steel and having a carbon concentration actually measured;
an image processing step of generating a bright side image in which the grains of the tissue of the target material are shown as bright objects and other areas are shown as dark areas from a grayscale image based on the microscope image, and a dark side image in which the grains of the tissue of the target material are shown as dark objects and other areas are shown as bright areas;
a feature amount calculation step of calculating bright side feature amounts of the plurality of objects included in the bright side image and dark side feature amounts of the plurality of objects included in the dark side image;
a model generating step of generating a learning model that outputs an estimated value of the carbon concentration from the bright side feature amount and the dark side feature amount of the component to be evaluated, by using the bright side feature amount and the dark side feature amount and an actual measured value of the carbon concentration of the target component as teacher data;
Including,
The bright side feature amount and the dark side feature amount calculated in the feature amount calculation step include at least one of a circularity of an object included in the bright side image and a representative value of the brightness obtained from a frequency distribution of the brightness of pixels constituting the object included in the dark side image. The method for manufacturing a learning model used for quality evaluation according to claim 1 , wherein the representative value is a median value. A learning system for producing a learning model by machine learning to be used for quality evaluation of a component made of heat-treated carbon steel or alloy steel,
an image acquisition device for acquiring a microscopic image of a target region at a predetermined depth from a surface of a target component made of heat-treated carbon steel or alloy steel and having a carbon concentration actually measured;
an image processing unit that generates, from a grayscale image based on the microscope image, a bright side image in which the grains of the tissue of the target material are shown as bright objects and other areas are shown as dark regions, and a dark side image in which the grains of the tissue of the target material are shown as dark objects and other areas are shown as bright regions;
a feature amount calculation unit that calculates bright side feature amounts of the plurality of objects included in the bright side image and dark side feature amounts of the plurality of objects included in the dark side image;
a learning unit that uses the bright side feature amount and the dark side feature amount and an actual measured value of the carbon concentration of the target component as teacher data to train a learning model so as to output an estimated value of the carbon concentration from the bright side feature amount and the dark side feature amount of the component to be evaluated;
Including,
a learning system in which the bright side feature and the dark side feature calculated by the feature calculation unit include at least one of a circularity of an object included in the bright side image and a representative value of the brightness obtained from a frequency distribution of the brightness of a plurality of pixels constituting the object included in the dark side image. A quality evaluation system for evaluating the quality of an evaluation target component made of heat-treated carbon steel or alloy steel, comprising:
an image acquisition device for acquiring a microscopic image of a target region at a predetermined depth from a surface of the evaluation target component;
an image processing unit that generates, from a grayscale image based on the microscope image, a bright side image in which grains of the texture of the evaluation target component are shown as bright objects and other areas are shown as dark regions, and a dark side image in which grains of the texture of the evaluation target component are shown as dark objects and other areas are shown as bright regions;
a feature amount calculation unit that calculates bright side feature amounts of the plurality of objects included in the bright side image and dark side feature amounts of the plurality of objects included in the dark side image;
an evaluation calculation unit that uses the bright side feature amount and the dark side feature amount as a data set, and outputs an estimated value of the carbon concentration of the evaluation target component according to a learning model obtained by learning with the learning system according to claim 3;
A quality evaluation system comprising: A quality evaluation method for evaluating the quality of an evaluation target component made of heat-treated carbon steel or alloy steel, comprising:
an image acquisition step of acquiring a microscope image of a target region at a predetermined depth from a surface of the evaluation target component;
an image processing step of generating, from a grayscale image based on the microscope image, a bright side image in which the grains of the texture of the evaluation target component are shown as bright objects and other areas are shown as dark areas, and a dark side image in which the grains of the texture of the evaluation target component are shown as dark objects and other areas are shown as bright areas;
a feature amount calculation step of calculating bright side feature amounts of the plurality of objects included in the bright side image and dark side feature amounts of the plurality of objects included in the dark side image;
an evaluation calculation step of outputting an estimated value of the carbon concentration of the evaluation target component by a learning model generated by the manufacturing method according to claim 1 or 2 using the bright side feature amount and the dark side feature amount as a data set;
Quality assessment methods including:

Images