JP7741370B2 - Fracture surface characteristic region determination model generating device, fracture surface characteristic region determination model generating method, program, fracture surface characteristic region determination device, and fracture surface characteristic region determination method - Google Patents

Description

The present invention relates to a fracture surface characteristic region determination model generation device, a fracture surface characteristic region determination model generation method, a program, and a fracture surface characteristic region rate calculation method using these.

Fracture tests such as the Charpy impact test and the DWTT (Drop Weight Tear Test) are used to evaluate the toughness of metals such as steel products. Commonly used indicators for measuring metal toughness are the ductile surface ratio, which is the ratio of ductile surfaces to the fracture surface area, and the brittle surface ratio, which is the ratio of brittle surfaces to the fracture surface area. Hereinafter, the "ductile surface ratio" and "brittle surface ratio" will be collectively referred to as the "fracture surface ratio."

When measuring the fracture surface ratio, it is necessary to determine which parts of the fracture surface are ductile and which are brittle, but this judgment requires skill and is subject to variability depending on the person. In addition, because the boundary region between the ductile and brittle parts has a complex shape, accurately calculating the fracture surface ratio takes a lot of time. Specifically, the conventional method involves overlaying a grid on an image of the fracture surface, counting the squares in the area corresponding to the ductile or brittle surface within the fracture surface, and calculating the fracture surface ratio as the proportion of these areas on the fracture surface.

In light of this situation, for example, Patent Document 1 discloses a method for automatically analyzing fracture surfaces by determining their characteristics through image processing.

Japanese Unexamined Patent Publication No. 143770/1983

Kenta Yamagiwa, "Introduction to the Use of Deep Learning in Fracture Surface Analysis," Journal of the Society of Materials Science, Japan, Vol. 69, No. 9, pp. 644-649, September 2020 Takayuki Okatani, "Research Trends in Deep Learning for Image Recognition: Development of Convolutional Neural Networks and Their Applications," Journal of the Japanese Society for Artificial Intelligence, Vol. 31, No. 2, pp. 169-179, March 2016. Dang Ha, "A guide to receptive field arithmetic for convolutional neural networks" [Retrieved April 16, 2021], Internet <URL: https://medium.com/mlreview/a-guide-to-receptive-field-arithmetic-for-convolutional-neural-networks-e0f514068807> Olaf Ronneberger and two others, "U-Net: Convolutional Networks for Biomedical Image Segmentation" [Retrieved April 16, 2021], Internet <URL: https://arxiv.org/abs/1505.04597>

Meanwhile, in many industrial fields, there is a growing trend to use classifiers trained using machine learning techniques to extract experience and knowledge from massive amounts of data and use this knowledge to automate processes. In particular, in the field of image recognition, classifiers based on neural networks, including deep learning, have been shown to dramatically improve the accuracy of problems such as image classification and semantic segmentation, which have long been important issues.

For example, Non-Patent Document 1 discloses an example of the use of deep learning in fracture surface analysis. Specifically, Non-Patent Document 1 describes the creation of a deep learning model that classifies fracture surfaces in SEM images by training a multilayer neural network using scanning electron microscope (SEM) images of fracture structures such as dimples, facets, and striations as training data. However, since obtaining SEM images requires a scanning electron microscope and takes time to capture, it would be desirable to be able to automatically determine the ductile and brittle surfaces of fracture surfaces from more easily obtainable macroscopic fracture surface images (i.e., fracture surface images that are more macroscopic than SEM images).

However, the method in Non-Patent Document 1 cannot be applied to the analysis of macroscopic fracture surface images. This is because the output desired from the analysis of macroscopic fracture surface images is not a discrete classification such as striation, but a continuously changing quantity such as 0 to 100% indicating the proportion of brittle surfaces. Therefore, it might be possible to change the output of the deep learning model described in Non-Patent Document 1 from a discrete value to a continuous value, but experiments by the inventors of this application have shown that simply changing the output to a continuous value does not result in sufficient accuracy.

The present invention has been made in consideration of the above-mentioned problems, and its object is to provide a fracture surface characteristic region determination model generation device, a fracture surface characteristic region determination model generation method, a program, and a method for calculating the characteristic region rate of a fracture surface using these, which generate a model that can automatically and accurately determine the ductile and brittle surfaces of a fracture surface from macroscopic fracture surface images that are easier to obtain than SEM images.

In order to solve the above problem, one aspect of the present invention provides a fracture surface characteristic region determination model generation device that includes a model generation unit that generates a fracture surface characteristic region determination model that determines characteristic regions in fracture surface images through machine learning using multiple pieces of training data that associate fracture surface images of a material with information representing the positions of characteristic regions in the fracture surface images, and an output unit that outputs the fracture surface characteristic region determination model.

The model generation unit may generate a fracture surface characteristic region determination model through deep learning by setting the unit region considered in determining the characteristic region of the fracture surface image to an area larger than the minimum necessary area capable of identifying fractured tissue.

The original image obtained by imaging the fracture surface of a material may be an image captured by an imaging device in which the fracture surface of the material is illuminated symmetrically with respect to the optical axis of the imaging device.

The fracture surface characteristic region determination model generation device may have an image multiplication unit that generates multiple fracture surface images from a single fracture surface image. The image multiplication unit multiplies the fracture surface images by flipping the original image obtained by capturing the fracture surface of the material horizontally, vertically, or rotating it, and the model generation unit uses the original image and the fracture surface images multiplied by the image multiplication unit as fracture surface images for the training data.

The characteristic region may be a brittle or ductile surface.

The fracture surface characteristic region determination model may determine the characteristic region of the fracture surface image input to the fracture surface characteristic region determination model, and output a determined fracture surface image showing the characteristic region on the fracture surface image.

The model generation unit may re-train the fracture surface characteristic region determination model using, as training data, a corrected fracture surface image in which the user has corrected the characteristic region of the determined fracture surface image obtained using the fracture surface characteristic region determination model output from the output unit.

In order to solve the above problem, according to another aspect of the present invention, there is provided a fracture surface characteristic region determination model generation method including: a model generation step for generating a fracture surface characteristic region determination model for determining characteristic regions in a fracture surface image through machine learning, using as learning data a plurality of data in which fracture surface images of a material are associated with information representing the positions of characteristic regions in the fracture surface images; and an output step for outputting the fracture surface characteristic region determination model.

Furthermore, to solve the above-mentioned problems, according to another aspect of the present invention, there is provided a program that causes a computer to function as a fracture surface characteristic region determination model generation device, comprising: a model generation unit that generates a fracture surface characteristic region determination model for determining characteristic regions in a fracture surface image through machine learning using, as learning data, a plurality of data sets in which fracture surface images of a material are associated with information representing the positions of characteristic regions in the fracture surface images; and an output unit that outputs the fracture surface characteristic region determination model.

In order to solve the above problem, according to another aspect of the present invention, there is provided a method for calculating a characteristic area ratio of a fracture surface, including: a fracture surface image acquisition step of capturing an image of the fracture surface of a material to acquire a fracture surface image; a characteristic area determination step of determining a characteristic area in the fracture surface image acquired in the fracture surface image acquisition step using a fracture surface characteristic area determination model generated by the fracture surface characteristic area determination model generation device, and outputting a determined fracture surface image showing the characteristic area on the fracture surface image; and a calculation step of calculating the ratio of the characteristic area in the fracture surface based on the determined fracture surface image.

As described above, according to the present invention, machine learning is performed using training data that associates fracture surface images with information representing the positions of characteristic regions (ductile and brittle surfaces of the fracture surface) in the fracture surface images. This makes it possible to generate a model that can automatically and accurately determine the ductile and brittle surfaces of the fracture surface, even for macro fracture surface images that are easier to obtain than SEM images.

1 is an example of a fracture surface image of a metal test piece obtained by a Charpy impact test, showing an original image and a fracture surface image showing a brittle surface portion. 1 is a block diagram showing an example of the configuration of a fracture surface characteristic region determination system according to an embodiment of the present invention; 10 is a flowchart showing an example of a fracture surface characteristic region determination model generating method according to the embodiment; FIG. 10 is an explanatory diagram illustrating a multiplication process of fracture surface images. FIG. 10 is an explanatory diagram illustrating a method for checking the size of a Receptive Field. 10 is a flowchart showing an example of a method for calculating a characteristic area rate of a fracture surface according to the embodiment; FIG. 10 is a schematic diagram showing an example of an image of a fracture surface captured by an imaging device. 10A and 10B are schematic diagrams showing other examples of images of fracture surfaces taken by an imaging device. FIG. 2 is a block diagram showing an example of a hardware configuration of an information processing device that functions as a fracture surface characteristic region determination model generation device or a fracture surface characteristic region determination device according to the embodiment. 10 is a graph showing the results of a comparison between the fracture surface rate (user-determined) and the fracture surface rate (model-determined), as an example. As an example, the images show an example of the results of determining the brittle surface of an original image by a person and the results of determining the brittle surface using a fracture surface characteristic region determination model.

Preferred embodiments of the present invention will be described in detail below with reference to the accompanying drawings. Note that in this specification and drawings, components having substantially the same functional configuration will be designated by the same reference numerals, and redundant explanations will be omitted.

[1. Overview]
First, an outline of one embodiment of the present invention will be described with reference to Fig. 1. Fig. 1 shows an example of a fracture surface image of a metal test piece obtained by a Charpy impact test, and shows an original image and a fracture surface image showing a brittle surface portion.

The original image shown in Figure 1 is a fracture surface image obtained by capturing an image of the fracture surface of a metal test piece obtained through a Charpy impact test using an imaging device. In order to calculate the fracture surface ratio, which is the ratio of the area of the ductile or brittle surface to the area of the fracture surface, it is necessary to identify the ductile or brittle surface in the original image.

Traditionally, the determination of ductile or brittle surfaces in fracture surface images of materials has been performed manually, for example, as follows: First, image editing software is used to mark the ductile or brittle surface areas on the fracture surface image, resulting in a determined fracture surface image as shown in the center of Figure 1. The black areas in the determined fracture surface image in the center of Figure 1 represent areas determined by the human to be brittle surfaces. Humans would overlay a grid on the determined fracture surface image and count the squares in the area corresponding to the brittle surface to calculate the brittle surface ratio. However, determining ductile or brittle surfaces in fracture surface images requires skill, and variations occur depending on the person. In addition, because the boundary area between the ductile and brittle areas has a complex shape, accurately calculating the fracture surface ratio takes a long time.

Therefore, in the method according to this embodiment, a plurality of data sets in which fracture surface images of a material are associated with information representing the positions of characteristic regions (e.g., ductile surfaces, brittle surfaces) in the fracture surface images are used as training data to generate a fracture surface characteristic region determination model that determines the characteristic regions of the fracture surface images through machine learning. That is, in the method according to this embodiment, for example, a post-determined fracture surface image such as that shown in the center of Figure 1 is used as training data. By inputting a newly acquired fracture surface image of a material into the fracture surface characteristic region determination model, it is possible to obtain a post-determined fracture surface image in which characteristic regions are marked on the fracture surface image, such as that shown in the lower part of Figure 1.

As mentioned above, Non-Patent Document 1 describes the creation of a deep learning model that uses SEM images of a material's fracture surface as training data to train a multi-layer neural network and classify the fracture surfaces in the SEM images. The characteristics of a material's fracture surface can be classified into macroscopic and microscopic features. The method described in Non-Patent Document 1 focuses on the fracture structure pattern observed in the SEM image, i.e., the microscopic features. For example, if a dimple pattern appears in the SEM image, it is determined to be a ductile surface, and if a river pattern appears in the SEM image, it is determined to be a brittle surface.

In contrast, the method according to this embodiment uses macroscopic fracture surface images, which are easier to obtain than SEM images, to generate a fracture surface characteristic region determination model that determines the characteristic region of a fracture surface image. The method according to this embodiment focuses on the characteristics of the fracture surface of the material from a macroscopic perspective, and determines the characteristic region of the fracture surface from the color or shape (pattern) that can be visually confirmed. For example, a ductile surface is a dull grayish-white, while a brittle surface is a shiny silvery-white. In this way, the method according to this embodiment uses fracture surface images that capture a wider region as training data, rather than fracture surface images that can identify fracture structure on a micron level. For this reason, in this embodiment, the pixels to be used to determine the characteristic region in the fracture surface image represent an area that is wider than the minimum necessary area for identifying fracture structure.

The method according to this embodiment is explained in detail below.

2. System Configuration
First, a fracture surface characteristic region determining system 1 according to one embodiment of the present invention will be described with reference to Fig. 2. Fig. 2 is a block diagram showing an example of the configuration of the fracture surface characteristic region determining system 1 according to this embodiment.

As shown in FIG. 2, the fracture surface characteristic region determination system 1 according to this embodiment includes a fracture surface characteristic region determination model generation device 100, a fracture surface characteristic region determination device 200, an image storage device 300, and an imaging device 400.

(Device for generating a model for determining fracture surface characteristic regions)
The fracture surface characteristic region determination model generating device 100 generates a fracture surface characteristic region determination model for determining a characteristic region of a fracture surface image. The fracture surface characteristic region determination model generating device 100 includes a model generating unit 110, an output unit 120, and an image multiplying unit 130.

The model generation unit 110 generates a fracture surface characteristic region determination model through machine learning, using as training data multiple data sets in which fracture surface images of a material are associated with information representing the positions of characteristic regions in the fracture surface images. Here, information representing the positions of characteristic regions in the fracture surface images refers to information that can display an area marked as a characteristic region, as shown in the fracture surface image after determination in Figure 1. For example, the information representing the positions of characteristic regions in the fracture surface images may be image data, or information that identifies pixel positions in the fracture surface images. The fracture surface images may be image data such as jpg, bmp, or png.

The model generation unit 110 acquires multiple pieces of data to be used as training data from the image storage device 300, which will be described later. The model generation unit 110 uses deep learning as machine learning to generate a fracture surface characteristic region determination model, which is a deep learner, from the training data. Details of the model generation process performed by the model generation unit 110 will be described later. The model generation unit 110 outputs the generated fracture surface characteristic region determination model to the output unit 120.

The output unit 120 outputs the fracture surface characteristic region determination model to an external device. For example, the output unit 120 outputs the fracture surface characteristic region determination model to the fracture surface characteristic region determination device 200.

The image multiplication unit 130 generates multiple fracture surface images from a single fracture surface image. The model generation process by the model generation unit 110 requires a large amount of training data. In order to efficiently obtain a large number of fracture surface images in which the positions of characteristic regions are identified, the image multiplication unit 130 rotates or flips a single fracture surface image to generate multiple fracture surface images. In this case, the image multiplication unit 130 also rotates or flips information indicating the positions of characteristic regions in the fracture surface image in the same way as the fracture surface image, so that it can be used as training data for deep learning. In addition to rotation and flipping, other methods of multiplication may include changing the brightness or image size.

The image multiplication unit 130 acquires, for example, fracture surface images stored in the image storage device 300 and information representing the positions of characteristic regions in the fracture surface images, and generates multiple fracture surface images and information representing the positions of characteristic regions in the fracture surface images from the acquired fracture surface images and information representing the positions of characteristic regions in the fracture surface images, and records these in the image storage device 300. Note that in this embodiment, the image multiplication unit 130 is provided in the fracture surface characteristic region determination model generation device 100, but the present invention is not limited to this example and may be provided in a different device. Also, for example, images may be multiplied each time a fracture surface image and information representing the positions of characteristic regions in the fracture surface image are read from the image storage device 300 during model generation. In this case, the image multiplication unit 130 is included in the model generation unit 110.

(Fracture surface characteristic region determination device)
The fracture surface characteristic region determining device 200 determines a characteristic region in a fracture surface image using a fracture surface characteristic region determination model. The fracture surface characteristic region determining device 200 includes a determining unit 210, a correction processing unit 220, a ratio calculation unit 230, a display unit 240, an input unit 250, and a data output unit 260.

The determination unit 210 determines the characteristic regions in the fracture surface image using the fracture surface characteristic region determination model, and outputs a determined fracture surface image showing the characteristic regions on the fracture surface image. The determination unit 210 determines the characteristic regions of a fracture surface image of a material newly acquired by the imaging device 400 using the fracture surface characteristic region determination model input from the fracture surface characteristic region determination model generation device 100. The determination unit 210 outputs the determined fracture surface image showing the characteristic regions on the fracture surface image to the correction processing unit 220, ratio calculation unit 230, display unit 240, and data output unit 260. The determined fracture surface image is data in which the original fracture surface image is associated with information indicating the position of the characteristic regions in the fracture surface image.

The correction processing unit 220 corrects the characteristic regions in the fracture surface image determined by the determination unit 210 based on a correction instruction from the user. When the characteristic regions in the fracture surface image determined by the determination unit 210 are displayed on the display unit 240, the user can correct the characteristic regions in the fracture surface image via the input unit 250. When a correction instruction from the user is input from the input unit 250, the correction processing unit 220 corrects the characteristic regions in the fracture surface image based on the correction instruction. The correction processing unit 220 outputs the corrected fracture surface image in which the characteristic regions have been corrected to the display unit 240. The correction processing unit 220 also outputs the corrected fracture surface image in which the characteristic regions have been corrected to the ratio calculation unit 230 and the data output unit 260. The corrected fracture surface image is data in which the original image of the corrected fracture surface image is associated with information indicating the position of the characteristic regions in the corrected fracture surface image. If the determination unit 210 determines the characteristic regions with sufficiently high accuracy, the correction by the correction processing unit 220 may be omitted.

The ratio calculation unit 230 calculates the ratio of characteristic regions on the fracture surface of the fracture surface image (hereinafter also referred to as the "characteristic region ratio"). When a fracture surface image in which the position of the characteristic region has been identified (a post-determination fracture surface image or a post-correction fracture surface image) is input from the determination unit 210 or the correction processing unit 220, the ratio calculation unit 230 calculates the characteristic region ratio and outputs the calculation result to the display unit 240.

The display unit 240 is an output unit that displays information, such as a display device. The display unit 240 displays the fracture surface image (determined fracture surface image or corrected fracture surface image) in which the position of the feature region has been identified by the determination unit 210 or the correction processing unit 220, the feature region rate calculated by the rate calculation unit 230, etc., and notifies the user.

The input unit 250 is an interface unit that accepts instructions from the user, and is an operating device operated by the user, such as a mouse, keyboard, or touch panel. The user can operate the input unit 250 to correct the characteristic area of the fracture surface image. The input unit 250 outputs input information, such as the input correction instructions, to the correction processing unit 220.

The data output unit 260 records the determined fracture surface image input from the determination unit 210 and the corrected fracture surface image input from the correction processing unit 220 in the image storage device 300. The data output from the data output unit 260 and recorded in the image storage device 300 can be used as learning data for the fracture surface characteristic region determination model generation device 100.

(Image storage device)
The image storage device 300 is a storage device that stores a plurality of data items associated with fracture surface images of a material and information indicating the positions of characteristic regions in the fracture surface images. The image storage device 300 stores, in advance, determined fracture surface images in which a human has determined and marked characteristic regions on the fracture surface images of the material. The image storage device 300 can also store determined fracture surface images and corrected fracture surface images output from the fracture surface characteristic region determination device 200. The data stored in the image storage device 300 is used as learning data for the model generation process performed by the fracture surface characteristic region determination model generation device 100. The image storage device 300 also stores fracture surface images multiplied by the fracture surface characteristic region determination model generation device 100 and information indicating the positions of characteristic regions in the multiplied fracture surface images.

(Imaging device)
The image capturing device 400 captures an image of a fracture surface of a material. The image captured by the image capturing device 400 is output to the fracture surface characteristic region determination device 200 as a fracture surface image with a resolution (macro viewpoint) that allows the characteristics of the fracture surface of the material to be visually determined.

The fracture surface characteristic region determination system 1 according to this embodiment has been described above. Note that the configuration of each device in the fracture surface characteristic region determination system 1 shown in Figure 2 is an example; a single device may have the functions of multiple devices, or multiple functions included in a single device may be performed by different devices.

[3. Method for generating a model for determining fracture surface characteristic regions]
Next, a method for generating a fracture surface characteristic region determination model according to this embodiment will be described with reference to Figs. 3 to 5. Fig. 3 is a flowchart showing an example of the method for generating a fracture surface characteristic region determination model according to this embodiment. Fig. 4 is an explanatory diagram illustrating the multiplication process for fracture surface images. Fig. 5 is an explanatory diagram illustrating a method for checking the size of the receptive field.

(S110: Acquire learning data)
3, the fracture surface characteristic region determination model generating device 100 first acquires, as learning data, a plurality of data sets each associating a fracture surface image of a material with information indicating the position of a characteristic region in the fracture surface image from the image storage device 300 (S110). Previously acquired data sets are stored in the image storage device 300. The fracture surface images of these data sets are preferably images captured by the same imaging device as the imaging device 400 that captures the fracture surface images input to the fracture surface characteristic region determination device 200, but may also be images captured by a different imaging device.

Here, a large amount of training data is required for the model generation process described below. Therefore, in order to efficiently obtain a large number of fracture surface images in which the positions of characteristic regions are identified, multiple fracture surface images may be generated from a single fracture surface image, and the proliferated fracture surface images may also be used as training data. In this case, the image proliferator 130 of the fracture surface characteristic region determination model generation device 100 acquires, for example, a fracture surface image recorded in the image storage device 300, and rotates or flips the acquired fracture surface image to generate multiple fracture surface images. Specifically, as shown in FIG. 4, for example, the image proliferator 130 generates multiple fracture surface images by flipping the original image horizontally, flipping it vertically, or rotating it by a predetermined angle (for example, rotating it 90 degrees to the right). This allows a large number of fracture surface images to be obtained efficiently.

(S120: Model generation)
Next, the model generation unit 110 of the fracture surface characteristic region determination model generation device 100 generates a fracture surface characteristic region determination model through machine learning using the training data acquired in step S110 (S120). The model generation unit 110 uses the fracture surface images of the training data as input information and optimizes the model so that information representing the positions of the characteristic regions output by the generated fracture surface characteristic region determination model approaches the information representing the positions of the characteristic regions in the training data.

In this embodiment, deep learning is used as the machine learning method. Here, we will explain the size of the receptive field (also called receptive field) in a deep learning model. A convolutional neural network (CNN), which is part of the deep learning network structure, repeats convolution operations to determine the relationship between a given pixel and its surrounding pixels. The size of the receptive field is a quantity that represents how far away information from pixels is ultimately used when determining whether a pixel is a feature region (see, for example, Non-Patent Document 2).

In this embodiment, when using deep learning, the unit area (i.e., the receptive field) considered in determining the characteristic region of a fracture surface image is set to an area wider than the minimum necessary area for identifying fractured tissue. As can be seen from the image sizes exemplified in Non-Patent Document 1, the minimum necessary area for identifying fractured tissue is on the order of 10 μm, but it is desirable to set the receptive field larger than this. There is a high probability that ductile surfaces exist around ductile surfaces, and that brittle surfaces exist around brittle surfaces. In other words, ductile surfaces and brittle surfaces tend to cluster together. Therefore, as in the model generation process of this embodiment, the receptive field is set wider than conventional methods, and the characteristic region is determined by taking into account the characteristics of the fracture surface of pixels located as far away as possible from the pixel being determined, thereby improving the accuracy of the determination.

The size of the receptive field is determined by the CNN used. Figure 5 shows how to determine the size of the receptive field when a CNN consisting of a convolutional layer, a pooling layer, a transposed convolutional layer, and an activation layer is given. Figure 5 shows a schematic CNN, an input image G _in to the CNN, and an output image G _out from the CNN.

First, the weight of the convolutional layer (transposed convolutional layer) constituting the CNN is set to "1.0", the bias is set to "0.0", and the activation layer is removed. Next, an input image G in is prepared in which only one central pixel (area A in Figure 5) is set to a value other than "0" and the other pixels are set to a value of "0". When the input image G _{in is} then input to the CNN, an output image G _out of the same image size as the input image G _in is obtained. In this output image G _out , the size of the region of pixels with values other than "0.0" (area B in Figure 5) becomes the size of the receptive field.

Note that the size of the receptive field can also be confirmed by methods other than those shown in Figure 5 (for example, Non-Patent Document 3).

Once the model generation unit 110 generates a fracture surface characteristic region determination model through deep learning, it outputs the fracture surface characteristic region determination model to the output unit 120.

(S130: Model output)
The output unit 120 of the fracture surface characteristic region determination model generating device 100 outputs the fracture surface characteristic region determination model generated in step S120 to the fracture surface characteristic region determining device 200 (S130).

The above describes the method for generating a model for determining fracture surface characteristic regions according to this embodiment. The method for generating a model for determining fracture surface characteristic regions according to this embodiment makes it possible to generate a model that can automatically determine characteristic regions such as ductile surfaces and brittle surfaces on fracture surfaces from macroscopic fracture surface images, which are easier to obtain than SEM images.

[4. Method for calculating the characteristic area ratio of a fracture surface]
Next, a method for calculating a characteristic area rate of a fractured surface according to this embodiment will be described with reference to Fig. 6 to Fig. 8. Fig. 6 is a flowchart showing an example of the method for calculating a characteristic area rate of a fractured surface according to this embodiment. Figs. 7 and 8 are schematic diagrams showing examples of images of fractured surfaces captured by the imaging device 400.

(S210: Fracture surface image acquisition)
As shown in Fig. 6, first, an image of the fracture surface of the material is captured using an imaging device 400 to obtain a fracture surface image (S210). The fracture surface image is obtained by placing the imaging device 400 opposite the fracture surface 15 of the test piece 10 and capturing the image, as shown in Figs. 7 and 8. At this time, the fracture surface 15 of the test piece 10 is illuminated by an illumination device 410. For example, as shown in Fig. 7, the illumination device 410 may illuminate the fracture surface 15 of the test piece 10 obliquely.

The lighting device 410 may also illuminate the fracture surface 15 of the test piece 10 symmetrically with respect to the optical axis of the imaging device 400. Specifically, in addition to the lighting device 410 on the right side of FIG. 7, a lighting device (left lighting device 410) may be positioned symmetrically to the right lighting device 410 with respect to the optical axis of the imaging device 400. Alternatively, a ring-shaped lighting device may be installed centered on the optical axis of the imaging device 400. Alternatively, as shown in FIG. 8, a half mirror 420 may be installed between the imaging device 400 and the fracture surface 15 of the test piece 10, and the illumination light emitted from the lighting device 410 may be reflected by the half mirror 420 coaxially with the optical axis of the imaging device 400 to illuminate the fracture surface 15 of the test piece 10. In this case, the fracture surface 15 of the test piece 10 can be illuminated in the same way as with a ring-shaped lighting device centered on the optical axis of the imaging device 400.

In this way, by illuminating the fracture surface 15 of the test piece 10 so that it is symmetrical with respect to the optical axis of the imaging device 400 and capturing an image of the fracture surface 15 of the test piece 10 using the imaging device 400, the captured fracture surface images can be multiplied and used as training data to improve model accuracy.

Data amplification is a technique in which the transformed fracture surface images used as training data are also used for training, under the assumption that the output will not change even if certain transformations are performed on them. By amplifying data in this way, it is possible to mitigate the weakness of deep learning, which requires large amounts of training data.

For example, as shown in Figure 4, data can be multiplied by flipping the original image horizontally, upside down, or rotating it by a specified angle (e.g., 90 degrees to the right). If the lighting is not symmetrical, images obtained by flipping the captured fracture surface image horizontally, upside down, or rotating it will not realistically be possible, making it difficult to achieve the effect of improving model accuracy by multiplying data. On the other hand, if the lighting is symmetrical, for example, if a ring-shaped lighting device is used, the obtained fracture surface image can be realistically flipped horizontally, upside down, or rotated. Therefore, compared to when the lighting is not symmetrical, data multiplication as shown in Figure 4 can be expected to have an effect of improving model accuracy.

Furthermore, the color of the illumination light from the lighting device 410 may be white and is not particularly limited, but if light with a shorter wavelength is used, for example, if the color of the illumination light is blue, it is possible to obtain a fracture surface image that captures more detailed features of the fracture surface.

(S220: Characteristic region determination)
Next, the determination unit 210 of the fracture surface characteristic region determination device 200 determines the characteristic region of the fracture surface image acquired in step S210 (S220) using the fracture surface characteristic region determination model generated by the fracture surface characteristic region determination model generation device 100. The determination unit 210 inputs the fracture surface image acquired in step S210 as input information to the fracture surface characteristic region determination model, and obtains a determined fracture surface image indicating the characteristic region on the fracture surface image from the fracture surface characteristic region determination model as output information.

(S230: Display of fracture surface image after determination)
Thereafter, the determination unit 210 outputs the determined fracture surface image obtained as output information from the fracture surface characteristic region determination model to the display unit 240 (S230). The display unit 240 displays the determined fracture surface image showing the characteristic region on the fracture surface image.

(S240: Feature region correction)
When the determined fracture surface image is displayed on the display unit 240, the user corrects the characteristic regions of the determined fracture surface image as necessary (S240). Depending on the accuracy of the fracture surface characteristic region determination model, the characteristic regions may not be correctly determined. In such cases, if the user wishes to make corrections, the user may use image editing software to mark portions of the fracture surface image that are characteristic regions but have not been determined to be characteristic regions, or to delete markings of portions that are not characteristic regions but have been determined to be characteristic regions, thereby correcting the characteristic regions of the determined fracture surface image.

The user's correction instructions input from the input unit 250 are output to the correction processing unit 220. The correction processing unit 220 corrects the characteristic areas in the post-determination fracture surface image determined by the determination unit 210 based on the user's correction instructions. The correction processing unit 220 outputs the fracture surface image with the corrected characteristic areas to the display unit 240 so that the user can confirm the correction details.

(S250: Image recording)
When the user finishes correcting the characteristic region of the determined fracture surface image in step S240, the data output unit 260 records the corrected determined fracture surface image as a corrected fracture surface image in the image storage device 300 (S250). The corrected fracture surface image recorded in the image storage device 300 can be used for relearning the fracture surface characteristic region determination model.

(S260: Calculation of characteristic region rate)
The ratio calculation unit 230 then calculates the ratio of the characteristic region in the fracture surface based on the corrected fracture surface image (S260). The ratio calculation unit 230 calculates the ratio of the area of the characteristic region to the area of the fracture surface in the fracture surface image as the characteristic region ratio (= (area of characteristic region) / (total fracture surface area)). The feature region area and the total fracture surface area may be calculated by counting the number of pixels. The ratio calculation unit 230 outputs the calculated characteristic region ratio to the display unit 240 so that the user can know the characteristic region ratio.

The above describes the method for calculating the characteristic region rate of a fracture surface according to this embodiment. According to this embodiment, the fracture surface characteristic region determination model can be used to determine characteristic regions such as ductile surfaces and brittle surfaces on the fracture surface. Furthermore, the fracture surface characteristic region determination model generated during initial learning may not have a very high level of model accuracy. Even in this case, the user can simply correct the characteristic regions based on the fracture surface image after determination using the fracture surface characteristic region determination model. This significantly reduces the workload and shortens the inspection time required to obtain the characteristic region rate compared to when the user determines the characteristic regions from scratch and marks them on the fracture surface image.

Furthermore, by recording fracture surface images determined by the fracture surface characteristic region determination model and fracture surface images corrected by the user in the image storage device 300, it is possible to use this data for re-learning the fracture surface characteristic region determination model. This makes it easy to prepare a large amount of training data, thereby improving the accuracy of the fracture surface characteristic region determination model.

Note that in the explanation of Figure 6, a case has been described in which the user corrects the feature region of the fracture surface image after judgment using the fracture surface feature region judgment model. However, if the accuracy of the fracture surface feature region judgment model is high and correction of the feature region by the user is not necessary, the processing of step S260 may be such that the feature region rate is calculated based on the fracture surface image after judgment.

5. Hardware Configuration
The hardware configurations of the fracture surface characteristic region determination model generating device 100 and the fracture surface characteristic region determining device 200 according to this embodiment will be described with reference to Fig. 9. Fig. 9 is a block diagram showing an example of the hardware configuration of an information processing device 900 that functions as the fracture surface characteristic region determination model generating device 100 or the fracture surface characteristic region determining device 200 according to this embodiment.

The information processing device 900 includes a processor (CPU 901 in FIG. 9), ROM 903, and RAM 905. The information processing device 900 also includes a bus 907, an input I/F 909, an output I/F 911, a storage device 913, a drive 915, a connection port 917, and a communication device 919.

The CPU 901 functions as an arithmetic processing device and a control device. The CPU 901 controls all or part of the operations within the information processing device 900 in accordance with various programs recorded in the ROM 903, RAM 905, storage device 913, or removable recording medium 925. The ROM 903 stores programs used by the CPU 901 or calculation parameters, etc. The RAM 905 temporarily stores programs used by the CPU 901 or parameters that change as appropriate during program execution. These are interconnected by a bus 907, which is composed of an internal bus such as a CPU bus.

Bus 907 is connected to an external bus such as a PCI (Peripheral Component Interconnect/Interface) bus via a bridge.

The input I/F 909 is an interface that accepts input from an input device 921, which is an operating means operated by a user, such as a mouse, keyboard, touch panel, button, switch, or lever. The input I/F 909 is configured, for example, as an input control circuit that generates an input signal based on information input by the user using the input device 921 and outputs it to the CPU 901. The input device 921 may be, for example, a remote control device that uses infrared or other radio waves, or an external device 927 such as a PDA that supports operation of the information processing device 900. A user of the information processing device 900 can operate the input device 921 to input various data to the information processing device 900 and instruct processing operations.

The output I/F 911 is an interface that outputs input information to an output device 923 that can notify the user visually or audibly. The output device 923 may be, for example, a display device such as a CRT display device, a liquid crystal display device, a plasma display device, an EL display device, or a lamp. Alternatively, the output device 923 may be an audio output device such as a speaker or headphones, a printer, a mobile communication terminal, or a facsimile machine. The output I/F 911 instructs the output device 923 to output, for example, the processing results obtained from various processes executed by the information processing device 900. Specifically, the output I/F 911 instructs the display device to display the processing results of the information processing device 900 as text or images. The output I/F 911 also instructs the audio output device to convert audio signals, such as audio data that has been instructed to be played, into analog signals and output them.

The storage device 913 is one of the storage units of the information processing device 900 and is a device for storing data. The storage device 913 is composed of, for example, a magnetic storage device such as an HDD (Hard Disk Drive), a semiconductor storage device, an optical storage device, or a magneto-optical storage device. The storage device 913 stores programs executed by the CPU 901, various data generated by the execution of the programs, and various data obtained from external sources.

The drive 915 is a reader/writer for recording media, and is either built into or external to the information processing device 900. The drive 915 reads information recorded on the attached removable recording medium 925 and outputs it to the RAM 905. The drive 915 can also write information to the attached removable recording medium 925. The removable recording medium 925 may be, for example, a magnetic disk, optical disk, magneto-optical disk, or semiconductor memory. Specifically, the removable recording medium 925 may be CD media, DVD media, Blu-ray (registered trademark) media, CompactFlash (registered trademark), flash memory, SD memory card (Secure Digital memory card), etc. The removable recording medium 925 may also be, for example, an IC card (Integrated Circuit card) equipped with a contactless IC chip, or an electronic device.

The connection port 917 is a port for directly connecting a device to the information processing device 900. The connection port 917 is, for example, a USB (Universal Serial Bus) port, an IEEE 1394 port, a SCSI (Small Computer System Interface) port, an RS-232C port, etc. The information processing device 900 can directly obtain various data from an external device 927 connected to the connection port 917 and provide various data to the external device 927.

The communication device 919 is, for example, a communication interface configured with a communication device for connecting to a communication network 929. The communication device 919 is, for example, a wired or wireless LAN (Local Area Network), a communication card for Bluetooth (registered trademark), or WUSB (Wireless USB), etc. The communication device 919 may also be a router for optical communications, a router for ADSL (Asymmetric Digital Subscriber Line), or a modem for various communications. The communication device 919 can, for example, send and receive signals between the Internet and other communication devices in accordance with a predetermined protocol such as TCP/IP. The communication network 929 connected to the communication device 919 is configured with a network connected via a wired or wireless connection. For example, the communication network 929 is the Internet, a home LAN, infrared communication, radio wave communication, satellite communication, etc.

The above shows an example of the hardware configuration of the information processing device 900. Each of the above components may be configured using general-purpose components, or may be configured using hardware specialized for the function of each component. The hardware configuration of the information processing device 900 can be changed as appropriate depending on the technical level at the time of implementing this embodiment.

To verify the effectiveness of the method according to this embodiment, a fracture surface characteristic region determination model was generated using the fracture surface characteristic region determination system shown in Figure 2, and brittle surfaces in fracture surface images were determined and the brittle surface ratio was calculated. In this verification, 1,000 sample data were prepared as learning data, with 800 sample data used as training data and 200 sample data used as evaluation data. The fracture surface images of these sample data were captured using an imaging device with the lighting configuration shown in Figure 8. The illumination light color was white. The spatial resolution of the images was 6.7 μm/pixel. U-Net (Non-Patent Document 4), a well-known CNN, was used to generate the fracture surface characteristic region determination model. The receptive field size was set to 1.7 mm x 1.7 mm.

First, an initial fracture surface characteristic region determination model was generated using 100 sample data from the training data. Then, using the initially generated fracture surface characteristic region determination model, post-judgment fracture surface images were obtained for another 100 sample data from the training data, which were then corrected by the user to obtain corrected fracture surface images. Next, the initial fracture surface characteristic region determination model was re-trained using the 100 sample data from which the corrected fracture surface images were obtained as training data to generate a second fracture surface characteristic region determination model. Then, using the second fracture surface characteristic region determination model, post-judgment fracture surface images were obtained for yet another 100 sample data from the training data, which were then corrected by the user to obtain corrected fracture surface images. Then, using the 100 sample data from which the corrected fracture surface images were obtained as training data, the second fracture surface characteristic region determination model was re-trained to generate a third fracture surface characteristic region determination model. Similar processing was then repeated, and the fracture surface characteristic region determination model obtained after training using all 800 sample data provided as training data was used for performance evaluation.

As a performance evaluation based on evaluation data for the fracture surface characteristic region determination model, Figure 10 shows the results of a comparison between the brittle surface rate (fracture rate (user determination)) based on the results of a human determination of the brittle surface of a fracture surface image and the brittle surface rate (fracture rate (model determination)) based on the results of a determination of the brittle surface of a fracture surface image using the fracture surface characteristic region determination model. Figure 10 shows that the fracture surface rate (user determination) and the fracture surface rate (model determination) are nearly identical, demonstrating that the fracture surface characteristic region determination model has sufficient accuracy. Furthermore, the time required to calculate the brittle surface rate based on the results of a determination of the brittle surface of a fracture surface image using the characteristic region determination model was reduced to approximately one-tenth of the time required to calculate the brittle surface rate based on the results of a human determination of the brittle surface of a fracture surface image.

Figure 11 also shows an example of the results of a human judgment of brittle surfaces in an original image, and the results of a brittle surface judgment using the fracture surface characteristic region judgment model. In the fracture surface image after judgment, the white areas are areas judged to be brittle surfaces. Figure 11 shows that the judgment results using the fracture surface characteristic region judgment model are nearly identical to the judgment results by a human.

The above describes in detail preferred embodiments of the present invention with reference to the accompanying drawings, but the present invention is not limited to such examples. It is clear that a person with ordinary skill in the technical field to which the present invention pertains can conceive of various modifications or alterations within the scope of the technical ideas set forth in the claims, and it is understood that these also naturally fall within the technical scope of the present invention.

In the above explanation, an example was shown in which a ductile surface or a fracture surface was determined as the characteristic region of a fracture surface image, but the present invention is not limited to such an example. For example, it is also possible to generate a fracture surface characteristic region determination model that can distinguish between the two regions of the ductile surface and the fracture surface as characteristic regions of a fracture surface image and determine them simultaneously.

REFERENCE SIGNS LIST 1 fracture surface characteristic region determination system 10 test piece 15 fracture surface 100 fracture surface characteristic region determination model generation device 110 model generation unit 120 output unit 130 image multiplication unit 200 fracture surface characteristic region determination device 210 determination unit 220 correction processing unit 230 ratio calculation unit 240 display unit 250 input unit 260 data output unit 300 image storage device 400 imaging device 410 lighting device 420 half mirror 900 information processing device

Claims (10)

a model generation unit that generates, by machine learning, a fracture surface characteristic region determination model for determining a characteristic region in a fracture surface image using , as learning data, a plurality of data pieces in which fracture surface images of the material, each having a resolution sufficient to allow visual determination of the characteristics of the fracture surface of the material, and information representing the position of a characteristic region in the fracture surface image are associated with each other;
an output unit that outputs the fracture surface characteristic region determination model;
Equipped with
the characteristic region is a brittle surface or a ductile surface,
The model generation unit generates the fracture surface characteristic region determination model by deep learning, setting a unit region to be considered in determining whether a pixel in the fracture surface image is the characteristic region to be an region larger than the minimum necessary region capable of identifying fractured tissue . 2. The fracture surface characteristic region determination model generation device according to claim 1, wherein the original image obtained by imaging the fracture surface of the material is an image captured by the imaging device while illuminating the fracture surface of the material so as to be symmetrical with respect to the optical axis of the imaging device. an image multiplication unit that generates a plurality of fracture surface images from one fracture surface image;
the image multiplication unit multiplies the fracture surface image by flipping the original image obtained by capturing the fracture surface of the material horizontally, vertically, or rotating the original image,
3. The fracture surface characteristic region determination model generating device according to claim 1, wherein the model generating unit uses the original image and the fracture surface image multiplied by the image multiplying unit as the fracture surface images of the training data. 4. The fracture surface characteristic region determination model generation device according to claim 1 , wherein the fracture surface characteristic region determination model determines a characteristic region of a fracture surface image input to the fracture surface characteristic region determination model, and outputs a determined fracture surface image indicating the characteristic region on the fracture surface image. The model generation unit
using, as the learning data, a corrected fracture surface image obtained by correcting the characteristic region by a user with respect to the determined fracture surface image obtained using the fracture surface characteristic region determination model output from the output unit,
The fracture surface characteristic region determination model generating device according to claim 4 , wherein the fracture surface characteristic region determination model is retrained. a model generation step of generating a fracture surface characteristic region determination model for determining a characteristic region in a fracture surface image by machine learning using, as learning data, a plurality of data in which fracture surface images of the material , each having a resolution sufficient to allow visual determination of the characteristics of the fracture surface of the material, and information representing the position of a characteristic region in the fracture surface images, are associated with each other;
an output step of outputting the fracture surface characteristic region determination model;
Including ,
the characteristic region is a brittle surface or a ductile surface,
The model generation step is a method for generating a fracture surface characteristic region determination model, in which the unit region to be considered in determining whether a pixel in the fracture surface image is the characteristic region is set to an area larger than the minimum necessary area capable of identifying fractured tissue, and the fracture surface characteristic region determination model is generated by deep learning . Computer,
a model generation unit that generates, by machine learning, a fracture surface characteristic region determination model for determining a characteristic region in a fracture surface image using , as learning data, a plurality of data pieces in which fracture surface images of the material, each having a resolution sufficient to allow visual determination of the characteristics of the fracture surface of the material, and information representing the position of a characteristic region in the fracture surface image are associated with each other;
an output unit that outputs the fracture surface characteristic region determination model;
Equipped with
the characteristic region is a brittle surface or a ductile surface,
The model generation unit is a program that functions as a fracture surface characteristic region determination model generation device, which generates the fracture surface characteristic region determination model by deep learning, by setting the unit region to be considered in determining whether a pixel in the fracture surface image is the characteristic region to an region that is larger than the minimum necessary region that can identify fractured tissue . A fracture surface characteristic region determination device for determining a characteristic region in a fracture surface image using a fracture surface characteristic region determination model for determining a characteristic region in a fracture surface image , the fracture surface characteristic region determination model being generated by machine learning using a plurality of data sets , each data set being a fracture surface image of a material with a resolution sufficient to enable visual determination of the characteristics of the fracture surface of the material, and information representing the position of a characteristic region in the fracture surface image , as learning data,
the characteristic region is a brittle surface or a ductile surface,
The fracture surface characteristic region determination model is generated by deep learning, with a unit region to be considered in determining whether a pixel in the fracture surface image is the characteristic region being a region wider than the minimum necessary region capable of identifying fractured tissue.
Fracture surface characteristic area determination device. A fracture surface characteristic region determination method for determining a characteristic region in a fracture surface image using a fracture surface characteristic region determination model for determining a characteristic region in a fracture surface image , the fracture surface characteristic region determination model being generated by machine learning using, as training data , a plurality of data in which fracture surface images of a material having a resolution sufficient to enable visual determination of the characteristics of the fracture surface of the material and information representing the positions of characteristic regions in the fracture surface images are associated, the method comprising:
the characteristic region is a brittle surface or a ductile surface,
A fracture surface characteristic region determination method in which the fracture surface characteristic region determination model is generated by deep learning, with the unit region to be considered in determining whether a pixel in the fracture surface image is the characteristic region being an area larger than the minimum necessary region capable of identifying fractured tissue . The fracture surface characteristic region determination method according to claim 9 , further comprising the step of calculating a proportion of the characteristic region in the fracture surface of the material based on an area of the characteristic region and an area of the fracture surface of the material.