JP7528008B2 - IMAGE EVALUATION SYSTEM AND IMAGE EVALUATION METHOD - Google Patents

Description

An embodiment of the present invention relates to image evaluation technology.

Conventionally, ultrasonic inspection technology has been widely used to inspect structures. Inspectors detect the presence or absence of defects inside the structure from cross-sectional images obtained by ultrasonic inspection, and measure their location and depth. One method for measuring the depth of a defect is the tip echo method. This tip echo method measures the depth of a defect from the beam path and refraction angle of the echo from the upper or lower end of the defect. Inspectors make their judgments based on their own specialized knowledge and experience, but evaluation results may vary depending on the skill of the inspector. Therefore, technology is being developed for automatic evaluation in ultrasonic inspection using signal processing or artificial intelligence (AI).

As a first example, a technique is known that uses ultrasound images as input for machine learning to detect defects. This technique determines whether an ultrasound image contains a defect, but it cannot identify the location and size of the defect in the image.

As a second example, a technique is known in which a flaw detection image is generated from an ultrasonic response waveform, and this flaw detection image is compared with a normal pattern image using a pattern recognition algorithm. With this technique, if a flaw detection image differs from a normal pattern image, the position of the defect can be detected by determining the shape echo and defect echo in the flaw detection image. With this technique, it is necessary to extract an image containing individual echoes based on the flaw detection image. However, if the image has a high noise intensity and the defect echo is unclear, or if the shape echo and the defect echo overlap, the extraction work may be difficult and there is a risk of overlooking the defect. In addition, the extracted echoes are classified into shape echo and defect echo, and training data is created. However, the determination of whether an echo is a defect echo or a shape echo is always made based on whether it is displayed or not, and there is a risk that echoes originating from the structure of the weld or inclusions may also be determined to be defect echoes.

In recent years, attempts have been made to detect objects in images using AI for object recognition, mainly in the fields of autonomous driving technology and medicine. One well-known object recognition technique is segmentation. Segmentation is a machine learning algorithm that performs class classification on images at the pixel level, and can detect objects at the pixel level. Supervised learning is used to build a learning model for this segmentation. For example, learning is performed using teacher labels that are colored for each object on a pixel-by-pixel basis. When applying segmentation to defect detection in ultrasound images, it takes more effort and cost to create teacher labels than when applying it to camera images. When comparing camera images and ultrasound images, camera images often have a clear boundary between the object and the background. On the other hand, ultrasound images contain noise echoes such as scattered waves generated from weld metal in addition to ultrasonic echoes from objects. Therefore, ultrasound images often have unclear boundaries between objects and other things compared to general camera images. In other words, it is very difficult to distinguish objects on a pixel-by-pixel basis in ultrasound images.

Several methods have been proposed to automate the creation of teacher labels in the segmentation of ultrasound images. For example, there is a method in which a threshold is set to separate echoes to be detected from others. However, with this method, separation using a threshold becomes difficult when the noise intensity is high and the defect echo is unclear in the image, or when shape echoes and defect echoes overlap. There is also a method that separates defect echoes from noise echoes based on whether they are always displayed. However, with this method, there is a risk that echoes originating from the structure or inclusions of the weld will also be classified as defect echoes.

Special Publication No. 2020-503509 International Publication No. 2015/001624

The problem that the present invention aims to solve is to provide an image evaluation technique that can improve the accuracy of detecting the position of a detection target contained in an object during image evaluation.

The image evaluation system according to an embodiment of the present invention is at least one of an image obtained by a flaw detection device that detects flaws in a detection target contained in an object and an image generated by simulating the image, and includes a learning image acquisition unit that acquires a plurality of learning images used for machine learning, a teacher label generation unit that generates teacher labels that set specific common areas indicating specific areas of the detection target present in each of the learning images as the position of the detection target, and a judgment model generation unit that generates a judgment model that can calculate a judgment value that can determine whether or not the detection target exists for each unit indicating a position in an image, using the learning images and the teacher labels corresponding to each of the learning images. and an image obtained by the flaw detection device, the image being equipped with a judgment image acquisition unit that acquires at least one judgment image to be used for judgment, and a judgment unit that uses the judgment model to calculate the judgment value for each unit that indicates a position in the judgment image , wherein the unit is a pixel, the machine learning relates to segmentation, the specific common area is an edge of the detection object, the teacher label generation unit generates the teacher label that labels only the edge of the detection object, and the teacher label is an image having the same number of pixels as the learning image and is a binary image composed of pixel values that indicate the specific common area and pixel values that indicate the rest .

Embodiments of the present invention provide an image evaluation technique that can improve the accuracy of detecting the position of a detection target included in an object during image evaluation.

FIG. 1 is a block diagram showing an image evaluation system. FIG. 1 is a configuration diagram showing an ultrasonic flaw detection device. FIG. 13 is an image diagram showing a learning ultrasound image. FIG. 11 is an image showing teacher labels. FIG. 4 is an explanatory diagram showing a flow of selecting a channel from a learning ultrasound image. 6 is a graph showing a received waveform of an echo that detects a detection target. 6 is a graph showing the absolute value of the received waveform of an echo that detected a detection target. FIG. 13 is an explanatory diagram showing pixels of teacher labels. FIG. 11 is an image diagram showing an ultrasonic image for determination. FIG. FIG. 11 is an image diagram showing a binarized determined image. FIG. 13 is an explanatory diagram showing an image that appears in a determined image. FIG. 11 is an explanatory diagram showing the coordinates of a specific common area. 1 is a flowchart showing a method for generating a teacher label. 11 is a flowchart showing a method for generating a determination model. 13 is a flowchart showing a method for evaluating a determination ultrasound image.

Below, we will explain in detail the embodiments of the image evaluation system and image evaluation method with reference to the drawings.

In this embodiment, an ultrasonic inspection method generally called a phased array ultrasonic inspection is used as an example. Among them, a linear imaging method in which an ultrasonic element is electronically scanned while forming an ultrasonic beam in a certain direction, a sector imaging method in which an angle at which an ultrasonic beam is formed is changed in a fan shape while a driven ultrasonic element is fixed or electronically scanned , a total focusing method (TFM) in which a beam is focused by comprehensively setting a focus in an arbitrary coordinate region, or an aperture synthesis method can be used as an imaging method using ultrasonic waves. Note that an ultrasonic inspection method in which mechanical scanning is performed using a single probe may also be used. In the following description, a typical linear imaging method is exemplified.

Reference numeral 1 in FIG. 1 denotes an image evaluation system according to this embodiment. This image evaluation system 1 evaluates the presence or absence of a defect (detection target K) in an object T to be inspected. In this embodiment, an image acquired by a predetermined ultrasonic flaw detection method is automatically analyzed using a computer equipped with artificial intelligence (AI) to evaluate the presence or absence of a defect in the object T. This evaluation of the presence or absence of a defect can be utilized for, for example, diagnosing the soundness of the object T.

In this embodiment, the image to be analyzed may be either a still image or a video. In the following explanation, a still image acquired by ultrasound examination is used as an example.

The image evaluation system 1 includes an ultrasonic flaw detector 10, an ultrasonic flaw detection device 20, an ultrasonic image processing device 30, and an ultrasonic image evaluation device 100.

The ultrasonic flaw detection device 20, the ultrasonic image processing device 30, and the ultrasonic image evaluation device 100 are configured as a computer having hardware resources such as a CPU, ROM, RAM, and HDD, and in which software-based information processing is realized using the hardware resources as the CPU executes various programs. Furthermore, the image evaluation method of this embodiment is realized by having the computer execute various programs.

The components of the image evaluation system 1 do not necessarily need to be provided on multiple computers. For example, the components of the image evaluation system 1 may be provided on a single computer.

As shown in FIG. 2, the ultrasonic flaw detector 10 emits ultrasonic waves U to an object T to be inspected and detects the reflected waves reflected by the object K to be detected. In the following description, the reflected waves may be referred to as echoes. This ultrasonic flaw detector 10 includes a probe 11 in which multiple ultrasonic elements 11A that emit ultrasonic waves U are arranged.

An example of the object T is a component made of a metal material. In particular, the detection object K occurring at the weld is evaluated by the image evaluation system 1. Note that the object T may be a material other than a metal material, for example, a composite material such as reinforced concrete, mortar, or fiber-reinforced plastic. The detection object K occurring inside or on the surface of such an object T is evaluated by the image evaluation system 1.

The detection target K can be arbitrarily determined by the user of the image evaluation system 1. Detection target K (defect) can be arbitrarily determined to be a crack, flaw, cavity, round hole, peeling, thinning, inclusion, etc. occurring inside or on the surface of the target T.

The ultrasonic flaw detector 10 of this embodiment is controlled by applying a voltage by the ultrasonic flaw detector 20. As the ultrasonic flaw detector 10, a phased array ultrasonic flaw detection method is exemplified in which a beam of ultrasonic waves U is formed in a fixed direction while electronically scanning the ultrasonic element 11A to be driven. Note that this embodiment can be applied to other methods. For example, it may be applied to a sector imaging method, a total focusing method (TFM), an aperture synthesis method, or the like. Furthermore, it may be applied to an ultrasonic flaw detection method in which a single probe is used to mechanically scan and acquire the flaws.

When inspecting the object T, an acoustic propagation medium 2 called a wedge is placed between the ultrasonic flaw detector 10 and the object T. This acoustic propagation medium 2 is used to make the ultrasonic waves U enter the object T at a highly directional angle.

As the acoustic propagation medium 2, an isotropic material is used that allows the ultrasonic waves U to propagate and has a known acoustic impedance. If the surface of the target object T is flat, it is not necessary to use the acoustic propagation medium 2.

Isotropic materials used as the acoustic propagation medium 2 include, for example, acrylic, polyimide, gel, and other polymers. The acoustic propagation medium 2 may be made of a material that has an acoustic impedance similar to or the same as that of the front panel (not shown) of the ultrasonic element 11A. It may also be made of a material that has an acoustic impedance similar to or the same as that of the target object T. A composite material that changes the acoustic impedance stepwise or gradually may also be used.

In addition, damping material may be placed inside and outside the acoustic propagation medium 2 so that multiple reflected waves inside the acoustic propagation medium 2 do not affect the flaw detection results. A mountain-shaped wave-eliminating shape may also be provided. A multiple reflection reduction mechanism may also be provided.

In the following explanation, the acoustic propagation medium 2 may be omitted when explaining how ultrasonic waves U are incident on the target object T from the ultrasonic flaw detector 10.

An acoustic contact medium (not shown) for propagating ultrasonic waves U is used at the contact portion of the path from the ultrasonic flaw detector 10 to the object T. For example, when the acoustic propagation medium 2 is used, an acoustic contact medium (not shown) is used at the contact portion between the ultrasonic flaw detector 10 and the acoustic propagation medium 2, and at the contact portion between the acoustic propagation medium 2 and the object T. When the acoustic propagation medium 2 is not used, an acoustic contact medium (not shown) is used at the contact portion between the ultrasonic flaw detector 10 and the object T. For this acoustic contact medium, a medium capable of propagating ultrasonic waves U, such as water, glycerin, machine oil, castor oil, acrylic, polystyrene, or gel, is used.

As shown in FIG. 1, a detection signal containing information on the reflected wave detected by the ultrasonic flaw detector 10 is input to an ultrasonic flaw detector 20. The detection signal obtained by the ultrasonic flaw detector 20 is then processed by an ultrasonic image processing device 30.

The ultrasonic image processing device 30 acquires ultrasonic images for detecting flaws in a detection target K contained in an object T using ultrasonic waves U. This ultrasonic image processing device 30 can generate ultrasonic images along the scanning direction of the ultrasonic element 11A based on the reflected waves of the ultrasonic waves U.

Figure 3 illustrates an example of an ultrasound image as a representative linear scan image. This ultrasound image is used as a learning ultrasound image 31 as a learning image in this embodiment. In this learning ultrasound image 31, the background B is displayed in black, the incidence range 32 where the ultrasound U is incident on the object T is displayed in gray, the surface 33 and bottom surface 34 of the object T are displayed as white lines, and the detection object K is displayed in white. Note that this embodiment can also be implemented with an ultrasound image that is a color image represented by RGB or the like, but for clarity of explanation, the ultrasound image will be described as a grayscale image. In the example of Figure 3, one detection object K is shown in the learning ultrasound image 31. Note that the ultrasound image is also used as a judgment ultrasound image 35 (Figure 9) as a judgment image in this embodiment.

As shown in FIG. 2, the ultrasonic wave U incident on the object T from the ultrasonic flaw detector 10 has its incidence and refraction angles determined according to the delay time, and propagates inside the object T. The propagated ultrasonic wave U is reflected or scattered by the detection object K, etc., and reaches the ultrasonic flaw detector 10 again. In addition to the detection object K, the ultrasonic wave U is also reflected or scattered by the bottom surface 34 or corners (not shown) of the object T, and reaches the ultrasonic flaw detector 10.

The reflected waves (scattered waves) that reach the ultrasonic flaw detector 10 are visualized based on the arrival time and the information of the ultrasonic element 11A. At this time, if a detection target K is present in the object T, an echo (echo) of the detection target K appears in the training ultrasonic image 31.

In this embodiment, the ultrasound image processing device 30 generates a training ultrasound image 31 (FIG. 3) or a judgment ultrasound image 35 (FIG. 9). The training ultrasound image 31 or the judgment ultrasound image 35 is input from the ultrasound image processing device 30 to the ultrasound image evaluation device 100.

The training ultrasonic image 31 of this embodiment may be a CG image generated using computer graphics (CG) to imitate an ultrasonic image obtained by the ultrasonic flaw detection device 20. Also, a wide variety of training ultrasonic images 31 may be generated by manually or automatically editing the ultrasonic image obtained by the ultrasonic flaw detection device 20.

As shown in FIG. 1, the ultrasound image evaluation device 100 performs machine learning using a training ultrasound image 31, and identifies the position of a detection target K contained in a judgment ultrasound image 35 used for judgment based on the results of this machine learning.

In other words, the analysis using the computer in this embodiment can use analysis techniques based on artificial intelligence learning. For example, learning models generated by machine learning using neural networks, learning models generated by other machine learning, deep learning algorithms, regression analysis, and other mathematical algorithms can be used.

The system of this embodiment includes a computer equipped with artificial intelligence that performs machine learning. For example, the system may be configured with one computer equipped with a neural network, or may be configured with multiple computers equipped with neural networks.

Here, a neural network is a mathematical model that represents the characteristics of brain functions through computer simulation. For example, this model shows that artificial neurons (nodes) that form a network through synaptic connections change the strength of synaptic connections through learning, and acquire problem-solving abilities. Furthermore, neural networks acquire problem-solving abilities through deep learning.

In addition, deep reinforcement learning may be used in the neural network, in which a reward function is set for each information item to be learned, and the information item with the highest value is extracted based on the reward function.

In this embodiment, a supervised machine learning method is used. For example, semantic segmentation is used. However, the present invention is not limited to this, and multiple types of machine learning methods may be combined.

Other machine learning methods that can be used besides semantic segmentation include the Fully Convolutional Neural Network, R-CNN (Region Convolutional Neural Network) which has a proven track record in image recognition, k-nearest neighbors, support vector machines (SVM), and deep learning.

Note that there are various methods for deep learning, such as autoencoder, recurrent neural network (RNN), long short-term memory (LSTM), and generative adversarial network (GAN). These methods may be applied to this embodiment.

As shown in FIG. 9, the ultrasonic image for determination 35 has a background B displayed in black, an incident range 32 where ultrasonic waves U are incident on the object T displayed in gray, a surface 33 and a bottom surface 34 of the object T displayed as white lines, and a detection object K displayed in white. In the example of FIG. 9, two detection objects K are shown in the ultrasonic image for determination 35. Then, a determined image 36 is generated from this ultrasonic image for determination 35 using a determination model.

As shown in Fig. 10, in the determined image 36, the background B is displayed in black, and the pixels in an area 37 that is more likely to contain the edge of the detection target K are displayed in brighter colors. In other words, the area is displayed in a range of colors from gray to white. In the example of Fig. 10, two areas 37 are shown in the determined image 36. A binarized image 38 is then generated from this determined image 36.

11, in the binarized image 38, the background B is displayed in black, and a specific common area 39 where an edge of the detection target K exists is displayed in white. In the example of Fig. 11, two specific common areas 39 are captured in the binarized image 38. Then, the position and depth of the detection target K can be obtained from this binarized image 38.

As shown in FIG. 1, the ultrasound image evaluation device 100 includes an input unit 110, a calculation unit 120, a judgment model generation unit 130, a memory unit 140, and an output unit 150.

An ultrasound image obtained by irradiating ultrasound U onto an object T is input from the outside to the input unit 110. In this embodiment, an example is shown in which an ultrasound image is input from an ultrasound image processing device 30, but an ultrasound image may also be input from another device.

There are two types of ultrasound images input to the input unit 110: a training ultrasound image 31 (FIG. 3) used for machine learning and a judgment ultrasound image 35 (FIG. 9). This input unit 110 serves as the training image acquisition unit and judgment image acquisition unit of this embodiment.

In this embodiment, when machine learning is performed, multiple training ultrasound images 31 are input to the input unit 110. Then, a group of teacher labels is generated based on this group of training ultrasound images. Furthermore, a judgment model is generated using the group of training ultrasound images and the group of teacher labels. On the other hand, when using this judgment model to judge the detection object K contained in the judgment ultrasound image 35, the judgment ultrasound image 35 is input to the input unit 110. Note that the judgment ultrasound image 35 to be input may be one or multiple.

The calculation unit 120 includes a teacher label generation unit 121, a determination unit 122, an extraction unit 123, and a detection target position calculation unit 124. These are realized by the CPU executing a program stored in the memory or HDD.

The storage unit 140 stores images and judgment models obtained by ultrasonic inspection. This storage unit 140 includes a learning ultrasound image storage unit 141, a teacher label storage unit 142, a judgment model storage unit 143, a judgment ultrasound image storage unit 144, and a detection target position storage unit 145. These are collections of information stored in memory, a HDD, or the cloud, and organized so that they can be searched or accumulated.

The output unit 150 outputs predetermined information. For example, it displays an image showing the position of the detection target K in the ultrasonic image 35 for determination. The output unit 150 may be any device capable of displaying digital data, such as a display for a PC, a television, a projector, or a head-mounted display. It may also be a device that converts the signal into an analog signal once and then displays the image on the screen, such as a cathode ray tube. In other words, the ultrasonic image evaluation device 100 includes a device that displays an image that outputs the analysis result. Furthermore, the output unit 150 may be a printer that prints predetermined information on a paper medium. The output unit 150 may also be a device that writes predetermined information into a removable and portable storage medium, such as a USB memory.

The output unit 150 may display the composite signal of the ultrasonic echoes, the visualization results, the coordinates of the ultrasonic flaw detector 10 and its relative position to the detection target K, the delay time, the focal depth, the flaw detection refraction angle, and other flaw detection conditions. The output unit 150 may also have a user interface function that generates an alarm by sound or light emission according to the set conditions, or inputs operations as a touch panel.

The ultrasound image evaluation device 100 may have other configurations. For example, it may have a communication unit (not shown). This communication unit communicates with other computers via a communication line such as the Internet.

The components of the ultrasound image evaluation device 100 do not necessarily have to be provided on a single computer. For example, a single ultrasound image evaluation device 100 may be realized using multiple computers connected to each other via a network.

The teacher label generation unit 121 generates a teacher label 40 in which a specific common area 39 (FIG. 11) indicating a specific area of the detection target K present in each training ultrasound image 31 is set as the position of the detection target K. One teacher label 40 is generated corresponding to one training ultrasound image 31. Then, a group of teacher labels corresponding to the group of training ultrasound images is generated.

As shown in FIG. 4, the teacher label 40 is an image (two-dimensional data) of the same size as the corresponding training ultrasound image 31. In other words, it is an image having the same number of pixels as the training ultrasound image 31. In this teacher label 40, the background B is displayed in black, and the specific common area 39 of the detection target K is displayed in white. In other words, the teacher label 40 is a binary image in which the pixel value of the specific common area 39 is "1" and the other pixel values are "0". The position (coordinates) of the specific common area 39 in this teacher label 40 corresponds to the position of the upper end of the detection target K shown in the training ultrasound image 31.

A specific common area 39 is set in the teacher label 40, which corresponds to the part where the echo of the detection target K is present. The specific common area 39 can be set arbitrarily according to the type of detection target K, for example, the crack end in the case of a crack, or the center in the case of a blowhole. This setting may be performed by the user or automatically by the ultrasound image evaluation device 100.

When the detection target K is a crack, the end of the echo of the detection target K is defined as the specific common area 39. Then, the specific common area 39 of the training ultrasound image 31 corresponds to the position of the end of the crack. Then, the ultrasound image evaluation device 100 detects the specific common area 39 from the training ultrasound image 31. Furthermore, the depth of the crack can be found by calculating the distance between the specific common area 39 and the echo of the bottom surface 34 of the object T.

Next, the method for generating the teacher label 40 will be described using the flowchart in FIG. 14. Note that the detection target K is a crack, and its edge is a specific common region 39.

First, in step S11, a plurality of training ultrasound images 31 (training images) are acquired by the ultrasound image processing device 30. These training ultrasound images 31 are input to the input unit 110 (training image acquisition unit) of the ultrasound image evaluation device 100. The multiple training ultrasound images 31 input to the input unit 110 are stored in the training ultrasound image storage unit 141.

In the next step S12, a user, such as an inspector, researcher, or expert, checks the training ultrasound images 31. The user carefully examines each training ultrasound image 31 to see whether or not a crack is visible.

In the next step S13, the teacher label generation unit 121 selects a channel of the ultrasonic flaw detector 10 when a crack is captured in the training ultrasonic image 31. For example, among the multiple (e.g., N) ultrasonic elements 11A equipped in the ultrasonic flaw detector 10, a received waveform 41 (Figure 6) that includes an echo from the end of the crack is identified. Then, the channel of the ultrasonic flaw detector 10 that received this received waveform 41 is selected. Here, the number of that channel is designated as m (Figure 5).

In the next step S14, the teacher label generating unit 121 acquires an echo of the end of the crack from the received waveform 41 of channel number m, and calculates the beam path distance Xd at the peak position of this echo. Note that the channel number m may be selected by the teacher label generating unit 121 or by the user.

For example, as shown in FIG. 6, the received waveform 41 of channel number m includes a waveform 42 of an echo from the surface 33 of the object T, a waveform 43 of an echo from the bottom surface 34 of the object T, and a waveform 44 of an echo from the end of a crack. The amplitude value of the received waveform 41 of channel number m is Am(X). Furthermore, the waveform 45 obtained by converting the received waveform 41 into an absolute value is shown in FIG. 7.

Here, the teacher label generation unit 121 calculates the peak value |Am(Xd)| of the echo at the end of the crack from |Am(X)|, which is the absolute value of Am(X). The teacher label generation unit 121 can then calculate the beam path distance Xd to this position (Figure 5).

Returning to FIG. 14, in the next step S15, the teacher label generation unit 121 determines the coordinates Ex, Ey of the center of the specific common area 39 of the teacher label 40 from the channel number m, the beam path distance Xd, and the refraction angles α, β (FIG. 5).

In the next step S16, the teacher label generation unit 121 sets a square with a side length of d and centered at the coordinates Ex, Ey (Figure 8). Here, a binary image is generated in which the square is set to "1" and the rest is set to "0". This binary image becomes the teacher label 40 (Figure 4).

In the next step S17, the truth label generation unit 121 stores the generated truth label 40 in the truth label storage unit 142. Then, the method for generating the truth label 40 is completed.

As shown in FIG. 8, the size (maximum length) of the specific common area 39 in this embodiment is represented by d. For example, it is a square of d×d. Here, the diagonal of the specific common area 39 is set to be equal to or greater than one-quarter of the wavelength λ of the echo of the crack end.

As shown in FIG. 6, a length of one-quarter of the wavelength λ corresponds to the length from the rising edge of the waveform to the peak, and is the minimum length that includes the echo features from the end of the crack. If the size (maximum length) of the specific common area 39 is less than one-quarter of the wavelength λ, the echo features will not be fully included in the specific common area 39, and there is a possibility that the end of the crack will not be detected correctly.

Even if the echo is made up of several successive waves, it is necessary to extract a representative waveform from the waves and determine the wavelength.

If the specific common area 39 is a circle, the diameter of the circle is set to a size equal to or greater than a quarter of the wavelength λ. If the specific common area 39 is an ellipse, the major axis of the ellipse is set to a size equal to or greater than a quarter of the wavelength λ. The specific common area 39 can also be set to a polygon or an indefinite shape other than a circle. In that case, the maximum length of each is set to a size equal to or greater than a quarter of the wavelength λ.

In this embodiment, the size d of the specific common area 39 is equal to or greater than one-fourth the wavelength of the frequency of the ultrasonic wave U used by the ultrasonic flaw detection device 20, so that the specific common area 39 sufficiently contains the characteristics of the detection target K, and the detection target K can be correctly detected. Furthermore, the detection target K can be distinguished from noise.

As shown in FIG. 1, the judgment model generation unit 130 uses the training ultrasound images 31 and the teacher labels 40 corresponding to each training ultrasound image 31 to generate a judgment model that can calculate a judgment value that can determine whether or not a detection target K exists for each unit indicating a position in the image.

In this embodiment, the unit indicating a position in an image is, for example, a pixel. The position (coordinates) at which a specific pixel exists in an image is managed by a conventionally known method.

In this embodiment, the unit indicating a position in an image is a pixel, but other configurations are also possible. For example, a unit may consist of multiple adjacent pixels. Also, the image may be divided into multiple grids, and each square of the grid may be considered as a unit.

Next, we will explain how to generate a judgment model using the flowchart in Figure 15.

First, in step S21, the judgment model generation unit 130 acquires multiple training ultrasound images 31 that are input to the input unit 110 and stored in the training ultrasound image storage unit 141.

In the next step S22, the judgment model generation unit 130 performs supervised learning based on multiple training ultrasound images 31 and the teacher labels 40 corresponding to each training ultrasound image 31.

In the next step S23, the judgment model generation unit 130 generates a judgment model by supervised learning. By performing supervised learning, a judgment model is generated that has the ability to determine whether or not the specific common region 39 of the detection target K is included in the image and output the judgment value for each pixel. This judgment model makes it possible to output a judged image 36 (Figure 10) as an output value when a judgment ultrasound image 35 (Figure 9) is accepted as an input value.

In addition, when generating a judgment model (supervised learning), it is desirable to prepare as many training ultrasound images 31 as possible and use them to generate a judgment model. It is desirable that at least two of the training ultrasound images 31 are images in which the detection target K is present. In addition, the training ultrasound images 31 may be images in which multiple detection targets K are captured per image. Furthermore, the training ultrasound images 31 may include images in which the detection target K is not present and only echoes of the surface 33 and the bottom surface 34 are present.

In the next step S24, the judgment model generation unit 130 stores the generated judgment model in the judgment model storage unit 143. Then, the judgment model generation method ends.

As shown in FIG. 1, the determination unit 122 uses a determination model to calculate a determination value indicating whether or not a detection target K exists for each pixel of the determination ultrasound image 35.

Next, the method for evaluating the evaluation ultrasound image 35 will be explained using the flowchart in FIG. 16.

First, in step S31, a judgment ultrasound image 35 (judgment image) is acquired by the ultrasound image processing device 30. This judgment ultrasound image 35 is input to the input unit 110 (judgment image acquisition unit) of the ultrasound image evaluation device 100. The judgment ultrasound image 35 input to the input unit 110 is stored in the judgment ultrasound image storage unit 144.

In the next step S32, the determination unit 122 inputs the determination ultrasound image 35 stored in the determination ultrasound image storage unit 144 into the trained determination model to generate a determined image 36.

As shown in FIG. 10, the determined image 36 is an image (two-dimensional data) of the same size as the determination ultrasound image 35. Each pixel of the determined image 36 is represented by a continuous value from "0" to "1" as a determination value.

As shown in FIG. 12, in the determined image 36, the pixels are displayed brighter in areas 37 where the edge of the detection target K is more likely to exist. For example, when the determination value is "0", the pixel value is black. When the determination value is "1", the pixel value is white. When the determination value is greater than "0" and less than "1", the pixel value is gray. In other words, in the determined image 36, the closer the determination value is to "1", the higher the possibility that a specific common area 39 exists.

Returning to FIG. 16, in the next step S33, the extraction unit 123 extracts the position of the detection target K from the determined image 36 (FIG. 10). This extraction unit 123 performs a binarization process on the determined image 36 to generate a binarized image 38 (FIG. 11) from which the specific common region 39 has been extracted.

As shown in Figures 11 and 13, the determined image 36 is converted to two gradations to generate a binary image 38. For example, a threshold value is set in advance by the user. Then, if the value of each pixel in the determined image 36 exceeds the threshold value, it is replaced with a white value of "1", and if it is below the threshold value, it is replaced with a black value of "0". In this way, an image of a specific common area 39 can be made to appear in the binary image 38.

Returning to FIG. 16, in the next step S34, the detection target position calculation unit 124 calculates the position of the detection target K (position of the crack) based on the specific common region 39 extracted by the extraction unit 123. Furthermore, the depth dimension of the detection target K is calculated based on this position of the detection target K.

For example, the detection target position calculation unit 124 calculates the coordinates (Gx1, Gy1 and Gx2, Gy2) of the center position of the specific common area 39 of the binarized image 38 (FIG. 13). This coordinate does not necessarily have to be the center of the specific common area 39 in the binarized image 38. For example, it may be the coordinate with the maximum intensity in the specific common area 39 of the binarized image 38, or the coordinate may be the center of gravity of a pixel group exceeding a predetermined threshold. In other words, it is sufficient if it can be uniquely determined by a specific logic.

The detection target position calculation unit 124 calculates the coordinates of the center position of the specific common region 39 in the binarized image 38 and the relative distance (depth of the crack) to the echo of the surface 33 or bottom surface 34 of the object T. For example, it calculates the distance (D1 and D2) from the bottom surface 34 of the object T to the position of the edge of the detection target K (Figure 9). When calculating the distance, a conversion is made from the number of pixels to the dimension of length.

The position of the surface 33 or bottom surface 34 of the object T is assumed to be determined in advance by a predetermined method. For example, the coordinates of the surface 33 or bottom surface 34 of the object T may be calculated using the maximum value of each echo as the coordinate, or the center of an echo exceeding a predetermined threshold value.

In the next step S35, the detection target position calculation unit 124 stores the calculated position and depth of the detection target K (crack) in the detection target position memory unit 145 and outputs it using the output unit 150. Then, the method for evaluating the determination ultrasound image 35 is completed.

In this embodiment, instead of labeling the entire echo of the detection target K, only the ends of the detection target K are labeled. This makes it possible to significantly reduce the effort required to create the teacher labels 40. In addition, because only the ends of the detection target K are detected, rather than the entire target K, even if the shape of the detection target K differs in each ultrasound image, the ends (specific common areas 39) that are commonly required for measuring its depth can be detected stably.

In addition, since the specific common area 39 is the end of the detection target K, the accuracy of determining the depth at which the detection target K exists can be improved.

In addition, the extraction unit 123 extracts the position of the detection target K from the determined image 36 (Figure 10) generated based on the determination value calculated from the determination ultrasound image 35 (Figure 9), thereby extracting the position of the detection target K from the determined image 36 and detecting the detection target K present in the determination ultrasound image 35.

In addition, the extraction unit 123 extracts the position of the detection target K by binarizing the judgment value for each unit indicating the position in the image based on a threshold value, thereby making it possible to extract the part of the judged image 36 where the detection target K is likely to exist.

In addition, by using pixels as the unit in this embodiment, processing can be performed for each pixel, which is the smallest unit of an image, and the evaluation ultrasound image 35 can be evaluated in detail.

In this embodiment, the judgment of an arbitrary value (pixel value) using a reference value (threshold value) may be a judgment of "whether the arbitrary value is equal to or greater than the reference value" or a judgment of "whether the arbitrary value exceeds the reference value". Alternatively, it may be a judgment of "whether the arbitrary value is equal to or less than the reference value" or a judgment of "whether the arbitrary value is less than the reference value". Furthermore, the reference value may not be fixed, but may be variable. Therefore, a value within a predetermined range may be used instead of the reference value, and a judgment of whether the arbitrary value falls within the predetermined range may be made. Furthermore, the error that occurs in the device may be analyzed in advance, and a predetermined range including the error range centered on the reference value may be used for the judgment.

In the flowchart of this embodiment, an example is shown in which each step is executed in series, but the order of steps is not necessarily fixed, and the order of some steps may be reversed. Also, some steps may be executed in parallel with other steps.

The system of this embodiment includes a control device with a highly integrated processor such as a dedicated chip, FPGA (Field Programmable Gate Array), GPU (Graphics Processing Unit), or CPU (Central Processing Unit), a storage device such as ROM (Read Only Memory) or RAM (Random Access Memory), an external storage device such as HDD (Hard Disk Drive) or SSD (Solid State Drive), a display device such as a display, an input device such as a mouse or keyboard, and a communication interface. This system can be realized with a hardware configuration that uses a normal computer.

The program executed by the system of this embodiment is provided in advance in a ROM or the like. Alternatively, this program may be provided stored in an installable or executable file format on a non-transitory computer-readable storage medium such as a CD-ROM, CD-R, memory card, DVD, or flexible disk (FD).

The programs executed by this system may also be stored on a computer connected to a network such as the Internet and provided by downloading them via the network. This system may also be configured by combining separate modules that independently perform the functions of the components and connect them together via a network or dedicated lines.

In this embodiment, the analysis of images acquired by ultrasonic testing is illustrated, but other aspects are also possible. For example, this embodiment may be applied to the analysis of images acquired by X-ray testing, visible light testing, microscopic testing, infrared testing, ultraviolet testing, etc.

In addition, in this embodiment, the ultrasound image has been described as being a grayscale image, but other configurations are also possible. For example, the ultrasound image may be a color image expressed in RGB or the like. In that case, for example, pixel values above a threshold value may be colored red, and pixel values below the threshold value may be colored blue, to give the display form a distinctive feature.

According to the embodiment described above, by providing a judgment unit that uses a judgment model to calculate a judgment value for each unit indicating a position in the judgment image, it is possible to improve the accuracy of detecting the position of a detection target included in an object in image evaluation.

Although several embodiments of the present invention have been described, these embodiments are presented as examples and are not intended to limit the scope of the invention. These embodiments can be implemented in various other forms, and various omissions, substitutions, modifications, and combinations can be made without departing from the spirit of the invention. These embodiments or modifications thereof are within the scope of the invention and its equivalents as set forth in the claims, as well as the scope and spirit of the invention.

1...Image evaluation system, 2...Acoustic propagation medium, 10...Ultrasonic flaw detector, 11...Probe, 11A...Ultrasonic element, 20...Ultrasonic flaw detection device, 30...Ultrasonic image processing device, 31...Learning ultrasonic image, 32...Incidence range, 33...Surface of object, 34...Bottom surface of object, 35...Ultrasonic image for judgment, 36...Judgmented image, 37...Area where edge is likely to exist, 38...Binarized image, 39...Specific common area, 40...Teaching label, 41...Received waveform, 42...Waveform of echo from surface of object, 43...Waveform of echo from bottom surface of object, 44 ...waveform of echo at the end of the crack, 45...waveform converted to absolute value, 100...ultrasonic image evaluation device, 110...input section, 120...calculation section, 121...teacher label generation section, 122...judgment section, 123...extraction section, 124...detection target position calculation section, 130...judgment model generation section, 140...storage section, 141...learning ultrasonic image storage section, 142...teacher label storage section, 143...judgment model storage section, 144...judgment ultrasonic image storage section, 145...detection target position storage section, 150...output section, B...background, K...detection target, T...object, U...ultrasonic waves.

Claims (5)

a learning image acquisition unit that acquires a plurality of learning images to be used in machine learning, the learning images being at least one of an image obtained by a flaw detection device that detects flaws in a detection target included in an object to be inspected and an image generated by simulating the same;
a truth label generating unit that generates a truth label by setting a specific common area indicating a specific area of the detection target present in each of the learning images as a position of the detection target;
a judgment model generation unit that generates a judgment model that can calculate a judgment value that can determine whether or not the detection target is present for each unit indicating a position in an image, using the learning images and the teacher labels corresponding to each of the learning images;
an image acquisition unit for determining whether or not the flaw is present; and an image acquisition unit for acquiring at least one image for determining whether or not the flaw is present;
a determination unit that calculates the determination value for each unit indicating a position in the determination image by using the determination model;
Equipped with
the unit is a pixel,
The machine learning relates to segmentation,
the specific common area is an edge of the detection target,
the truth label generation unit generates the truth label by labeling only an edge of the detection object;
The teacher label is an image having the same number of pixels as the learning image, and is a binary image composed of pixel values indicating the specific common region and pixel values indicating other regions.
Image evaluation system. The flaw detection device is an ultrasonic flaw detection device that performs flaw detection using ultrasonic waves,
The maximum length of the specific common area is equal to or greater than a quarter of the wavelength of the ultrasonic frequency used by the flaw detection device.
The image evaluation system according to claim 1 . An extraction unit is further provided that extracts a position of the detection target from a determined image generated based on the determination value calculated from the determination image.
3. The image evaluation system according to claim 1 or 2 . The determination value is a continuous value calculated for each unit,
The extraction unit binarizes the determination value for each unit based on a threshold value to extract a position of the detection target.
The image evaluation system according to claim 3 . A step in which a learning image acquisition unit acquires a plurality of learning images to be used in machine learning, the learning images being at least one of images acquired by a flaw detection device that detects flaws in a detection target included in an object to be inspected or images generated by simulating the same;
A teacher label generating unit generates a teacher label in which a specific common area indicating a specific area of the detection target present in each of the learning images is set as a position of the detection target;
a determination model generating unit generating a determination model capable of calculating a determination value capable of determining whether or not the detection target is present for each unit indicating a position in an image, using the learning images and the teacher labels corresponding to each of the learning images;
A step in which at least one judgment image, which is an image obtained by the flaw detection device and is used for judgment, is acquired by a judgment image acquisition unit;
a determination unit calculating the determination value for each unit indicating a position in the determination image by using the determination model;
Including ,
the unit is a pixel,
The machine learning relates to segmentation,
the specific common area is an edge of the detection target,
the truth label generation unit generates the truth label by labeling only an edge of the detection object;
The teacher label is an image having the same number of pixels as the learning image, and is a binary image composed of pixel values indicating the specific common region and pixel values indicating other regions.
Image evaluation methods.

Images