JP7745558B2 - Computer program, information processing method, and information processing device - Google Patents

Description

The present invention relates to a computer program, an information processing method, and an information processing device.

Patent document 1 discloses an ultrasound guidance device that creates a guidance plan regarding a method for guiding an operator of an ultrasound diagnostic device, and guides the operator so that an echo image of a subject containing a specific anatomical image is captured.

Special table 2019-521745 publication

However, the ultrasound guidance device in Patent Document 1 only guides the operation of an ultrasound diagnostic device based on a guidance plan, and even if the operator follows the operation guide, whether images suitable for diagnosing a specific disease are obtained depends on the operator's skill. Furthermore, if a certain number of images suitable for diagnosing a specific disease cannot be selected from the obtained images, there is a problem in that a highly accurate diagnosis cannot be made.

The object of the present invention is to provide a computer program, information processing method, and information processing device that can select and collect images suitable for diagnosing a specific disease from a series of multiple images obtained by a scanning probe scanning a subject's organs, and output the amount of collected images.

The computer program of this embodiment causes a computer to perform the following processes: generate a series of images based on signals obtained from a scanning probe scanning an organ of a subject; determine whether the generated images are suitable for diagnosing a specified disease; store the images suitable for diagnosing the specified disease; and output a collection of the images suitable for diagnosing the specified disease.

The information processing method of this aspect generates a series of images based on signals obtained from a scanning probe scanning an organ of a subject, determines whether the generated images are suitable for diagnosing a specified disease, stores the images suitable for diagnosing the specified disease, and outputs a collection of the images suitable for diagnosing the specified disease.

The information processing device of this aspect includes a generation unit that generates a series of multiple images based on signals obtained from a scanning probe that scans the organs of a subject, a determination unit that determines whether the multiple images generated by the generation unit are suitable for diagnosing a specified disease, a memory unit that stores the images that the determination unit determines to be suitable for diagnosing the specified disease, and an output unit that outputs the collected amount of images suitable for diagnosing the specified disease.

In accordance with the above, it is possible to provide a computer program, information processing method, and information processing device that can select and collect images suitable for diagnosing a specific disease from a series of multiple images obtained by a scanning probe scanning a subject's organs, and output the amount of collected images.

1 is a schematic diagram illustrating an example of the configuration of an ultrasound diagnostic apparatus according to a first embodiment. 1 is a block diagram illustrating an example of the configuration of an information processing device according to a first embodiment. 1 is a functional block diagram illustrating an example of the configuration of an information processing device according to a first embodiment. FIG. 2 is a block diagram showing an example of the configuration of an appropriateness learning model according to the first embodiment. FIG. 2 is a block diagram showing an example of the configuration of an individual learning model. FIG. 1 is a schematic diagram showing an example of an echo image of a normal lung suitable for lung diagnosis. 1A and 1B are schematic diagrams showing examples of an echo image of an abnormal lung suitable for lung diagnosis and an echo image that is inappropriate for lung diagnosis. 1A and 1B are schematic diagrams showing examples of an echo image of an abnormal lung suitable for lung diagnosis and an echo image that is inappropriate for lung diagnosis. 1A and 1B are schematic diagrams showing examples of an echo image of an abnormal lung suitable for lung diagnosis and an echo image that is inappropriate for lung diagnosis. FIG. 1 is a block diagram showing an example of the configuration of an integrated learning model. 1 is a flowchart illustrating an information processing procedure according to the first embodiment. FIG. 2 is a schematic diagram showing an example of an ultrasound diagnostic monitor screen. FIG. 10 is an explanatory diagram showing a method for detecting a B line. FIG. 10 is a block diagram showing an example of the configuration of an index learning model according to a second embodiment. FIG. 11 is a block diagram showing an example of the configuration of an index learning model according to a third embodiment. FIG. 10 is a schematic diagram illustrating an example of the configuration of an ultrasound diagnostic apparatus according to a fourth embodiment. 10 is a flowchart illustrating an information processing procedure according to a fourth embodiment. FIG. 2 is a schematic diagram showing a predetermined scanning region. 13 is a flowchart illustrating an information processing procedure according to a fifth embodiment. FIG. 13 is a schematic diagram illustrating an example of the configuration of an ultrasound diagnostic apparatus according to a sixth embodiment. 13 is a flowchart illustrating an information processing procedure according to a sixth embodiment. FIG. 13 is a block diagram showing an example of the configuration of an appropriateness learning model according to a seventh embodiment.

Specific examples of a computer program, information processing method, and information processing device according to embodiments of the present invention are described below with reference to the drawings. The present invention is not limited to these examples, but is defined by the claims, and is intended to include all modifications within the meaning and scope of the claims. Furthermore, at least some of the embodiments described below may be combined in any manner.

Figure 1 is a schematic diagram illustrating an example configuration of an ultrasound diagnostic device according to embodiment 1. The ultrasound diagnostic device according to embodiment 1 includes an information processing device 1 and an ultrasound probe 2. The information processing device 1 and the ultrasound probe 2 are wirelessly connected, and can transmit and receive various signals. The ultrasound probe 2 may also be configured to be connected to the information processing device 1 via a wired cable.

The ultrasound probe 2 is a device that scans the subject's organs with ultrasound, and ultrasound scanning is controlled by the information processing device 1. The ultrasound probe 2 includes, for example, multiple piezoelectric elements, an acoustic matching layer, and an acoustic lens. The piezoelectric elements generate ultrasound waves in accordance with a drive signal output from the information processing device 1. The ultrasound waves generated by the piezoelectric elements are transmitted from the ultrasound probe 2 to the subject's living body via the acoustic matching layer and acoustic lens. The acoustic matching layer is a component that matches the acoustic impedance between the piezoelectric elements and the subject. The acoustic lens is an element that focuses the ultrasound waves propagating from the piezoelectric elements and transmits them to the subject. The ultrasound waves transmitted from the ultrasound probe 2 to the subject are reflected by discontinuous surfaces in the subject's organs where acoustic impedance is discontinuous and received by multiple piezoelectric elements. The amplitude of the reflected wave depends on the difference in acoustic impedance at the reflecting surface. The arrival time of the reflected wave depends on the depth of the reflecting surface. The piezoelectric elements convert the vibration pressure of the reflected ultrasound waves into an electrical signal. Hereinafter, this electrical signal will be referred to as an echo signal. The ultrasound probe 2 outputs the echo signal to the information processing device 1.

Figure 2 is a block diagram showing an example configuration of an information processing device 1 according to embodiment 1. The information processing device 1 is a computer equipped with a control unit 11, memory 12, storage unit 13, operation unit 14, display unit 15, and communication unit 16. Note that the information processing device 1 may be a multi-computer consisting of multiple computers, or may be a virtual machine virtually constructed by software.

The control unit 11 is a computing device such as one or more central processing units (CPUs), microprocessing units (MPUs), graphics processing units (GPUs), general-purpose computing on graphics processing units (GPGPUs), or tensor processing units (TPUs). By reading and executing a computer program 131 stored in the storage unit 13, the control unit 11 controls ultrasound scanning by the ultrasound probe 2, sequentially generates a series of multiple echo images in real time based on signals obtained from the ultrasound probe 2, determines whether the generated multiple echo images are suitable for diagnosing a predetermined lung disease (predetermined disease), displays the collection volume and target collection volume of echo images suitable for diagnosing the predetermined lung disease in real time in parallel with the echo image generation process, and, if more echo images than the target collection volume have been collected, calculates and displays an index for diagnosing the predetermined lung disease based on the echo images.

The communication unit 16 includes a processing circuit, a communication circuit, etc. for wireless communication processing, and transmits and receives various signals to and from the ultrasound probe 2. Specifically, the communication unit 16 generates ultrasound waves by transmitting a drive signal to the ultrasound probe 2 in accordance with the control of the control unit 11. The communication unit 16 then receives the echo signals output from the ultrasound probe 2.

Memory 12 is a volatile memory such as DRAM (Dynamic RAM) or SRAM (Static RAM), and temporarily stores computer program 131 read from storage unit 13 when executing the calculation processing of control unit 11, or various data generated by the calculation processing of control unit 11.

The memory unit 13 is a storage device such as a hard disk, an EEPROM (Electrically Erasable Programmable ROM), or a flash memory. The memory unit 13 stores a computer program 131 and an appropriateness learning model 17 required by the control unit 11 to collect echo images and perform diagnostic processing for specified lung diseases.

The computer program 131 is a program for causing a computer to function as the information processing device 1 according to this embodiment 1. The computer program 131 causes the computer to execute the information processing method according to this embodiment 1, such as collecting echo images and diagnosing a specified lung disease.

The computer program 131 may be recorded on the recording medium 10 in a computer-readable manner. The storage unit 13 stores the computer program 131 read from the recording medium 10 by a reading device (not shown). The recording medium 10 is a semiconductor memory such as a flash memory, an optical disk, a magnetic disk, a magneto-optical disk, etc. Alternatively, the computer program 131 according to this embodiment may be downloaded from an external server (not shown) connected to a communications network and stored in the storage unit 13.

The operation unit 14 is an input device that accepts operations by the operator using the ultrasound diagnostic device. The operator may be, for example, a medical professional such as a doctor, laboratory technician, or nurse. The input device may be, for example, a pointing device such as a touch panel, or a keyboard.

The display unit 15 is an output device that outputs information such as echo images, echo image collection levels, and pulmonary congestion levels. The output device is, for example, an LCD display or an EL display.

Figure 3 is a functional block diagram showing an example configuration of an information processing device 1 according to embodiment 1. The control unit 11 of the information processing device 1 reads and executes a computer program 131 stored in the memory unit 13, thereby functioning as a probe control unit 11a, an image generation unit 11b, a pulmonary diagnosis adequacy determination unit 11c, a pulmonary diagnostic image memory unit 11d, a pulmonary diagnostic image collection level display processing unit 11e, a pulmonary congestion level calculation unit 11f, and a pulmonary congestion level display processing unit 11g.

The probe control unit 11a controls the ultrasonic scanning process performed by the ultrasonic probe 2. Specifically, it generates ultrasonic waves by outputting a drive signal from the ultrasonic probe 2 and receives echo signals output from the ultrasonic probe 2.

The image generation unit 11b executes a process to generate echo images based on the echo signals received by the communication unit 16. The image generation unit 11b generates a series of echo images in real time each time the communication unit 16 receives an echo signal. The echo images are, for example, B-mode images in which the intensity of the reflected waves is represented by brightness, and reproduce two-dimensional tomographic images of organs. Note that the type of echo image is not particularly limited.

The lung diagnosis appropriateness determination unit 11c performs processing to determine whether the generated echo image is suitable for diagnosing a predetermined lung disease, such as pulmonary congestion. In the following description, the predetermined lung disease is assumed to be pulmonary congestion.

The pulmonary diagnostic image storage unit 11d performs a process to store echo images that are determined to be suitable for diagnosing pulmonary congestion.

The pulmonary diagnostic image collection level display processing unit 11e executes processing to display on the display unit 15 the collection level of echo images suitable for diagnosing pulmonary congestion and the target collection volume required to calculate the pulmonary congestion level. The pulmonary diagnostic image collection level display processing unit 11e calculates the collection volume of echo images in real time and displays it on the display unit 15.

When more than the target collection volume of echo images are collected, the pulmonary congestion calculation unit 11f performs a process to calculate the pulmonary congestion level based on the echo images.

The pulmonary congestion level display processing unit 11g performs processing to display the pulmonary congestion level calculated by the pulmonary congestion level calculation unit 11f on the display unit 15.

Figure 4 is a block diagram showing an example configuration of the appropriateness learning model 17 according to embodiment 1. The appropriateness learning model 17 comprises a plurality of individual learning models 171 and an integrated learning model 172. A plurality of echo images are input to each of the plurality of individual learning models 171. Each individual learning model 171 is a learning model that extracts features of the input echo images and outputs the extracted features to the integrated learning model 172. The integrated learning model 172 receives as input the features of the echo images output from the plurality of individual learning models 171. The integrated learning model 172 is a learning model that, when the features of a plurality of echo images are input, outputs an image appropriateness indicating the degree to which the plurality of echo images are appropriate for diagnosing pulmonary congestion.

FIG. 5 is a block diagram showing an example configuration of the individualized learning model 171. The individualized learning model 171 is a trained model that has been trained to extract and output feature quantities related to the diagnosis of pulmonary congestion from an echo image through machine learning using teacher data and unsupervised learning using an autoencoder. The individualized learning model 171 performs a predetermined calculation on input values and outputs the calculation results, and the memory unit 13 stores data such as coefficients and thresholds of functions that define this calculation as the individualized learning model 171. By reading the data stored as the individualized learning model 171, the control unit 11 can execute calculation processing to extract features of the echo image.
In the present embodiment 1, the learning process of the individual learning model 171 is performed by a learning computer. Note that, like the computer program 131, data related to the learned individual learning model 171 may be provided in the form of distribution via a communication network or recorded on a recording medium 10.

In the first embodiment, the individualized learning model 171 is, for example, a neural network having an input layer 171a to which an echo image is input and an intermediate layer 171b that extracts features of the echo image. The individualized learning model 171 is configured using, for example, an autoencoder. As shown in FIG. 5 , the autoencoder includes an input layer 171a to which an echo image is input, a first intermediate layer 171b that dimensionally compresses the input image to extract features, a second intermediate layer 171c that restores the echo image from the extracted features, and an output layer 171d that outputs the restored echo image. The first intermediate layer 171b and the second intermediate layer 171c are also referred to as a convolution layer and a deconvolution layer. The individualized learning model 171 is configured with an input layer 171a to which an echo image of the encoder is input and an intermediate layer 171b that dimensionally compresses the input image to extract features. 5, the second intermediate layer 171c and output layer 171d, which are drawn with dashed lines, are not essential components of the individual learning model 171. Note that the configuration of the individual learning model 171 is not particularly limited as long as it is configured so that feature amounts of the echo image can be extracted and provided to the subsequent integrated learning model 172, and the above autoencoder may be configured as the individual learning model 171 as is.
Below, an example of constructing an individualized learning model 171 using the input layer 171a and first hidden layer 171b of an autoencoder will be described.

The input layer 171a of the neural network has multiple neurons to which the pixel values of each pixel in the echo image are input, and passes each input data to the intermediate layer 171b.

The first intermediate layer 171b has multiple layers consisting of multiple neurons. The intermediate layer 171b is a layer that compresses the dimensionality of image data. For example, the intermediate layer 171b compresses the dimensionality of echo images by performing convolution processing. Through dimensional compression, each layer extracts features of the echo image of normal lungs and features of the echo image of abnormal lungs from the input data and passes them on sequentially from the previous layer to the subsequent layer. The final layer of the first intermediate layer 171b outputs the features extracted from the echo images. Although the meaning of the features cannot be immediately interpreted, it is thought that image features that appear in normal and abnormal lungs, such as the presence or absence of a real image, A-lines 31, B-lines 33, ground-glass opacity 34, and bat sign 32, are related.

The following describes a learning method for the individual learning model 171. First, an autoencoder before learning is prepared. The autoencoder includes an input layer 171a, a first hidden layer 171b, a second hidden layer 171c, and an output layer 171d.
The computer collects multiple echo images of normal lungs and multiple echo images of abnormal lungs. That is, it collects multiple echo images suitable for diagnosing a predetermined lung disease. Next, the computer uses the collected echo images to perform machine learning or deep learning on the pre-trained autoencoder so that the echo images input to the input layer 171a and the images output from the output layer 171d are the same.
Specifically, the computer inputs multiple echo images, which serve as training data, into the pre-trained autoencoder, and obtains an image output from the output layer 171d after arithmetic processing in the first and second intermediate layers 171b and 171c. The computer then compares the image output from the output layer 171d with the input echo image and optimizes parameters used in the arithmetic processing in the intermediate layers 171b and 171c so that the image output from the output layer 171d approaches the input echo image. These parameters include, for example, weights (coupling coefficients) between neurons. While the parameter optimization method is not particularly limited, the computer may optimize various parameters using, for example, the steepest descent method. The input layer 171a and the first intermediate layer 171b are then extracted from the trained autoencoder to generate the individualized training model 171.

Note that while supervised learning has been exemplified above, the individual learning model 171 may also be generated by supervised learning using a CNN (Convolutional Neural Network). Also, while an example has been described in which the individual learning model 171 is a neural network, it may also be a model configured as an SVM (Support Vector Machine), a Bayesian network, a regression tree, or the like.

Figure 6 is a schematic diagram showing an example of an echo image of a normal lung suitable for lung diagnosis. An appropriate echo image of a normal lung includes a clear A-line 31. The A-line 31 is an image caused by multiple reflections occurring between the pleura and the ultrasound probe 2. An appropriate echo image of a normal lung in a sagittal plane also includes an image known as the Batt sign 32. The Batt sign 32 is a curved, convex image obtained when ultrasound is reflected by the ribs. These A-line 31 and Batt sign 32 are features contained in an echo image suitable for diagnosing a specific lung disease (diagnosing the lungs as normal).

Figures 7A to 7C are schematic diagrams showing examples of echo images of abnormal lungs suitable for lung diagnosis and unsuitable echo images. Figure 7A is a schematic diagram of an echo image of abnormal lungs in which B-lines 33 are visible, and Figure 7B is a schematic diagram of an echo image of abnormal lungs in which ground-glass opacities 34 are visible. Figure 7C is an echo image without a solid image, and is unsuitable for diagnosing a specific lung disease. B-lines 33 are images caused by thickening of the interlobular septa and accumulation of fluid in the alveoli. Ground-glass opacities 34 are images caused by abnormal lungs such as pneumonia. These B-lines 33 and ground-glass opacities 34 are features contained in echo images suitable for diagnosing a specific lung disease.

FIG. 8 is a block diagram showing an example configuration of the integrated learning model 172. The integrated learning model 172 is a trained model that has been trained through unsupervised learning, such as machine learning using teacher data and clustering, to output an image suitability indicating the degree to which a plurality of echo images are suitable for diagnosing a predetermined lung disease, based on the feature quantities of the echo images. The integrated learning model 172 performs a predetermined calculation on input values and outputs the calculation results. The memory unit 13 stores data such as coefficients and thresholds of functions that define this calculation as the integrated learning model 172. By reading the data stored as the integrated learning model 172, the control unit 11 can execute calculation processing to determine the suitability of an echo image based on the feature quantities of the echo image.
In the present embodiment 1, the learning process of the integrated learning model 172 is performed by a learning computer. Data related to the trained integrated learning model 172 may be provided in the form of distribution via a communication network, similar to the computer program 131, or may be provided in the form of being recorded on the recording medium 10.

In this embodiment 1, the integrated learning model 172 is a neural network having, for example, an input layer 172a to which features of multiple echo images are input, an intermediate layer 172b that extracts features of the echo images, and an output layer 172c that outputs the extracted features.

The input layer 172a of the neural network has multiple neurons to which the features of multiple echo images are input, and passes each input data to the intermediate layer 172b.

The intermediate layer 172b has multiple layers made up of multiple neurons. Each layer extracts features related to the appropriateness of the echo image from the input data and passes them on to the subsequent layer in sequence, with the final layer passing them on to the output layer 172c.

The output layer 172c has neurons that output the calculation results, and these neurons output image suitability indicating the degree to which multiple echo images are suitable for diagnosing a specified lung disease.

In this embodiment 1, an example was described in which the integrated learning model 172 is a neural network, but it may also be a model configured as an SVM (Support Vector Machine), a Bayesian network, a regression tree, or the like.

The training method for the integrated learning model 172 will be described below. First, the computer collects feature quantities of multiple echo images that serve as the basis for training data. Then, the computer generates training data by assigning training data indicating whether the images are suitable for diagnosing a predetermined lung disease to the multiple feature quantities. Next, the computer generates the integrated learning model 172 by using the generated training data to perform machine learning or deep learning on the pre-trained neural network model.
Specifically, the computer inputs feature quantities of multiple echo images included in the training data into a pre-training neural network model, and after arithmetic processing in the intermediate layer 172b, obtains an image appropriateness level output from the output layer 172c. The computer then compares the image appropriateness level output from the output layer 172c with the image appropriateness level indicated by the training data, and optimizes parameters used in the arithmetic processing in the intermediate layer 172b so that the image appropriateness level output from the output layer 172c approaches a correct value. The parameters include, for example, weights (coupling coefficients) between neurons. The method for optimizing the parameters is not particularly limited, and the computer may optimize various parameters using, for example, the steepest descent method.

The information processing device 1 obtains a trained integrated learning model 172 by repeatedly performing the above process based on the training data of a large number of patients included in the training data.

Fig. 9 is a flowchart showing an information processing procedure according to the first embodiment, and Fig. 10 is a schematic diagram showing an example of an ultrasound diagnostic monitor screen. The control unit 11 of the information processing device 1 displays a monitor screen as shown in Fig. 10 on the display unit 15 (step S111). The monitor screen includes an echo image display unit 151, a collection level gauge 152, a lung diagnostic image display unit 153, a congestion level display unit 154, a start button 155, a stop button 156, etc.
The echo image display unit 151 displays echo images generated based on the echo signals in real time. The collection level gauge 152 displays the collection level of echo images suitable for diagnosing a predetermined pulmonary disease among the generated echo images. The collection level gauge 152 displays the target collection volume of echo images required for diagnosing a predetermined pulmonary disease using a predetermined number of meter blocks 152a. Furthermore, when the number of echo images corresponding to one meter block 152a has been collected, the collection level gauge 152 displays the collection volume of echo images by changing the color of the meter blocks 152a, starting from the bottom. When the color of all meter blocks 152a has changed, the collection volume has reached the target collection volume. The lung diagnostic image display unit 153 displays representative echo images suitable for diagnosing a predetermined pulmonary disease. The congestion level display unit 154 displays the diagnosis result of the predetermined pulmonary disease. The start button 155 is an operation button for starting the collection of echo images and the diagnostic process for the predetermined pulmonary disease, and the stop button 156 is an operation button for stopping the process.

Next, the control unit 11 receives the echo signals output from the ultrasound probe 2 and generates an echo image based on the received echo signals (step S112).The control unit 11 then displays the generated echo signals on the echo image display unit 151 (step S113).

Next, the control unit 11 calculates the image appropriateness by inputting each of the multiple echo images into the individual learning model 171 (step S114), and determines whether the multiple echo images are appropriate for diagnosing a predetermined lung disease based on the calculated image appropriateness (step S115). If the control unit 11 determines that the multiple echo images are inappropriate (step S115: NO), the control unit 11 returns the process to step S112.

If it is determined that the image is suitable for diagnosing a specific lung disease (step S115: YES), the control unit 11 stores the echo image determined to be suitable in the memory unit 13 (step S116). In other words, it collects the echo image.

Next, the control unit 11 calculates the collection amount of echo images suitable for diagnosing the predetermined lung disease (step S117) and displays the calculated collection amount on the collection level gauge 152 (step S118). The control unit 11 that displays the collection amount functions as an output unit that outputs the collection amount of the images suitable for diagnosing the predetermined disease.
The control unit 11 also displays a representative echo image suitable for lung diagnosis as a sample on the lung diagnostic image display unit 153 (step S119).

Next, the control unit 11 determines whether the amount of echo images collected has reached a predetermined target amount (step S120). If it determines that the target amount has not been reached (step S120: NO), the control unit 11 returns the process to step S112.

If it is determined that the target collection volume has been reached (step S120: YES), the control unit 11 calculates the pulmonary congestion level (step S121) and displays the calculated pulmonary congestion level on the congestion level display unit 154 (step S122). Processing then ends.

Figure 11 is an explanatory diagram showing a method for detecting B-lines 33. The left diagram in Figure 11 is an echo image generated based on the echo signal. However, it is a polar coordinate image in which the horizontal axis indicates the depth direction and the vertical axis indicates the angle indicating the direction in which the ultrasound is transmitted. The brightness of each pixel corresponds to the amplitude of the echo signal.

The control unit 11 integrates the brightness value of each pixel in the depth direction for the echo image. The diagram in the center of Figure 11 is a graph conceptually showing the integration results. The horizontal axis represents the angle, and the vertical axis represents the integrated value.

The control unit 11 then differentiates the integral value with respect to the angular direction. The right diagram in Figure 11 is a graph conceptually illustrating the differentiation results. The horizontal axis represents the angle, and the vertical axis represents the differential value. The control unit 11 determines that a location where the differential value is equal to or greater than a predetermined value is a B-line 33. The control unit 11 then counts the number of echo images in which a B-line 33 is present among the multiple echo images collected, and determines that pulmonary congestion exists if a B-line 33 is present in a predetermined percentage or more, for example, 37.5% or more of the echo images.

If the control unit 11 determines that pulmonary congestion exists, it displays, for example, the percentage of B-lines 33 present on the congestion level display unit 154. If the control unit 11 determines that pulmonary congestion exists, it displays on the congestion level display unit 154 that there are no findings.

As described above, the ultrasound diagnostic device of this embodiment 1 can select and collect images suitable for diagnosing a specific lung disease from a series of multiple images obtained by a scanning probe scanning the subject's organs, and can output the amount of collected images.

In addition, the image collection level and target collection level suitable for diagnosing a specified lung disease can be displayed in real time.

Furthermore, images suitable for diagnosing specific lung diseases can be displayed.

Furthermore, when the collection level of images suitable for diagnosing a specified lung disease reaches the target collection level, the degree of pulmonary congestion can be calculated and displayed based on the collected images.

Note that, although embodiment 1 describes an example of scanning the lungs of a subject, the present invention can also be applied to scanning other organs. Furthermore, although embodiment 1 describes an example of scanning the organs of a subject using ultrasound, the present invention can also be applied to cases where a scanning probe that optically acquires tomographic images of the organ is used, such as a probe for optical coherence tomography diagnosis.

In addition, in this embodiment 1, an example has been described in which the display unit 15 of the information processing device 1 constituting the ultrasound diagnostic device displays the collection level and target collection level of images suitable for diagnosing a specified lung disease, scanned images, representative images suitable for diagnosing a specified lung disease, etc., but the information processing device 1 may also be configured to output and display these various pieces of information to an external monitor device.

Furthermore, in this embodiment 1, an example was described in which multiple individual learning models 171 and an integrated learning model 172 are used to determine whether an image is suitable for diagnosing a specified lung disease, but it may also be configured to use a single learning model to individually determine the suitability of multiple images.

(Embodiment 2)
The ultrasound diagnostic device according to the second embodiment differs from the first embodiment in that it uses a learning model to diagnose a predetermined lung disease based on the feature quantities of an echo image. Since the other configurations of the information processing device 1 are the same as those of the information processing device 1 according to the first embodiment, the same reference numerals are used for the same parts, and detailed descriptions thereof will be omitted.

FIG. 12 is a block diagram showing an example of the configuration of an index learning model 218 according to the second embodiment. The storage unit 13 of the information processing device 1 according to the second embodiment stores the index learning model 218. The index learning model 218 is a trained model that has been trained to output the degree of pulmonary congestion from the feature quantities of an echo image through unsupervised learning, such as machine learning using teacher data and clustering. The feature quantities of the echo image include, for example, the presence or absence of B-lines 33, the number of B-lines 33, the presence or absence of ground-glass opacity 34, and the contrast of the bat sign 32. The index learning model 218 performs a predetermined calculation on input values and outputs the calculation results. The storage unit 13 stores data such as coefficients and thresholds of functions that define this calculation as the index learning model 218. By reading the data stored as the index learning model 218, the control unit 11 can execute calculation processing to calculate the degree of pulmonary congestion from the feature quantities of the echo image.
In the second embodiment, a learning computer performs the learning process of the index learning model 218. Data related to the learned index learning model 218 may be provided in the form of distribution via a communication network, similar to the computer program 131, or may be provided in the form of being recorded on the recording medium 10.

In this embodiment 2, the indicator learning model 218 is a neural network having, for example, an input layer 218a to which features of an echo image are input, an intermediate layer 218b that extracts features of the echo image, and an output layer 218c that outputs the extracted features.

The input layer 218a of the neural network has multiple neurons to which the features of the echo image are input, and passes each input data to the intermediate layer 218b.

The intermediate layer 218b has multiple layers consisting of multiple neurons. Each layer extracts features of pulmonary congestion from the input data and passes them on to the subsequent layer in sequence, with the final layer passing them on to the output layer 218c.

The output layer 218c includes a neuron that outputs the degree of pulmonary congestion, and the neuron outputs the degree of pulmonary congestion.

In this embodiment 2, an example was described in which the indicator learning model 218 is a neural network, but it may also be a model configured as an SVM (Support Vector Machine), a Bayesian network, a regression tree, or the like.

The learning method of the index learning model 218 will be described. First, the computer collects feature quantities of multiple echo images that serve as the source of training data. Then, the computer generates training data by assigning training data indicating the degree of pulmonary congestion to the feature quantities of the echo images. Next, the computer generates the index learning model 218 by using the generated training data to perform machine learning or deep learning on the pre-training neural network model.
Specifically, the computer inputs feature quantities of multiple echo images included in the training data into the pre-training neural network model, and after arithmetic processing in the intermediate layer 218b, obtains the pulmonary congestion level output from the output layer 218c. The computer then compares the pulmonary congestion level output from the output layer 218c with the pulmonary congestion level indicated by the training data, and optimizes parameters used in the arithmetic processing in the intermediate layer 218b so that the pulmonary congestion level output from the output layer 218c approaches the correct value. The parameters include, for example, weights (coupling coefficients) between neurons. While the parameter optimization method is not particularly limited, the computer may optimize various parameters using, for example, the steepest descent method.

The information processing device 1 obtains a trained indicator learning model 218 by repeatedly performing the above process based on the training data of a large number of patients included in the training data.

By inputting the feature quantities of the echo image determined to be suitable for diagnosing pulmonary disease into the index learning model 218 configured in this manner, the degree of pulmonary congestion can be calculated.
The feature quantities of the echo image can be calculated using, for example, a learning model (not shown). The learning model is, for example, a neural network having an input layer to which the echo image is input, an intermediate layer that extracts the feature quantities of the echo image, and an output layer. The learning model is a convolutional neural network (CNN) that includes multiple convolution layers, pooling layers, fully connected layers, etc. The feature quantities include, for example, the presence or absence of a real image, the presence or absence of an A-line 31, the presence or absence of a B-line 33, the number of B-lines 33, the presence or absence of a ground-glass opacity 34, the presence or absence of a bat sign 32, etc. The above-described contents of each neuron and output data are merely examples and are not particularly limited.
The learning method for the learning model will be described below. First, the computer collects multiple echo images that serve as the source of training data. Then, the computer generates training data by adding training data indicating feature quantities such as the presence or absence of a real image and the presence or absence of a B-line 33 to the multiple echo images. Next, the computer uses the generated training data to perform machine learning or deep learning on the pre-training neural network model, thereby generating an individualized learning model 171.
Specifically, the computer inputs the echo images included in the training data into a pre-training neural network model, performs arithmetic processing in the intermediate layer, and obtains feature values output from the output layer. The computer then compares the feature values output from the output layer with the feature values indicated by the training data, and optimizes the parameters used in the arithmetic processing in the intermediate layer so that the data output from the output layer approaches the correct value.

As described above, with the ultrasound diagnostic device of this embodiment 2, the degree of pulmonary congestion can be calculated by inputting features of the echo image, such as the presence or absence of B-lines 33, the number of B-lines 33, the presence or absence of ground-glass opacities 34, and the contrast of the bat sign 32, into the index learning model 218.

(Embodiment 3)
The ultrasound diagnostic device according to embodiment 3 differs from embodiment 1 in that the ultrasound image itself is input into a learning model to diagnose a predetermined lung disease. The other configurations of the information processing device 1 are the same as those of the information processing device 1 according to embodiment 2, and therefore the same parts are denoted by the same reference numerals and detailed description thereof will be omitted.

Figure 13 is a block diagram showing an example configuration of an index learning model 318 according to embodiment 3. The memory unit 13 of the information processing device 1 according to embodiment 3 stores the index learning model 318. The index learning model 318 is a neural network having, for example, an input layer 318a to which an echo image is input, an intermediate layer 318b that extracts the echo image, and an output layer 318c that outputs the extracted feature amount.

The input layer 318a of the neural network has multiple neurons to which the pixel values of each pixel in the echo image are input, and passes each input data to the intermediate layer 318b.

The intermediate layer 318b has multiple layers each consisting of multiple neurons. For example, the indicator learning model 318 in embodiment 3 is a CNN, which includes multiple convolutional layers, pooling layers, fully connected layers, etc. Each layer extracts features of pulmonary congestion from the input data and passes them on sequentially from the previous layer to the subsequent layer, with the final layer passing them on to the output layer 318c.

The output layer 318c includes a neuron that outputs the degree of pulmonary congestion, and the neuron outputs the degree of pulmonary congestion.

The method for generating the index learning model 318 is the same as in embodiment 2. By inputting an echo image determined to be suitable for diagnosing pulmonary disease into the index learning model 318 configured in this manner, the degree of pulmonary congestion can be calculated.

As described above, according to the ultrasound diagnostic apparatus of this embodiment 3, the degree of pulmonary congestion can be calculated by inputting an echo image into the index learning model 318.

(Embodiment 4)
The ultrasound diagnostic device according to embodiment 4 differs from embodiment 1 in that it generates a predetermined number of echo images or more at a plurality of predetermined scanning regions to diagnose pulmonary diseases. Since the other configurations of the information processing device 1 are the same as those of the information processing device 1 according to embodiment 1, the same reference numerals are used for the same parts, and detailed descriptions thereof will be omitted.

Figure 14 is a schematic diagram illustrating an example configuration of an ultrasound diagnostic apparatus according to embodiment 4. The ultrasound probe 2 according to embodiment 4 includes an acceleration sensor 421 (positioning sensor) and outputs an acceleration signal to the information processing device 1. The information processing device 1 receives the acceleration signal output from the ultrasound probe 2 and estimates the position of the ultrasound probe 2 based on the received acceleration signal. Specifically, the information processing device 1 uses the position of the ultrasound probe 2 relative to the subject when the ultrasound sensor was started as a reference position and estimates the position of the ultrasound probe 2 relative to this reference position based on the acceleration signal.

Figure 15 is a flowchart showing the information processing procedure according to embodiment 4. The control unit 11 of the information processing device 1 displays the monitor screen in a procedure similar to steps S111 to S113 of embodiment 1 (step S411), generates an echo image (step S412), and displays the generated echo image on the display unit 15 (step S413).

Next, the control unit 11 receives the acceleration signal output from the ultrasound probe 2 and estimates the position of the ultrasound probe 2 relative to the scanning area, i.e., the subject, based on the received acceleration information (step S414).

Figure 16 is a schematic diagram showing predetermined scanning regions. In this embodiment 4, as shown in Figure 16, it is assumed that four scanning regions (regions indicated by numbers "1," "2," "3," and "4" in Figure 16) obtained by dividing the right lung into upper, lower, left, and right sections, and four scanning regions (regions indicated by numbers "5," "6," "7," and "8" in Figure 16) obtained by dividing the left lung into upper, lower, left, and right sections, are scanned in numerical order. The information processing device 1 estimates the position of the ultrasound probe 2 when the start button 155 on the monitor screen is first operated and the first real image begins to be obtained as scanning region "1." Thereafter, the control unit 11 calculates the position of the ultrasound probe 2 relative to scanning region "1" based on the acceleration signal, and estimates the position of the ultrasound probe 2 relative to the subject.

Next, the control unit 11 calculates the amount of echo images generated at the above-mentioned specified multiple scanning locations (step S415) and determines whether or not a specified amount of echo images or more have been generated at each location (step S416).

If it is determined that there is a scanning region where the amount of echo images generated is less than the predetermined amount (step S416: YES), the control unit 11 displays an instruction image (movement instruction information) on the display unit 15 indicating that the ultrasound probe 2 should be moved to the scanning region where the amount of echo images generated is less than the predetermined amount (step S417). Note that if the ultrasound probe 2 is currently located in the scanning region where the amount of echo images generated is less than the predetermined amount, the instruction image is not displayed. Furthermore, if there are multiple regions where the amount of echo images generated is less than the predetermined amount, it is preferable to configure the system to instruct the user to move the probe to the scanning region with the lowest number of scanning regions, for example, "1" to "8."

When the processing of step S417 is completed, or when it is determined in step S416 that there are no scans less than the predetermined amount (step S416: NO), the control unit 11 inputs each of the generated multiple echo images into the individual learning model 171, thereby determining whether the echo images are suitable for diagnosing a predetermined lung disease (step S114). The processing from step S114 onwards is the same as in embodiment 1, so details are omitted.

As described above, the ultrasound diagnostic device of this embodiment 4 can scan each part of the lungs thoroughly to generate and collect echo images, thereby enabling accurate diagnosis of lung diseases.

(Embodiment 5)
The ultrasound diagnostic device according to embodiment 5 differs from embodiment 1 in that it can instruct the surgeon on the position of the ultrasound probe 2 so that echo images suitable for diagnosing lung diseases are efficiently collected. Since the other configurations of the information processing device 1 are the same as those of the information processing device 1 according to embodiment 1, the same reference numerals are used for the same parts and detailed description will be omitted.

The ultrasonic probe 2 according to embodiment 5, like that of embodiment 4, is equipped with an acceleration sensor 421 (posture sensor) and outputs an acceleration signal to the information processing device 1. The information processing device 1 receives the acceleration signal output from the ultrasonic probe 2 and estimates the posture of the ultrasonic probe 2 based on the received acceleration signal.

Figure 17 is a flowchart showing the information processing procedure according to embodiment 5. The control unit 11 of the information processing device 1 displays the monitor screen in a procedure similar to steps S111 to S113 of embodiment 1 (step S511), generates an echo image (step S512), and displays the generated echo image on the display unit 15 (step S513).

Next, the control unit 11 receives the acceleration signal output from the ultrasound probe 2 and estimates the posture of the ultrasound probe 2 relative to the subject based on the received acceleration information (step S514).

Next, the control unit 11 inputs each of the generated multiple echo images into the individual learning model 171 to calculate an image suitability indicating the degree to which the echo image is suitable for diagnosing the specified lung disease (step S515), and determines whether the echo image is suitable for diagnosing the specified lung disease based on the calculated image suitability (step S516).

If it is determined that the image is suitable for diagnosing a specified lung disease (step S516: YES), the control unit 11 stores the echo image determined to be appropriate in the memory unit 13 (step S517) and stores posture information indicating the posture of the ultrasound probe 2 (step S518).

As in steps S117 to S120 of embodiment 1, the control unit 11 calculates the amount of echo images collected (step S519), displays this on the collection level gauge 152 (step S520), and displays a representative echo image suitable for lung diagnosis as a sample on the lung diagnostic image display unit 153 (step S521). The subsequent processing is the same as in embodiment 1, so details are omitted.

If it is determined in step S516 that the echo image is not suitable for diagnosing the specified pulmonary disease (step S516: NO), the control unit 11 determines whether the amount of echo images suitable for diagnosing the specified pulmonary disease has decreased (step S523). If it is determined that the amount of echo images suitable for diagnosing the specified pulmonary disease has decreased (step S523: YES), the control unit 11 reads out the posture information of the ultrasound probe 2 stored in the memory unit 13, displays the read posture information or an instruction image (posture change instruction information) instructing a posture change based on the posture information on the display unit 15 (step S524), and returns the process to step S512. If it is determined that the amount of echo images suitable for diagnosing the specified pulmonary disease has not decreased (step S523: NO), the control unit 11 returns the process to step S512.

As described above, the ultrasound diagnostic device of this embodiment 5 can instruct the surgeon to position the ultrasound probe 2 in a way that generates an echo image suitable for diagnosing a specific lung disease.

(Embodiment 6)
The ultrasound diagnostic apparatus according to embodiment 6 differs from embodiment 1 in that it collects echo images taking into account the respiratory cycle of the subject. Since the other configurations of the information processing apparatus 1 are the same as those of the information processing apparatus 1 according to embodiment 1, the same reference numerals are used for the same parts and detailed description thereof will be omitted.

Figure 18 is a schematic diagram illustrating an example configuration of an ultrasound diagnostic apparatus according to embodiment 6. The ultrasound diagnostic apparatus according to embodiment 6 includes a respiratory cycle sensor 603. The respiratory cycle sensor 603 is, for example, a PPG (Photoplethysmography) pulse wave sensor, an acceleration sensor 421, or a body movement sensor that detects body movement due to breathing using a piezoelectric element or the like. The respiratory cycle sensor 603 transmits a signal corresponding to the respiratory cycle of the subject to the information processing device 1. The information processing device 1 receives the signal transmitted from the respiratory cycle sensor 603.

Figure 19 is a flowchart showing the information processing procedure according to embodiment 6. The control unit 11 of the information processing device 1 displays the monitor screen in a procedure similar to steps S111 to S113 of embodiment 1 (step S611), generates an echo image (step S612), and displays the generated echo image on the display unit 15 (step S613).

Next, the control unit 11 receives the respiratory cycle signal transmitted from the respiratory cycle sensor 603 (step S614) and associates the information of the received respiratory cycle signal with the echo image generated at the same time (step S615).

Next, the control unit 11 selects multiple echo images generated at the same breathing timing (step S616). For example, the control unit 11 selects multiple echo images associated with information on the cycle signal of the inhalation timing. Alternatively, the control unit 11 may select multiple echo images associated with information on the breathing cycle signal of the same inhalation timing.

Then, the control unit 11 calculates the image appropriateness by inputting the selected echo image of the same breathing timing into the appropriateness learning model 17 (step S617). After completing the processing of step S617, the control unit 11 executes the processing of steps S115 to S122 of embodiment 1.

As described above, the ultrasound diagnostic apparatus of this embodiment 6 is configured to input echo images generated at the same respiratory cycle timing into the appropriateness learning model 17, thereby making it possible to more accurately determine the appropriateness of the echo images.

When the appropriateness learning model 17 is trained by machine learning using only echo images at the exhalation timing, it is preferable to select an echo image at the exhalation timing and input it to the appropriateness learning model 17. Similarly, when the appropriateness learning model 17 is trained by machine learning using only echo images at the inhalation timing, it is preferable to select an echo image at the inhalation timing and input it to the appropriateness learning model 17. Furthermore, a first appropriateness learning model trained by machine learning using echo images at the inhalation timing and a second appropriateness learning model trained by machine learning using echo images at the inhalation timing may be provided, and the echo image generated at the exhalation timing may be input to the first appropriateness learning model, and the echo image generated at the inhalation timing may be input to the second appropriateness learning model.

(Embodiment 7)
The ultrasound diagnostic device according to embodiment 7 differs from embodiment 1 in that it collects echo images obtained using a plurality of ultrasound frequencies. Since the other configurations of the information processing device 1 are the same as those of the information processing device 1 according to embodiment 1, the same reference numerals are used for the same parts, and detailed description thereof will be omitted.

Figure 20 is a block diagram showing an example configuration of an appropriateness learning model 17 according to embodiment 7. The information processing device 1 periodically switches between a drive signal for transmitting ultrasound waves of a first frequency and a drive signal for transmitting ultrasound waves of a second frequency, outputs the signals to the ultrasound probe 2, and receives echo signals. The switching period is short enough to allow the same scanning area to be scanned at both the first and second frequencies. By using ultrasound waves of different frequencies, different echo images can be obtained for the same scanning area. Generally, the higher the frequency of ultrasound, the higher the resolution but the lower the penetration power, and the lower the frequency, the lower the resolution and the higher the penetration power.

The control unit 11 of the information processing device 1 calculates the image appropriateness by inputting multiple echo images obtained with ultrasound of a first frequency and multiple echo images obtained with ultrasound of a second frequency into the appropriateness learning model 17.

By inputting echo images obtained using ultrasound of different frequencies for the same scanning area into the appropriateness learning model 17, more detailed information about the echo image can be obtained, and the image appropriateness can be calculated to more accurately indicate whether the image is appropriate for diagnosing pulmonary disease.

As described above, with the ultrasound diagnostic device of this embodiment 7, by inputting multiple echo images obtained using ultrasound of different frequencies into the appropriateness learning model 17, it is possible to more accurately determine and collect whether or not the echo images are suitable for diagnosing a specific lung disease.

REFERENCE SIGNS LIST 1 Information processing device 2 Ultrasonic probe 10 Recording medium 11 Control unit 11a Probe control unit 11b Image generation unit 11c Pulmonary diagnosis appropriateness determination unit 11d Pulmonary diagnostic image storage unit 11e Pulmonary diagnostic image collection degree display processing unit 11f Pulmonary congestion degree calculation unit 11g Pulmonary congestion degree display processing unit 12 Memory 13 Storage unit 14 Operation unit 15 Display unit 16 Communication unit 17 Adequacy degree learning model 31 A-line 32 Bat sign 33 B-line 34 Ground-glass opacity 131 Computer program 151 Echo image display unit 152 Collection degree gauge 152a Meter block 153 Pulmonary diagnostic image display unit 154 Congestion degree display unit 155 Start button 156 Stop button 171 Individual learning model 172 Integrated learning model 218 Indicator learning model 603 Respiratory cycle sensor 421 Acceleration sensor

Claims (15)

generating a series of images based on signals obtained from a scanning probe scanning an organ of the subject;
selecting the plurality of images generated at the same respiratory timing based on a respiratory cycle signal output from a respiratory cycle sensor that detects the respiratory cycle of the subject;
determining whether the selected images at the same respiratory timing are suitable for diagnosing a predetermined disease;
storing the plurality of images at the same breathing timing determined to be suitable for diagnosing the predetermined disease as a plurality of suitable images;
and outputting the collection of the plurality of appropriate images. The process of generating the plurality of images includes:
sequentially generating the plurality of images in real time based on signals obtained from the scanning probe;
The process of outputting the collected amount includes:
The computer program of claim 1 , further comprising a process for displaying the collection volume and a predetermined target collection volume in real time in parallel with a process for generating the plurality of images based on signals obtained from the scanning probe. determining whether the plurality of suitable images has been collected at least as large as the target collection amount;
calculating an index for diagnosing the predetermined disease based on the plurality of appropriate images having a collection amount equal to or greater than the target collection amount suitable for diagnosing the predetermined disease;
The computer program according to claim 2 , for causing the computer to execute a process of displaying the calculated index. calculating an index for diagnosing the predetermined disease based on the plurality of appropriate images suitable for diagnosing the predetermined disease;
The computer program according to claim 1 or 2, for causing the computer to execute a process of outputting the calculated index. the scanning probe is an ultrasound probe;
The process of generating the plurality of images includes:
generating a series of the plurality of images based on signals obtained from the scanning probe scanning the lungs of the subject ;
The process of calculating the index includes:
Detecting B-lines in at least the plurality of suitable images;
The computer program according to claim 3 or 4, wherein the index relating to pulmonary congestion is calculated based on the detected B-lines. the scanning probe is an ultrasound probe;
The process of generating the plurality of images includes:
generating a series of the plurality of images based on signals obtained from the scanning probe scanning the lungs of the subject ;
The process of calculating the index includes:
Detecting B-lines, ground-glass opacities, and bat signs based on ultrasound reflections from ribs in the plurality of suitable images;
The computer program according to claim 3 or 4, wherein the index relating to pulmonary congestion is calculated based on the presence or absence or number of B-lines, the presence or absence of ground-glass opacity, and the contrast of the Batt's sign. the scanning probe is an ultrasound probe;
The process of generating the plurality of images includes:
generating a series of the plurality of images based on signals obtained from the scanning probe scanning the lungs of the subject ;
The process of calculating the index includes:
5. The computer program according to claim 3, further comprising a process of calculating the index related to pulmonary congestion by inputting the plurality of appropriate images into an index learning model that outputs the index related to pulmonary congestion when the plurality of appropriate images are input. the predetermined disease is pulmonary congestion;
determining whether the plurality of suitable images has been collected at least as large as the target collection amount;
detecting B-lines in the plurality of suitable images that are equal to or greater than the target acquisition volume suitable for diagnosing pulmonary congestion;
determining that pulmonary congestion exists when a predetermined ratio or more of the images in which the B-line is detected are present among the plurality of appropriate images;
If pulmonary congestion is determined, the proportion of B-lines present in the plurality of appropriate images is displayed.
3. A computer program product according to claim 2, for causing the computer to execute a process . The computer program according to any one of claims 1 to 8, for causing the computer to execute a process of displaying a representative image suitable for diagnosing the predetermined disease from among the plurality of appropriate images . a plurality of individual learning models that extract and output feature values of an input image when a single image is input, and extract feature values of each of the plurality of images by inputting each of the selected images into the plurality of individual learning models;
A computer program according to any one of claims 1 to 9, wherein when the features of the plurality of images are input, the features output from the plurality of individual learning models are input into an integrated learning model that outputs an image suitability indicating the degree to which the plurality of images are suitable for diagnosing the specified disease, thereby determining whether the plurality of images are the plurality of appropriate images suitable for diagnosing the specified disease. estimating a scanning region of the subject based on a signal output from a positioning sensor provided on the scanning probe;
The computer program according to any one of claims 1 to 10, for causing the computer to execute a process of outputting movement instruction information for instructing movement of the scanning probe so that a predetermined number or more of the plurality of images are generated at each of a plurality of scanning regions of the subject. estimating the orientation of the scanning probe based on a signal output from an orientation sensor provided on the scanning probe;
storing posture information indicating a posture of the scanning probe when the plurality of appropriate images suitable for diagnosing the predetermined disease are generated;
The computer program according to any one of claims 1 to 11, further comprising: outputting posture change instruction information for instructing a change in posture of the scanning probe based on the stored posture information when the generation rate of the plurality of appropriate images suitable for diagnosing the predetermined disease decreases. the scanning probe is an ultrasound probe that scans an organ of the subject by periodically switching between ultrasound waves of a first frequency and ultrasound waves of a second frequency;
The process of generating the plurality of images includes:
The computer program according to any one of claims 1 to 12, which generates a series of images based on signals obtained from the scanning probe that scans the organ of the subject while switching between ultrasonic frequencies. generating a series of images based on signals obtained from a scanning probe scanning an organ of the subject;
selecting the plurality of images generated at the same respiratory timing based on a respiratory cycle signal output from a respiratory cycle sensor that detects the respiratory cycle of the subject;
determining whether the selected images at the same respiratory timing are suitable for diagnosing a predetermined disease;
storing the plurality of images at the same breathing timing determined to be suitable for diagnosing the predetermined disease as a plurality of suitable images;
and outputting a collection of the plurality of appropriate images suitable for diagnosing the predetermined disease. a generator for generating a series of images based on signals obtained from a scanning probe that scans an organ of a subject;
a determination unit that selects, from the plurality of images generated by the generation unit, the plurality of images generated at the same respiratory timing based on a respiratory cycle signal output from a respiratory cycle sensor that detects the respiratory cycle of the subject, and determines whether the selected plurality of images at the same respiratory timing are images suitable for diagnosing a predetermined disease;
a storage unit that stores the plurality of images at the same breathing timing that are determined by the determination unit to be suitable for diagnosing the predetermined disease as a plurality of appropriate images;
and an output unit that outputs a collection of the plurality of appropriate images suitable for diagnosing the predetermined disease.