JP6915969B2 - Medical image processing equipment, medical image processing method and ultrasonic diagnostic equipment - Google Patents

Description

Embodiments of the present invention relate to an image processing apparatus, an image processing method, and an ultrasonic diagnostic apparatus.

Conventionally, medical devices (medical diagnostic imaging devices) such as X-ray CT (Computed Tomography) and ultrasonic scanning devices have been widely used.

In the actual application, it is necessary to support the interpretation by the doctor by collating the images acquired by these medical devices with the images acquired at different times such as before and after the treatment. For example, after treating a specific site such as a site where a blood vessel or a tumor is present, the morphology of the specific site may change. In this case, when collating images at different times, it becomes difficult to match a plurality of images of a specific part. So-called "matching" refers to matching the geographic coordinates of an image so that the same area between different images is matched.

In particular, when the change in the environment in which the images are acquired is large, the time interval is long, or the images acquired by different devices are collated, as shown in FIG. 10, even if the same subject is used, between the images before and after the treatment. There is a problem that it is difficult to identify a specific part on the image before treatment and a corresponding part, and it is difficult to match, especially in the image after treatment, if pixel matching by gray scale is used. ..

In the conventional method, in order to match those images with a large error, the operator usually manually inputs a part where a specific site exists or a common area of interest into a collated image such as a post-treatment image. Matching is performed based on such inputs.

For example, Patent Document 1 discloses a method of converting different coordinate systems into the same coordinate system by using a linking facility. However, this method still cannot solve the problem of matching failure due to the large position deviation in the image.

Further, Patent Document 2 discloses a method of matching images by acquiring images using preset reference feature points for images acquired by a plurality of medical devices MR, CT and the like. However, according to this method, a matching error that occurs when the image positions are different cannot be avoided.

Further, Patent Document 3 discloses a method of performing matching by selecting a point cloud on an ultrasonic image. However, this method still requires positioning in a preset point cloud, and it is not possible to specify the correctness of the data.

Therefore, in the conventional technique, it is necessary to set a reference region in advance, and there is a problem that it is difficult to accurately match an image having a large deformation (due to treatment or respiratory movement) or a small overlapping portion. Existing.

U.S. Pat. No. 8831708 U.S. Patent Application Publication No. 2015/0209015 U.S. Pat. No. 8731264

An object to be solved by the present invention is to provide an image processing apparatus, an image processing method, and an ultrasonic diagnostic apparatus capable of automatically matching a specific portion on an image with high accuracy.

The image processing device of the embodiment includes an image acquisition unit, a recognition unit, and a matching unit. The image acquisition unit acquires a first image and a second image including a specific part in the subject, which are imaged at different timings in chronological order. The recognition unit recognizes the specific site in each image acquired by the image acquisition unit, and recognizes a plurality of first specific site candidates in the first image and a plurality of second specific site candidates in the second image. The matching unit is based on the difference between each feature amount of the plurality of first specific site candidates and each feature amount of the plurality of second specific site candidates, and each first specific site candidate and each second specific site candidate. By matching with the site candidate, the correspondence relationship representing the same specific site in the first image and the second image is specified.

FIG. 1 is a block diagram showing a configuration of an image processing device according to the first embodiment. FIG. 2 is a block diagram showing a configuration of a matching unit according to the first embodiment. FIG. 3 is a flowchart showing the flow of the image matching process according to the first embodiment. FIG. 4 is a flowchart illustrating the difference calculation of a plurality of feature quantities according to the first embodiment. FIG. 5 is a schematic view showing a configuration in which an image processing device is used in a cooperative system using a plurality of medical devices. FIG. 6 is an explanatory diagram illustrating a specific example of specifying a specific portion in the present embodiment. FIG. 7 is a block diagram showing a configuration of an image processing device according to a second embodiment. FIG. 8 is a flowchart showing the flow of the image matching process according to the second embodiment. FIG. 9 is a diagram comparing the present embodiment with the conventional technique. FIG. 10 is a schematic diagram illustrating image variation at different times.

The image processing apparatus according to the technical proposal of the present embodiment includes an image acquisition means for acquiring a first image and a second image including a specific part in a subject, which are taken at different timings in time series, and the image. A recognition means that recognizes the specific part in each image acquired by the acquisition means and recognizes a plurality of first specific part candidates in the first image and a plurality of second specific part candidates in the second image, and the plurality of recognition means. Based on the difference in the feature amount between each of the first specific site candidates and each of the plurality of second specific site candidates, the correspondence between the first specific site candidate and the second specific site candidate representing the same specific site is specified. It is provided with a matching means to be used.

In addition, the image processing method according to another technical proposal of the present embodiment is an image acquisition step of acquiring a first image and a second image including a specific part in a subject, which are taken at different timings in time series. And the recognition step of recognizing the specific part in each image acquired in the image acquisition step and recognizing a plurality of first specific part candidates in the first image and a plurality of second specific part candidates in the second image. , The first specific site candidate and the second specific site candidate representing the same specific site based on the difference in the feature amount between each of the plurality of first specific site candidates and each of the plurality of second specific site candidates. Includes a matching step to identify the correspondence.

By a method of fully recognizing a specific part candidate in each image and selecting using a comprehensive index of a plurality of feature quantities, matching of a specific part on the image can be performed automatically and with high accuracy, and images can be matched with each other. Even if the error is large, it is possible to perform matching with high accuracy without specifying a reference point or a region of interest.

Because of the features and effects described above, the present embodiment also makes it possible to collate images from different image processing devices.

The present embodiment relates to an image processing apparatus that processes an image. This image processing device is a device connected to an image collecting device and having a CPU (Central Process Unit) such as an independent computer, and can be realized by executing software having each function of the image processing device. It can also be realized by a hardware method as a circuit capable of executing each function of the image processing device. Moreover, the image processing device according to the present embodiment may be preliminarily incorporated in the medical image collecting device as a part of the medical image collecting device such as a CT (Computed Tomography) device or an ultrasonic device.

Hereinafter, embodiments for carrying out this embodiment will be described with reference to the drawings. Further, in different embodiments, the same symbols are used for the same member, and the description of duplication will be appropriately omitted.

(First Embodiment)
FIG. 1 is a block diagram showing a configuration of an image processing device according to a first embodiment of the present embodiment. As shown in FIG. 1, the image processing device 100 includes an image acquisition unit 10, a recognition unit 20, and a matching unit 30.

The image acquisition unit 10 acquires a plurality of images including a specific part in the subject, which are captured at different timings in chronological order. For example, the image acquisition unit 10 acquires and stores a plurality of ultrasonic images collected at different times by the ultrasonic device from the ultrasonic device connected to the image processing device 100. The image acquisition unit 10 may be a circuit or a software module capable of realizing the above-mentioned functions. It is preferable that each image acquired by the image acquisition unit 10 is a three-dimensional medical image, but a two-dimensional medical image may be included.

The recognition unit 20 recognizes a specific part in each image acquired by the image acquisition unit 10, and uses a plurality of recognized specific parts as specific part candidates in each image. The recognition unit 20 may be a circuit or a software module capable of realizing the above-mentioned functions.

The so-called specific site refers to a region having similar characteristics on the image and represents a specific site on the medical image of the subject, for example, a site where a tumor is present, a blood vessel site, a specific tissue assembly, or medical intervention. The part of the element and the like can be mentioned.

For recognition of a specific part, a conventional method of extracting an image feature region may be used. For example, it can be extracted by the MSER (Maximally Stable Extremal Regions) method. Specifically, for example, the pixels constituting the image are analyzed using any one of the grayscale change amount as an index, the maximum area of the region, the maximum change rate, the minimum conversion amount of the stable region, and the like. The area that matches the selection index is extracted as a specific part.

In the present embodiment, since it is not necessary for the operator to input or confirm the specific part, a plurality of specific parts are usually recognized by fully recognizing the image. Those specific parts that match the recognition index are stored as specific part candidates. For example, the recognition unit 20 recognizes a plurality of first specific site candidates in the first image and a plurality of second specific site candidates in the second image, respectively.

The matching unit 30 matches between a plurality of images captured at different timings based on the difference between the feature amounts of the plurality of first specific site candidates and the feature amounts of the plurality of second specific site candidates. To specify the correspondence relationship representing the same specific part between a plurality of images. For example, when matching two images before and after treatment among a plurality of images, the recognition unit 20 recognizes the specific site candidates R1 to Rn on the first image, and two specific site candidates F1 to Fm. Assuming that it is recognized on the image of the eye (however, n and m are natural numbers), the matching unit 30 is either R of specific site candidates R1 to Rn or any of specific site candidates F1 to Fm. Or F is regarded as one specific part pair, and the difference between the feature amounts of R and F is calculated for each specific part pair (R, F) combined in this way, and the difference between the feature amounts is used for collation. Specifies a specific part pair representing the same specific part on two images. The image processing apparatus 100 can align the images based on the specific part pair representing the same specific part obtained by the matching unit 30, and the result is used for the subsequent superimposition or display processing. The matching unit 30 may be a circuit or a software module capable of realizing the above-mentioned functions. The specific site candidates R1 to Rn are examples of the first specific site candidate, and the specific site candidates F1 to Fm are examples of the second specific site candidate.

The selected feature amount is set in advance and is at least one value selected from, for example, the dispersion value of the pixels constituting the specific portion, the average brightness, the entropy, the Euclidean distance, the volume ratio, the modified Jaccard coefficient, and the like. In order to perform matching with high accuracy, it is preferable that the matching unit 30 calculates a plurality of feature quantities for each specific site candidate.

Hereinafter, a specific method for specifying a specific site pair will be described. For example, the matching unit 30 calculates for a certain feature amount using the difference of the feature amount between the specific part pairs as an index, and compares the indexes of each specific part pair, so that the index exceeds a predetermined threshold value. Alternatively, the highest specific site pair ordered by the index is defined as a specific site pair representing the same specific site.

Further, in the present embodiment, a method of forming a comprehensive index using a plurality of feature quantities and performing evaluation is proposed. The matching unit 30 calculates a comprehensive index for each specific site pair using the difference between a plurality of feature quantities that can be acquired on the image, and the specific site pair having the smallest comprehensive index represents the same specific site. Can be.

FIG. 2 is a block diagram showing a configuration of a matching unit 30 that executes a method of comprehensively evaluating by a plurality of feature quantities described above. As shown in FIG. 2, the matching unit 30 includes a feature amount difference calculation unit 31 and an evaluation unit 32.

The feature amount difference calculation unit 31 calculates the difference between the feature amount of the first specific part candidate and the feature amount of the second specific part candidate in the specific part pair for each of the plurality of feature amounts. For example, the difference between the plurality of feature amounts is obtained by obtaining the difference between the feature amounts of R and F in the specific site pair for each of the plurality of feature amounts.

The selected feature amount is preset, and is, for example, the dispersion value (SD), mean intensity (Mean Intensity), entropy (Entropy), Euclidean distance (Euclidean Distance), and volume ratio (Volume Ratio) of the pixels constituting a specific part. ), Modified Jaccard similarity (Modified Jaccard similarity), etc. As a method for calculating these features, a method in a conventional technique may be referred to, and for example, the following general formula may be mentioned as a conventional one.

As for the calculation for each feature amount, the method in the conventional technique may be referred to, so detailed description thereof will be omitted.

After the feature amount difference calculation unit 31 calculates the feature amount of the specific part R and the feature amount of the specific part F in each of the specific part pairs for each designated feature amount, the specific part R and the specific part F correspond to each other. Find the difference between features. For example, if the value of any feature amount of the specific part R is VR and the value of the feature amount of the specific part F is VF, the difference between the specific part R and the specific part F is (VR-VF).

The evaluation unit 32 calculates the comprehensive index of each specific part pair based on the difference of each feature amount calculated by the feature amount difference calculation unit 31, and selects the specific part pair having the smallest comprehensive index as the first image and the first image. 2 Specify as the same specific part in the image.

For example, the evaluation unit 32 can obtain the comprehensive index S (R, F) by the following formula (2).

However, N is the number of types of a plurality of feature amounts, VR is the value of any of the feature amounts of the specific part R, VF is the value of the same feature amount of the specific part F, and Wi is given to the _i- th feature amount. Represents the weight given.

Here, a method of calculating the comprehensive index using the square root is given as an example, but the method of obtaining S (R, F) is not limited to the above example, and other calculation methods may be used.

Regarding the method of determining the weight, the ratio of the sum of the differences of a plurality of features and the difference of each feature may be used as the weight. For example, when two feature quantities A and B are selected and the difference between the feature quantities A is 90 and the difference between the feature quantities B is 10, the weight (A) = 100/90 and the weight (B) = 100. It becomes / 10.

Further, a normalized weight coefficient may be given to each weight. For example, four feature quantities A, B, C, and D are selected, and the difference of the feature quantity A is 90, the difference of the feature quantity B is 10, the difference of the feature quantity C is 50, and the difference of the feature quantity D is 60. In the case of, these four values are normalized within the interval (0,1) by the following equation. That is, _Wi = (difference in the i-th feature amount-smallest difference) / (largest difference-smallest difference). In the above-mentioned example, the weight (A) = 1, the weight (B) = 0.00001, the weight (C) = 0.5, and the weight (D) = 0.625 are obtained.

Further, the weight value may be set by another method. For example, an input for a weight value can be directly received and determined from an operator or another device via an interface.

In the first embodiment, the image acquisition unit 10 corresponds to the "image acquisition means", the recognition unit 20 corresponds to the "recognition means", and the matching unit 30 corresponds to the "matching means". Hereinafter, the flow of the image matching process by the image processing device 100 will be described.

FIG. 3 is a flowchart showing the flow of the image matching process according to the first embodiment of the present embodiment. As shown in FIG. 3, at the start of the matching process, first, a plurality of image acquisition units 10 are taken from a medical device such as an ultrasonic device at different timings in chronological order, including a specific part in the subject. Acquire an image (step S301). For example, the image shown in FIG. 6 is acquired. 6 (A) and 6 (B) are explanatory views for explaining a specific example of specifying a specific site according to the present embodiment.

Next, in step S302, the recognition unit 20 recognizes a specific part in each image acquired by the image acquisition unit 10, and sets the recognized plurality of specific parts as specific part candidates in each image. For example, the recognition unit 20 recognizes the specific parts R1, R2, and R3 on the image in FIG. 6A, and recognizes the specific parts F1 and F2 on the image in FIG. 6B.

Then, the process proceeds to step S303. In step S303, the matching unit 30 calculates the difference in the feature amount of the specific part pair between the images to confirm the specific part pair representing the same specific part.

As described above, the specific calculation method is, for example, matching of specific parts using a plurality of feature quantities. FIG. 4 is a flowchart illustrating the difference calculation of a plurality of feature quantities according to the first embodiment of the present embodiment. In step S401, the feature amount difference calculation unit 31 of the matching unit 30 obtains the difference between a plurality of feature amounts in each of the specific part pairs for each specific part pair.

Then, the process proceeds to step S402. In step S402, the evaluation unit 32 of the matching unit 30 calculates the comprehensive index of each specific part pair based on the difference of the plurality of feature amounts calculated by the feature amount difference calculation unit 31, and the specific part having the smallest comprehensive index. The pair is a specific part pair representing the same specific part.

In the example shown in FIG. 6, the comprehensive index S (R, F) is calculated for all combinations of the specific parts R1, R2, R3 and the specific parts F1, F2, respectively, and the table shown in Table 1 is obtained. rice field.

In Table 1, the percentage represents the overall index S (R, F) of the corresponding specific site pair (R, F).

As shown in Table 1, the total index of the specific site pair consisting of F2 and R3 is 86%, which is much higher than that of other combinations. Therefore, it is confirmed that F2 and R3 correspond to each other, and that (R3, F2) represents the same specific site between (A) in FIG. 6 and (B) in FIG. Up to this point, the matching process is completed, and the image processing apparatus 100 performs alignment or superimposition display processing between the two images for interpretation by the doctor based on the specific site pair, and then presents the image to the doctor. be able to.

Although an example is shown in FIG. 6, the specification for the correspondence of specific parts is not limited to the above example. For example, the obtained comprehensive index is compared with a predetermined threshold value, and the specific part pair exceeding the threshold value is compared. May be confirmed as a specific site pair representing the same specific site.

According to the matching process described above, even if the morphology or position of the images acquired at different times differs greatly, a plurality of specific parts are extracted as specific part candidates from the image and comprehensively using a plurality of feature quantities. By evaluating, it is possible to automatically find a specific relationship of a specific part, and it is possible to perform image matching with high accuracy.

(Modified example of the first embodiment)
In the first embodiment, matching is performed on the same type of medical image, but this embodiment can also be applied to matching on different types of medical images from different medical devices.

FIG. 5 is a schematic view showing a configuration in which an image processing device is used in a cooperative system using a plurality of medical devices.

In the system shown in FIG. 5, a device in which the image processing device 100 is incorporated is connected to the CT device 200 and the ultrasonic device 300, respectively, so that a CT image is acquired from the CT device 200 and the ultrasonic device is used. An ultrasonic image can be acquired from 300, and a specific portion between the X-ray image and the ultrasonic image can be matched by the image processing apparatus 100.

Due to the large difference between medical images collected by different types of medical devices, it is common for conventional methods to favor different regions of interest as specific sites, affecting the matching effect. ..

By automatically recognizing a plurality of specific parts using the image processing device 100 according to the present embodiment and matching a specific part pair using a comprehensive index of a plurality of feature quantities, it is possible to match even between images having a large error. Yes, the accuracy of matching can be improved.

For example, in diagnosing a certain disease, it is conceivable to compare a CT image captured in advance with an ultrasonic image captured in real time. In such a case, the doctor determines the position of the ultrasonic probe by using the diseased part and the peripheral part drawn on the CT image as a mark, and captures the ultrasonic image. Here, although the same disease site and peripheral site as the CT image are depicted in the captured ultrasonic image, the positions and orientations of the two images are often different. Therefore, the image processing apparatus 100 according to the present embodiment recognizes the diseased part and the peripheral part as specific parts from both images, and identifies a plausible specific part pair from the recognized candidate group of the specific part. Match images. Therefore, the image processing device 100 can align the CT image and the ultrasonic image with reference to the diseased part and the peripheral part thereof, and can support the comparative diagnosis of both images by a doctor.

Further, for example, in the conventional matching method, in order to perform matching using the entire image, there is a case where the trap is trapped in the local minimum. However, the image processing apparatus 100 according to the present embodiment recognizes a candidate for a specific part from each of a plurality of images in which the same specific part is captured, and performs matching for each of the recognized candidates for the specific part locally. The possibility of being trapped in the minimum can be reduced.

In the above example, the case where the CT image and the ultrasonic image are matched as a plurality of images has been described, but the embodiment is not limited to this. For example, the plurality of images may include an image captured by another medical image diagnostic device, such as an MRI image captured by a magnetic resonance imaging (MRI) device.

(Second embodiment)
The second embodiment is based on the first embodiment. The difference between the second embodiment and the first embodiment is that finer matching is added to the matching process and the coordinate system is transformed with respect to the image.

Hereinafter, the differences between the second embodiment and the first embodiment will be mainly described, and the description of duplication will be appropriately omitted.

FIG. 7 is a block diagram showing a configuration of the image processing device 100A according to the second embodiment. As shown in FIG. 7, the image processing device 100A includes an image acquisition unit 10, a recognition unit 20, a coordinate conversion unit 40, a matching unit 30, and a matching correction unit 50.

The image acquisition unit 10 acquires a plurality of images including a specific part in the subject, which are captured at different timings in chronological order. The image acquisition unit 10 may be a circuit or a software module capable of realizing the above-mentioned functions.

The recognition unit 20 recognizes a specific part in each image acquired by the image acquisition unit 10, and uses a plurality of recognized specific parts as specific part candidates in each image. The recognition unit 20 may be a circuit or a software module capable of realizing the above-mentioned functions.

The coordinate conversion unit 40 performs coordinate conversion on the image to match the coordinate systems of different images. The coordinate conversion unit 40 may be a circuit or a software module capable of realizing the above-mentioned functions.

Regarding the coordinate conversion between images, for example, when matching a pair of images composed of a reference image and a collated image, the coordinate conversion unit 40 performs the coordinate conversion by moving or rotating the collated image to perform the collation. It can be converted so that the coordinate system of the image matches the coordinate system of the reference image. Of course, it is also possible to convert two images that require matching to the same coordinate system.

As a specific method of coordinate transformation, a conventional method may be used. For example, when matching ultrasonic images, the coordinate transformation is performed by performing matrix transformation using the magnetic field coordinate information of the ultrasonic image and imparting a displacement amount to the pixel matrix.

The matching unit 30 matches images whose coordinate systems are matched by the coordinate conversion unit 40 based on the specific portion recognized by the recognition unit 20. The matching unit 30 identifies a specific site pair representing the same specific site based on the difference between a plurality of feature amounts for each specific site pair consisting of specific site candidates. The matching unit 30 may be a circuit or a software module capable of realizing the above-mentioned functions.

Regarding the specific configuration of the matching unit 30 and the calculation method for the feature amount, the calculation method of the comprehensive index described in the first embodiment may be referred to.

Moreover, in the second embodiment, by calculating the comprehensive index using a plurality of feature parameters, it is possible to deal with the situation where the magnetic field coordinate information used by the coordinate conversion unit 40 is inaccurate. This is because a plurality of feature parameters are used in the calculation of the comprehensive index, so that the influence of certain erroneous parameters can be reduced.

The matching correction unit 50 rematches the images using the grayscale parameters based on the correspondence of the specific part pair specified by the matching unit 30. The matching correction unit 50 may be a circuit or a software module capable of realizing the above-mentioned functions.

That is, the matching correction unit 50 uses the conventional grayscale matching method as designating the correspondence between two images of the same specific part specified by the matching unit 30 with respect to the feature point or feature area in the conventional method. It is used to perform more fine matching with the image.

Since the correspondence relationship of the specific part on which the matching correction unit 50 relies is more accurate than the correspondence relationship received by the conventional method such as manual input by the operator, the grayscale matching method uses the image processing device. The matching result of 100A can be further optimized.

Conventional grayscale matching methods include a normalized cross correlation (NCC) method and a nelder-mead method. Here, detailed description will be omitted.

The matching correction unit 50 may be omitted because it further optimizes the matching result of the image processing device 100A.

In the second embodiment, the image acquisition unit 10 corresponds to the "image acquisition means", the recognition unit 20 corresponds to the "recognition means", the matching unit 30 corresponds to the "matching means", and the coordinate conversion unit 40 corresponds to the coordinate conversion unit 40. The matching correction unit 50 corresponds to the "coordinate conversion means" and corresponds to the "matching correction means". Hereinafter, the flow of the image matching process by the image processing device 100A will be described.

FIG. 8 is a flowchart showing the flow of the image matching process according to the second embodiment.

As shown in FIG. 8, at the start of the matching process, first, the image acquisition unit 10 is imaged from a medical device such as an ultrasonic device at different timings in chronological order, and a plurality of images including a specific part in the subject are included. The image is acquired (step S801).

Next, in step S802, the recognition unit 20 recognizes a specific part in each image acquired by the image acquisition unit 10, and sets the recognized plurality of specific parts as specific part candidates in each image.

Then, the process proceeds to step S803. In step S803, the coordinate conversion unit 40 performs coordinate conversion on the collated image, imparts a displacement amount to the pixels, and moves or rotates them to match the coordinate systems of different images.

After that, in step S804, the matching unit 30 calculates the difference in the feature amount of the specific part pair between the images to confirm the specific part pair representing the same specific part. FIG. 4 may be referred to for a specific flow.

Finally, in step S805, as a part where the matching correction unit 50 performs fine matching, the correspondence relationship of the specific part pair specified by the matching unit 30 is input, the grayscale parameter is used, and the image is matched again based on the grayscale. do.

FIG. 9 is a diagram comparing the present embodiment with the conventional technique. Of these, the reference image before treatment is shown on the left side of FIG. 9A, and the center of the screen is the lesion area. Moreover, the collation image after the treatment is shown on the right side of (A) of FIG.

As shown in FIG. 9A, the difference between the reference image and the collation image is large as a whole due to the effect of treatment and the movement of the subject.

FIG. 9B shows the result of inputting the region of interest by the operator and matching using the conventional grayscale matching method. In the superimposed display after the alignment, the site in the black frame and the lesion area in the reference image are associated with each other, so that there is a large error.

On the other hand, FIG. 9C shows the result of matching by the image processing apparatus according to the present embodiment. By the coordinate system transformation by movement or rotation, recognition of a specific part, and comprehensive evaluation by a plurality of feature quantities, the part surrounded by the black frame in FIG. 9C is associated with the lesion area in the reference image. Therefore, the matching accuracy is higher than that of the conventional technology.

(Modified example of the second embodiment)
In the second embodiment, after the recognition unit 20 recognizes the specific part, the coordinate system conversion is performed by the coordinate conversion unit 40, but the present embodiment is not limited to this, and the recognition unit 20 recognizes the specific part. Before that, the coordinate system conversion by the coordinate conversion unit 40 may be performed.

The image processing apparatus of this embodiment may be incorporated into a medical apparatus as a circuit capable of realizing the functions described in each embodiment, or may be incorporated into a medical apparatus as a computer-executable program such as a magnetic disk (floppy disk (registered trademark)). : Floppy), hard disk, etc.), compact disk (CD-ROM, DVD, etc.), optical disk (MO), semiconductor memory, etc. may be stored and distributed.

In addition, the OS (operating system), database management software, network software, and other MW (middle work) that are executed on the computer based on instructions from a program installed on the computer from the storage medium also have the above-described embodiments. It is possible to execute a part of each process to realize it.

Although some embodiments of the present invention have been described, these embodiments are presented as examples and are not intended to limit the scope of the invention. These novel embodiments can be implemented in various other embodiments, and various omissions, replacements, and changes can be made without departing from the gist of the invention. These embodiments and modifications thereof are included in the scope and gist of the invention, and are also included in the scope of the invention described in the claims and the equivalent scope thereof.

100 Image processing device 10 Image acquisition unit 20 Recognition unit 30 Matching unit

Claims (15)

An image acquisition unit that acquires a first medical image and a second medical image including a specific part in a subject, which are imaged by a medical image diagnostic device at different timings in chronological order.
A recognition unit that recognizes the specific site in each medical image acquired by the image acquisition unit and recognizes a plurality of first specific site candidates in the first medical image and a plurality of second specific site candidates in the second medical image. When,
Based on the difference between the feature amounts of the plurality of first specific site candidates and the feature amounts of the plurality of second specific site candidates, the first specific site candidate and the second specific site candidate It is provided with a matching unit that performs matching and specifies a correspondence relationship representing the same specific part in the first medical image and the second medical image .
The matching unit
For each of the specific site pairs that are a pair of the first specific site candidate and the second specific site candidate, the feature amount of the first specific site candidate and the feature amount of the second specific site candidate in the specific site pair The feature amount difference calculation unit that calculates the difference between multiple features
Based on the difference of each feature amount calculated by the feature amount difference calculation unit, the comprehensive index of each of the specific site pairs is calculated, and the specific site pair having the smallest comprehensive index is the first medical image and the said. With the evaluation unit that identifies as the same specific part in the second medical image
With
The evaluation unit acquires the comprehensive index by assigning a weighting coefficient inversely proportional to the difference of each feature amount to the difference of a plurality of feature amounts calculated by the feature amount difference calculation unit.
Medical image processing apparatus. The evaluation unit calculates the comprehensive index by performing normalization processing on the differences of a plurality of feature quantities calculated by the feature quantity difference calculation unit.
The medical image processing apparatus according to claim 1. The medical image processing apparatus according to claim 1, wherein the feature amount is at least one of a dispersion value, an average brightness, entropy, an Euclidean distance, a volume ratio, and a modified Jaccard coefficient. A coordinate conversion unit that performs coordinate conversion to match the coordinate systems of the first medical image and the second medical image acquired by the image acquisition unit by the coordinate conversion of the medical image is further provided.
The matching unit according to claim 1, wherein the matching unit obtains the difference in the feature amount between each of the plurality of first specific site candidates and each of the plurality of second specific site candidates based on the coordinate system after the coordinate conversion. Medical image processing equipment. Correspondence between the first specific site candidate and the second specific site candidate using the grayscale parameters of the first specific site candidate and the second specific site candidate corresponding to the same specific site specified by the matching unit. Is further provided with a matching correction unit that matches again,
The medical image processing apparatus according to claim 1. The image acquisition unit acquires an ultrasonic image in which the specific site is captured as a first medical image, and a CT image in which the specific site is captured at a timing different from that of the first medical image is used for the second medical. Get as an image,
The medical image processing apparatus according to claim 1. The specific site is a region having similar characteristics in a medical image.
The medical image processing apparatus according to claim 1. An image acquisition step of acquiring a first medical image and a second medical image including a specific part in a subject, which are imaged by a medical image diagnostic device at different timings in chronological order.
A recognition step of recognizing the specific site in each medical image acquired in the image acquisition step and recognizing a plurality of first specific site candidates in the first medical image and a plurality of second specific site candidates in the second medical image. When,
Based on the difference between the feature amounts of the plurality of first specific site candidates and the feature amounts of the plurality of second specific site candidates, the first specific site candidate and the second specific site candidate performing matching, see contains the matching identifying a correspondence relation representing the same specific sites in the first medical image and the second medical image,
The matching step
For each of the specific site pairs that are a pair of the first specific site candidate and the second specific site candidate, the feature amount of the first specific site candidate and the feature amount of the second specific site candidate in the specific site pair Feature difference calculation step that calculates the difference of each feature for each of multiple features,
Based on the difference of each feature amount calculated by the feature amount difference calculation step, the comprehensive index of each of the specific site pairs is calculated, and the specific site pair having the smallest comprehensive index is the first medical image and the said. With the evaluation step to identify as the same specific part in the second medical image
Including
In the evaluation step, the comprehensive index is acquired by assigning a weighting coefficient inversely proportional to the difference of each feature amount to the difference of the plurality of feature amounts calculated by the feature amount difference calculation step.
Image processing method for a medical. The medical image processing method according to claim 8 , wherein the evaluation step calculates the comprehensive index by performing normalization processing on the difference of a plurality of feature amounts calculated by the feature amount difference calculation step. The medical image processing method according to claim 8, wherein the feature amount is at least one of a dispersion value, an average brightness, entropy, an Euclidean distance, a volume ratio, and a modified Jaccard coefficient. The coordinate conversion of the medical image further includes a coordinate conversion step of performing a coordinate conversion to match the coordinate systems of the first medical image and the second medical image acquired by the image acquisition step.
In the matching step, on the basis of the coordinate system after the coordinate transformation, according to claim 8 for obtaining the feature amount of the difference between each of the respective said plurality of second specific site candidate of the plurality of first specific site candidate Medical image processing method. Based on the correspondence between the first specific site candidate and the second specific site candidate corresponding to the same specific site specified by the matching step, the first specific site candidate and the second specific site candidate are used using grayscale parameters. Further including a matching correction step of rematching the correspondence of specific site candidates,
The medical image processing method according to claim 8. In the image acquisition step, an ultrasonic image in which the specific site is captured is acquired as a first medical image, and a CT image in which the specific site is captured at a timing different from that of the first medical image is used for the second medical. Get as an image,
The medical image processing method according to claim 8. The specific site is a region having similar characteristics in a medical image.
The medical image processing method according to claim 8. An image acquisition unit that acquires a first medical image and a second medical image including a specific part in a subject, which are imaged by a medical image diagnostic device at different timings in chronological order.
A recognition unit that recognizes the specific site in each medical image acquired by the image acquisition unit and recognizes a plurality of first specific site candidates in the first medical image and a plurality of second specific site candidates in the second medical image. When,
Based on the difference between the feature amounts of the plurality of first specific site candidates and the feature amounts of the plurality of second specific site candidates, the first specific site candidate and the second specific site candidate It is provided with a matching unit that performs matching and specifies a correspondence relationship representing the same specific part in the first medical image and the second medical image .
The matching unit
For each of the specific site pairs that are a pair of the first specific site candidate and the second specific site candidate, the feature amount of the first specific site candidate and the feature amount of the second specific site candidate in the specific site pair The feature amount difference calculation unit that calculates the difference between multiple features
Based on the difference of each feature amount calculated by the feature amount difference calculation unit, the comprehensive index of each of the specific site pairs is calculated, and the specific site pair having the smallest comprehensive index is the first medical image and the said. With the evaluation unit that identifies as the same specific part in the second medical image
With
The evaluation unit acquires the comprehensive index by assigning a weighting coefficient inversely proportional to the difference of each feature amount to the difference of a plurality of feature amounts calculated by the feature amount difference calculation unit.
The ultrasonic diagnostic apparatus.

Images