JP6313741B2 - Image processing apparatus and method of operating image processing apparatus - Google Patents

Description

  The present invention relates to a technique for processing and displaying three-dimensional medical image data.

  In recent years, medical image data is mainly three-dimensional volume data. When a doctor performs image diagnosis, tomographic images are continuously switched and displayed (scrolled) to check whether there is an abnormal shadow. When a doctor finds an abnormal shadow by image diagnosis, the features of the abnormal shadow are grasped from detailed observation while repeatedly scrolling and displaying display parameters in the vicinity of a tomographic image including the abnormal shadow. Then, the features of the abnormal shadow are input as image findings, and a tomographic image representative of the abnormal shadow (for example, the size of the abnormal shadow is maximized) is selected and pasted on the report.

  In image diagnosis, detailed diagnosis can be performed by three-dimensional image data. However, when displaying as a two-dimensional tomographic image or when combining it into a report, a tomographic image must be selected from the three-dimensional image data, increasing the burden on the doctor. In this series of operations, inventions for reducing the burden on doctors have been proposed so far. Patent Document 1 proposes a method of automatically adjusting display parameters based on image feature amounts in a region of interest. Further, as a general method, a method of obtaining a centroid of a region of interest in three-dimensional image data and displaying a tomographic image of a cross section passing through the centroid can be considered.

JP2007-143982

  However, the conventional techniques have the following problems. Patent Document 1 is an invention in which display parameters are adjusted in accordance with the situation at the time of medical examination or precision examination, and a tomographic image cannot be selected from three-dimensional image data. Further, even if a method for selecting a tomographic image that passes through the center of gravity of the region of interest in the three-dimensional image data is used as means for selecting a tomographic image to be displayed, the tomographic image in which the tomographic image does not necessarily represent an abnormal shadow is used. Don't be. As a result, the doctor must eventually select a tomogram, and the burden on the doctor cannot be reduced.

The present invention has been made in view of the above-described problems, and selects and displays a tomographic image representing the characteristics of an acquired finding from three-dimensional medical image data. That is, the present invention is a characteristic quantity is a value that indicates the observation acquisition means for acquiring information findings are words representing the characteristics of the abnormal shadow in the three-dimensional medical image, the characteristics of the saw where including a plurality of tomographic images A feature amount acquiring means for acquiring the feature amount previously associated with the finding information in the tomographic image, and specifying at least one tomographic image from the plurality of tomographic images based on the feature amount An image processing apparatus is provided.

  According to the present invention, it is possible to automatically select and display a tomographic image representing a feature of an image finding to be noted. As a result, the burden on the doctor can be reduced.

1 is a diagram illustrating an example of the configuration of an image processing apparatus according to a first embodiment. 1 is a diagram showing a basic configuration of a computer that implements each unit of an image processing apparatus by software. The figure which shows the outline | summary of a process of the image processing apparatus in 1st Example. 3 is a flowchart showing a processing procedure of the first embodiment of the image processing apparatus. 10 is a flowchart illustrating a processing procedure of the second embodiment of the image processing apparatus. 10 is a flowchart showing a processing procedure of a third embodiment of the image processing apparatus.

  Preferred embodiments of an image processing apparatus and its control method according to the present invention will be described below with reference to the accompanying drawings. However, the scope of the invention is not limited to the illustrated examples.

<First embodiment>
The configuration of the image processing apparatus 1 according to the present embodiment will be described with reference to FIG. In the image processing apparatus 1, the three-dimensional medical image acquisition unit 1001 acquires an image from the database 2. Next, the notable finding acquisition unit 1002 acquires notable image findings as a result of image interpretation by the user. In this embodiment, the image finding acquired by the notable finding acquisition unit 1002 is set as the notable finding. Next, the feature amount type identifying unit 1003 identifies the image feature amount type to be calculated based on the acquired attention finding, and the feature amount calculating unit 1004 calculates the identified image feature amount. The tomographic image specifying unit 1005 calculates a tomographic image selection index in each tomographic image, and determines a tomographic image based on the tomographic image selection index. Finally, the display control unit 1006 displays the specified tomographic image.

  FIG. 2 is a diagram illustrating a configuration example when the image processing apparatus according to the first embodiment is realized by a computer. The image processing apparatus 1 includes a control unit 10, a monitor 104, a mouse 105, and a keyboard 106. The control unit 10 includes a central processing unit (CPU) 100, a main memory 101, a magnetic disk 102, and a display memory 103. Then, when the CPU 100 executes the program stored in the main memory 101, various controls such as communication with the database 2 and overall control of the image processing apparatus 1 are executed. The CPU 100 mainly controls the operation of each component of the image processing apparatus 1. The main memory 101 stores a control program executed by the CPU 100 and provides a work area when the CPU 100 executes the program. The magnetic disk 102 stores an operating system (OS), device drives for peripheral devices, various application software including a program for performing diagnosis support processing, which will be described later, and the like. The display memory 103 temporarily stores display data according to the display control unit 1006. The monitor 104 is, for example, a CRT monitor or a liquid crystal monitor, and displays an image based on data from the display memory 103. The mouse 105 and the keyboard 106 are used for a pointing input and a character input by a user (doctor). The above components are connected to each other via a shared bus 107 so that they can communicate with each other.

  In this embodiment, the image processing apparatus 1 can read medical image data from the database 2 via a LAN (Local Area Network) 3. Here, an existing PACS (Picture Archiving and Communicating System) can be used as the database 2. Note that the types of 3D medical image data include an X-ray CT image, an MRI image, a PET image, and a SPECT image.

  The device configuration described above can be configured using a general computer and its peripheral devices. Further, the control procedure of the image processing apparatus according to the present invention, which will be described later with reference to FIG. 4, can be realized as a computer program executed on a computer.

  FIG. 3 shows an outline of the image processing of this embodiment. In FIG. 3, MI3 is a three-dimensional medical image to be diagnosed and is input from a database. In this embodiment, a chest X-ray CT image is taken as an example, and a processing apparatus used when performing an image diagnosis of the presence or absence of abnormal shadows and features thereof will be described. IF3 is an image finding input area, and an image finding can be input from a pull-down menu. When the image finding is input, the type of the image feature amount previously associated with the image finding is identified. Then, the image feature amount is calculated for each tomographic image, and a tomographic image selection index is further calculated therefrom. Then, according to the tomographic image selection index, a tomographic image representing the characteristics of the input image findings is selected. In FIG. 3, “spinous process” is input as an image finding, and a tomographic image selection index is calculated from an image feature amount previously associated with “spinous process”. Then, the tomographic image T3 is selected from the calculation result.

  FIG. 4 is a flowchart of this embodiment. A specific processing procedure executed by the image processing apparatus 1 will be described with reference to this flowchart.

<Step S410>
In step S <b> 410, the three-dimensional medical image acquisition unit 1001 acquires a three-dimensional medical image from the database 2. In the present embodiment, the acquired image may be used as it is in the subsequent processing, or the user may specify a region of interest (ROI) and limit the region of the image. When the region is limited, for example, a region may be designated from a three-dimensional medical image via a GUI (not shown) and replaced with a three-dimensional medical image handled in subsequent processing.

<Step S420>
In step S420, the notable finding acquisition unit 1002 acquires, as a notable finding, an image finding that the user has input via a GUI (not illustrated) with respect to the three-dimensional medical image. In the present embodiment, the user performs image diagnosis on the acquired three-dimensional medical image, and inputs an image finding using an image finding input form as shown in IF3 of FIG. Here, the image findings that can be input are, for example, “spheroid”, “spinous process”, “unclear boundary”, and the like. The user inputs at least one of these. Note that the method for selecting image findings is not limited to this. For example, when past diagnosis results of the same patient are stored in the database, an image finding input based on the diagnosis result may be used as a notable finding.

<Step S430>
In step S430, the feature amount type identifying unit 1003 identifies the type of the image feature amount to be calculated based on the notable finding obtained by the notable finding acquiring unit 1002. In the present embodiment, the type of image feature amount is associated with each image finding in advance, and the type of image feature amount automatically calculated from the selected attention finding is determined. An example of associating image findings with image feature quantity types is as follows.
“Spherical shape”: Circularity C, aspect ratio A
“Spinous process”: contour length L, filling rate F
“Boundary ambiguity”: Sum of boundary edge components E

<Step S440>
In step S440, the feature amount calculation unit 1004 calculates the image feature amount whose type has been identified in step S430 for each tomographic image of the three-dimensional medical image based on the attention findings obtained by the attention finding acquisition unit 1002. To do.
Here, the circularity C, the aspect ratio A, the contour line length L, and the filling rate F are calculated from a binary image generated from a three-dimensional medical image. In the present embodiment, regarding the binary image generation method, a threshold value is determined by a discriminant analysis method from the histogram of a three-dimensional medical image, and binarization processing is performed. Using the generated binary image, the circularity C, the aspect ratio A, and the filling rate F are calculated by the following equations.

  Here, Area is the area of each region in the binary image. Fereth is the horizontal ferret diameter, and Feretv is the vertical ferret diameter, both of which are calculated from the circumscribed rectangle of each region in the binary image. In the case where there are a plurality of regions in the binary image, in this embodiment, an image feature amount is calculated with the region having the largest area as the target region. The contour line length L is the length of the contour line of the target region. The sum E of edge components is the sum of edge components on the contour line of the target region. The edge component is a component that can be obtained by applying a Sobel filter to a three-dimensional medical image.

  With these calculation methods, the image feature amount associated with the attention finding is calculated in each tomographic image. In this embodiment, an image feature amount is calculated for a tomographic image group (for example, a tomographic image group generated at intervals of 1 mm) in each of the axial, coronal, and sagittal cutting directions. Note that the method of selecting the tomographic image for calculating the image feature amount is not limited to this. For example, the cutting direction of the tomographic image is not limited to axial, coronal, sagittal, and a tomographic image group may be created with a free cutting direction. For example, an image feature is obtained from a group of tomographic images in a cutting direction obtained by setting an x-axis, a y-axis, and a z-axis passing through the center of the image and rotating one or two axes by 180 ° by 10 °. The amount may be calculated.

<Step S450>
In step S450, the tomographic image specifying unit 1005 determines a tomographic image representing the feature of the notable finding based on the image feature amount calculated by the feature amount calculating unit 1004. In the present embodiment, a value indicating that each tomographic image represents the “degree of attention finding” is calculated (digitized) from the image feature amount calculated in step S440, and this is referred to as a “tomographic image selection index” based on the attention finding. To do. As an example, an example of a mathematical formula for calculating a tomographic image selection index corresponding to the three image findings is shown below.

ここで，wは予め定めた重みである。本実施例ではw1〜w5 = 1.0としている。

Here, w is a predetermined weight. In this embodiment, w1 to w5 = 1.0.

  It should be noted that other selection criteria than “degree of attention finding” may be incorporated in the formula for calculating the index. For example, by adding a term representing the size of the region in the binary image, the proximity of the region of interest to the center of gravity, and the like to the above formula, a criterion other than the finding of interest may be considered.

  Using these equations, a tomographic image selection index corresponding to the notable finding is calculated for each tomographic image. Then, the tomographic image having the maximum tomographic image selection index is determined as a tomographic image to be displayed. When a plurality of findings are selected in step S420, in this embodiment, a tomographic image selection index is calculated for all the selected findings of interest, and a tomogram that displays a tomographic image having the largest total index value. Determine as an image. This is saved as data for the report.

<Step S460>
In step S460, the display control unit 1006 displays the tomographic image specified by the tomographic image specifying unit 1005.

  Thus, by selecting the tomographic image to be displayed based on the image feature amount associated with the image finding, the tomographic image representing the feature of the input image finding can be automatically selected.

<Modification 1>
In the above embodiment, when there are a plurality of observation findings in step S450, each tomographic image selection index is calculated, and the tomographic image is determined from the total value. However, a tomographic image that maximizes each tomographic image selection index may be selected and displayed in association with each finding of interest. In this case, the user may be allowed to select a tomographic image used for the report.

<Second embodiment>
In the present embodiment, an example will be described in which an image feature is calculated from a three-dimensional medical image and an attention finding is automatically selected without inputting a user instruction when inputting an image finding. Based on the image feature amount calculated from the acquired image, an image finding that will be included in the image can be identified, and this can be automatically selected as the finding of interest.

  Since the configuration of the apparatus is the same as that of the first embodiment, description thereof is omitted. However, it differs from the first embodiment in that the notable finding acquisition unit 1002 automatically selects the notable finding from the three-dimensional medical image. Further, when the image processing apparatus 1 operates based on a software command, the program stored in the ROM or HDD executes the processing of FIG. 5 and is a program for realizing the function for that. Is different.

  Details of step S520 and step S530 according to the second embodiment will be described with reference to FIG. Step S510 and the processing from step S540 to step S570 are the same as the processing from step S410 and step S430 to step S460 in the flowchart of FIG.

<Step S520>
In step S520, the notable finding acquisition unit 1002 calculates a feature amount for selecting the notable finding from the three-dimensional medical image acquired by the three-dimensional image acquiring unit 1001.

  In this embodiment, first, all types of tomographic image selection indices are calculated in each tomographic image using the equations (4) to (6) in step S450. Next, the tomographic image selection indices calculated from the respective tomographic images are totaled for the number of tomographic images, and are used as feature quantities for selecting the finding of interest. That is, in this embodiment, in each three-dimensional medical image, all types of prepared indices such as a spherical index, a lobed index, an irregular index, a spinous process index, and a boundary irregularity index are calculated. The total value for the number of images is calculated as the feature amount. Note that the method of calculating the feature amount for selecting the finding of interest is not limited to this. For example, the maximum value of the index calculated in each tomographic image may be used as the feature amount for selecting the finding of interest.

<Step S530>
In step S530, the notable finding acquisition unit 1002 identifies an image finding that will be included in the image based on the feature value for selecting the notable finding calculated in step S520, and selects this as the notable finding.

  In this embodiment, the image finding of the three-dimensional medical image is selected by performing threshold processing on the feature value for selecting the finding of interest calculated in step S520. A threshold value is set in advance for each feature amount, and an image finding corresponding to a feature amount equal to or greater than the threshold value is selected as a note finding. When there are a plurality of feature quantities that are equal to or greater than the threshold value, all of them are treated as attention findings and passed to the next step. However, when multiple image findings belonging to the same category (for example, image findings belonging to the “shape” category are “spheroid”, “leaf-like”, “irregular”), the feature amount is the largest among them. Pick one thing and make it a notable finding. Note that the method of selecting image findings is not limited to this. For example, an image finding may be selected from the feature amount by constructing a discriminator using Support Vector Machine, AdaBoost, or the like by learning using a combination of the feature amount and the image finding obtained for the past diagnosis result. .

  In this way, by selecting a notable finding from the feature amount, an image finding input operation by a doctor is automated. That is, all processes from the input of image findings to the display of tomographic images representing the characteristics of image findings are all automatic.

<Third embodiment>
In this example, after obtaining a notable finding using the method of the first example or the second example, a differential diagnosis is performed using an inference device, and a tomography that represents the feature of the notable finding contributing to the diagnosis Display an image. In this embodiment, it is assumed that the doctor inputs image findings other than the notable findings.

  Since the configuration of the apparatus is the same as that of the first embodiment, description thereof is omitted. However, it differs from the first embodiment in that the notable finding acquisition unit 1002 automatically selects a notable finding from a large number of image findings. Further, when the image processing apparatus 1 operates based on a software command, the program stored in the ROM or HDD executes the processing of FIG. 6 and is a program for realizing the function for that. Is different.

  Details of steps S630 and S640 according to the third embodiment will be described with reference to FIG. Note that the processing from steps S610 and S620 to steps S650 to S680 is the same as the processing from steps S410 and S420 and steps S430 to S460 in the flowchart of FIG. .

<Step S630>
In step S630, the notable finding acquisition unit 1002 inputs all acquired image findings to the inference unit. As inference means, existing inference methods such as Bayesian networks and neural networks can be used. In this embodiment, a Bayesian network is used as an inference means. A Bayesian network is an inference model using conditional probabilities. When applied to the present embodiment, an inference probability of each diagnosis name can be acquired using an image finding as an input. The diagnostic names handled in this example are “primary lung cancer”, “lung metastasis of cancer”, and “others”.

Also, in this example, if each diagnosis name is Di and each finding is Sj, the contribution probability of Sj is a prior probability P (Di) (probability of each diagnosis name when there is no input) and only Sj is input. It is calculated using the difference from the case probability P (Di | Sj). For example, the contribution C (Di | Sj) of the finding Sj to the diagnosis name Di is expressed by the following equation.

  The contribution in each diagnosis name is calculated for all image findings prepared as options. In this embodiment, the contribution of each image finding in the diagnosis name (estimated diagnosis name) with the highest probability in the inference result by the Bayesian network is sent to the next step.

<Step S640>
In step S640, the notable finding acquisition unit 1002 selects the notable finding according to the contribution calculated in step S630. In this embodiment, the attention finding is selected by performing threshold processing on the contribution calculated in step S630. A threshold value is set in advance for the degree of contribution, and an observation finding that has obtained a contribution degree equal to or greater than the threshold value is selected. When there are a plurality of observations whose contribution degree is equal to or greater than the threshold value, all of them are selected. If there is no notable finding whose contribution is equal to or greater than the threshold value, the notable finding with the largest contribution is selected. In addition to this selection method, an attention finding with the maximum contribution may be selected without using a threshold value.

  In this way, by selecting a notable finding based on the contribution to the diagnosis name obtained from the inference device, a tomographic image representing the characteristics of the image finding that supports the inference result can be displayed.

<Modification 2>
In the embodiment described above, in step S630, the contribution is calculated for the diagnosis name with the highest probability in the inference result by the Bayesian network, and is used as the target of the target finding selection. However, the attention finding may be selected based on the contribution in the diagnosis name input by the user (doctor). Specifically, when the diagnosis name input by the doctor as an impression is different from the diagnosis name inferred by the Bayesian network, the attention finding may be selected based on the degree of contribution to the diagnosis name selected by the doctor. According to this, an effect is obtained that a tomographic image representing an image finding that explains the basis of a doctor's diagnosis is selected.

<Other examples>
The present invention can also be realized by executing the following processing. That is, software (program) for realizing the functions of the above-described embodiments is supplied to a system or apparatus via a network or various storage media, and a computer (or CPU, MPU, etc.) of the system or apparatus reads the program. It is a process to be executed.

DESCRIPTION OF SYMBOLS 1 Image processing apparatus 2 Database 3 LAN (Local Area Network)
1001 Three-dimensional medical image acquisition unit 1002 Attention finding acquisition unit 1003 Feature quantity type identification unit 1004 Feature quantity calculation unit 1005 Tomographic image specifying unit 1006 Display control unit

Claims (17)

A finding acquisition means for acquiring findings information that is a wording that expresses the characteristics of abnormal shadows in a three-dimensional medical image including a plurality of tomographic images;
A characteristic quantity is a value indicating characteristics of look at, the feature amount associated beforehand with the information of the findings, the characteristic amount obtaining means for obtaining in the tomographic image,
Specifying means for specifying at least one tomographic image from the plurality of tomographic images based on the feature amount;
An image processing apparatus comprising:   The image processing apparatus according to claim 1, wherein the finding information is finding information selected by a user from predetermined finding candidates.   The image processing apparatus according to claim 1, wherein the finding information is finding information for presenting a feature indicated by a feature amount acquired in the tomographic image to a user. An evaluation unit that evaluates the degree of the feature amount representing the feature of the findings;
The image processing apparatus according to any one of claims 1 to 3, wherein the specifying unit specifies a tomographic image having the maximum degree among the plurality of tomographic images.   The image processing apparatus according to claim 4, wherein the evaluation unit evaluates a degree representing the feature of the finding based on at least one feature amount and a predetermined weight for the feature amount.   The image processing apparatus according to claim 1, further comprising an inference unit that infers a diagnosis name using the information on the findings.   The image processing apparatus according to claim 6, wherein the finding acquisition unit acquires finding information that has contributed to the inference among the finding information used for the inference.   The said finding acquisition means acquires the information of the said finding based on the information input into the diagnostic result preserve | saved at the database, It is any one of Claim 1 thru | or 7 characterized by the above-mentioned. Image processing apparatus. A finding acquisition means for acquiring findings information that is a wording that expresses the characteristics of abnormal shadows in a three-dimensional medical image including a plurality of tomographic images;
A characteristic quantity is a value indicating characteristics of look at, the feature amount associated beforehand with the information of the findings, the characteristic amount obtaining means for obtaining in the tomographic image,
Identifying means for identifying at least one tomographic image to be used for a report from the plurality of tomographic images based on the feature amount;
A report creation support apparatus characterized by comprising: An evaluation unit that evaluates the degree of the feature amount representing the feature of the findings;
10. The report creation according to claim 9, wherein the specifying unit specifies at least one tomographic image to be used for the report together with information indicating the degree based on the degree among the plurality of tomographic images. Support device.   The said finding acquisition means acquires the information of the said finding based on the information input into the diagnostic result preserve | saved at the database, It is any one of Claim 9 or Claim 10 characterized by the above-mentioned. Report creation support device. A finding acquisition means for acquiring findings information that is a wording that expresses the characteristics of abnormal shadows in a plurality of medical images;
A characteristic quantity is a value indicating characteristics of look at, the feature amount associated beforehand with the information of the findings, the characteristic amount obtaining means for obtaining in the medical imaging,
A specifying means for specifying at least one medical image having a high feature amount representing the feature of the findings among the plurality of medical images;
An image processing apparatus comprising: A first step of obtaining information of findings, which is a wording that expresses the characteristics of abnormal shadows in a three-dimensional medical image including a plurality of tomographic images;
A characteristic quantity is a value indicating characteristics of saw where obtained in the first step, the feature amount associated beforehand with the information of the findings, a second step of obtaining in the tomographic image ,
A third step of identifying at least one tomogram from the plurality of tomograms based on the feature amount;
A method of operating an image processing apparatus, comprising : A first step of acquiring a feature amount that is a value indicating a feature of an image in a three-dimensional medical image including a plurality of tomographic images;
A second step of acquiring findings information, which is a wording that expresses a feature of an abnormal shadow in the three-dimensional medical image, based on the feature amount acquired in the first step;
A third step of identifying at least one tomographic image from the plurality of tomographic images based on the feature amount previously associated with the finding information acquired in the second step;
A method of operating an image processing apparatus, comprising : Obtaining information of findings that are words expressing abnormal shadow characteristics in a three-dimensional medical image including a plurality of tomographic images;
A characteristic quantity is a value indicating characteristics of look at, the feature amount associated beforehand with the information of the findings, a step of acquiring in the tomographic image,
Identifying at least one tomographic image to be used for a report from the plurality of tomographic images based on the feature amount;
A method for operating a report creation support apparatus, comprising :   A program for causing a computer to execute the method according to any one of claims 13 to 15. A finding acquisition means for acquiring findings information on a three-dimensional medical image including a plurality of tomographic images;
A feature value acquiring means for acquiring feature quantity is a value indicating characteristics of seeing where Te the tomographic image smell,
Specifying means for specifying at least one tomographic image from the plurality of tomographic images based on the feature amount;
An inference means for inferring a diagnosis name using the information of the findings,
The finding acquisition means further acquires the finding information that contributed to the inference from the finding information used in the inference.

Images