JP7788864B2 - Medical image processing equipment - Google Patents

Description

The embodiments disclosed in this specification and drawings relate to a medical image processing device.

Conventionally, medical image processing devices may display the differences between specified image data to make it easier for doctors and other medical professionals to check changes in tumors or bones over time. For example, to examine changes over time, medical professionals specify image data captured during the current examination and image data captured during the previous examination. The medical image processing device then displays the differences between both sets of image data.

Medical professionals may want to compare image data with other image data, not just image data from a previous examination. In this case, medical professionals select the image data to compare with from an image list. However, medical professionals may have difficulty identifying the image data to select.

Japanese Patent Application Laid-Open No. 2005-277558

One of the problems that the embodiments disclosed in this specification and drawings attempt to solve is to assist in the selection of image data to be compared. However, the problems that the embodiments disclosed in this specification and drawings attempt to solve are not limited to the above problem. Problems corresponding to the effects of each configuration shown in the embodiments described below can also be positioned as other problems.

A medical image processing apparatus according to an embodiment includes a designation unit, a selection unit, a first generation unit, a display control unit, and a second generation unit. The designation unit designates first image data. The selection unit selects one or more second image data from a plurality of image datasets each having a plurality of image data. The first generation unit generates a difference image between the first image data and one or more of the second image data. The display control unit displays a list image showing a list of the difference images generated by the first generation unit. The second generation unit generates a difference image between the first image data and a plurality of image data in the image dataset each having the second image data on which the difference image selected from the list image was based.

FIG. 1 is a block diagram showing an example of the configuration of a medical image processing apparatus according to this embodiment. FIG. 2 is a diagram showing an example of a sample list image. FIG. 3 is a flowchart showing an example of a difference image generation process executed by the medical image processing apparatus according to this embodiment.

Hereinafter, a medical image processing device according to an embodiment will be described with reference to the drawings. In the following embodiments, parts with the same reference numerals perform similar operations, and duplicate descriptions will be omitted where appropriate.

(Present embodiment)
1 is a block diagram showing an example of the configuration of a medical image processing apparatus 10 according to this embodiment. The medical image processing apparatus 10 is an apparatus that supports the selection of image data to be compared when displaying the differences between two or more image data. For example, the medical image processing apparatus 10 is realized by a computer device such as a personal computer or a tablet terminal.

For example, the medical image processing device 10 has a network interface 110, an input interface 120, a display 130, a memory circuit 140, and a processing circuit 150.

The NW interface 110 is connected to the processing circuit 150 and controls the transmission and communication of various data between each device connected via the network. For example, the NW interface 110 is realized by a network card, network adapter, NIC (Network Interface Controller), etc.

The input interface 120 is connected to the processing circuit 150 and converts input operations received from an operator (medical professional) into electrical signals and outputs them to the processing circuit 150. Specifically, the input interface 120 converts input operations received from the operator into electrical signals and outputs them to the processing circuit 150. For example, the input interface 120 may be implemented by a trackball, switch buttons, a mouse, a keyboard, a touchpad that performs input operations by touching the operation surface, a touchscreen that integrates a display screen and a touchpad, a non-contact input circuit using an optical sensor, and a voice input circuit. Note that in this specification, the input interface 120 is not limited to those equipped with physical operating components such as a mouse and keyboard. For example, an electrical signal processing circuit that receives electrical signals corresponding to input operations from an external input device provided separately from the device and outputs these electrical signals to a control circuit is also included as an example of the input interface 120.

The display 130 is connected to the processing circuit 150 and displays various information and image data output from the processing circuit 150. For example, the display 130 may be implemented as a liquid crystal display, a CRT (Cathode Ray Tube) display, an organic EL display, a plasma display, a touch panel, etc.

The memory circuitry 140 is connected to the processing circuitry 150 and stores various data. The memory circuitry 140 also stores various programs that the processing circuitry 150 reads and executes to realize various functions. For example, the memory circuitry 140 may be implemented using semiconductor memory elements such as RAM (Random Access Memory) or flash memory, or a hard disk, optical disk, etc.

The processing circuitry 150 controls the overall operation of the medical image processing device 10. The processing circuitry 150 has, for example, an image designation function 151, a comparison target acquisition function 152, an image data acquisition function 153, a similarity calculation function 154, a comparison target identification function 155, a region detection function 156, a sample image generation function 157, a priority calculation function 158, a difference extraction function 159, a display control function 160, an operation control function 161, and a difference image generation function 162. In the embodiment, each processing function performed by the components of the image designation function 151, the comparison target acquisition function 152, the image data acquisition function 153, the similarity calculation function 154, the comparison target identification function 155, a region detection function 156, a sample image generation function 157, a priority calculation function 158, a difference extraction function 159, a display control function 160, an operation control function 161, and a difference image generation function 162 is stored in the memory circuitry 140 in the form of a computer-executable program. The processing circuitry 150 is a processor that reads programs from the storage circuitry 140 and executes them to realize the functions corresponding to each program. In other words, once each program has been read, the processing circuitry 150 has the functions shown in the processing circuitry 150 in Figure 1.

Note that while Figure 1 illustrates a single processor implementing the image designation function 151, comparison target acquisition function 152, image data acquisition function 153, similarity calculation function 154, comparison target identification function 155, part detection function 156, sample image generation function 157, priority calculation function 158, difference extraction function 159, display control function 160, operation control function 161, and difference image generation function 162, it is also possible to combine multiple independent processors to form the processing circuit 150, and have each processor execute a program to implement the function. Also, while Figure 1 illustrates a single memory circuit such as memory circuit 140 storing programs corresponding to each processing function, it is also possible to have multiple memory circuits distributed and have the processing circuit 150 read corresponding programs from individual memory circuits.

The term "processor" used in the above description refers to circuits such as a CPU (Central Processing Unit), GPU (Graphical Processing Unit), or an Application Specific Integrated Circuit (ASIC), a programmable logic device (e.g., a Simple Programmable Logic Device (SPLD), a Complex Programmable Logic Device (CPLD), and a Field Programmable Gate Array (FPGA)). The processor achieves its functions by reading and executing a program stored in the memory circuitry 140. Note that instead of storing a program in the memory circuitry 140, the processor may be configured so that the program is directly embedded in its circuitry. In this case, the processor achieves its functions by reading and executing the program embedded in its circuitry.

When medical professionals capture image data using a medical image diagnostic device, they may compare the image data captured this time with image data captured previously. However, when there are multiple sets of image data captured in the past, medical professionals may find it difficult to identify the image data to select for comparison. For example, if a patient suffers from multiple illnesses, the patient may undergo tests specific to each illness. In such cases, the contents of the current test may not match the contents of the previous test. If the test contents differ, medical professionals will not be able to make a meaningful diagnosis even if the differences in the image data are displayed. Therefore, the medical image processing device 10 supports the selection of image data to be compared using the following functions.

The image designation function 151 designates the first image data. The image designation function 151 is an example of a designation unit. Here, the first image data is medical image data captured by a medical image diagnostic device such as an X-ray CT (Computed Tomography) device, an MRI (Magnetic Resonance Imaging) device, an X-ray diagnostic device, an ultrasound diagnostic device, or a mammography device. The first image data includes the patient.

More specifically, the image designation function 151 designates image data selected by a medical professional or the like from image data captured by a medical image diagnostic device and stored in a PACS (Picture Archiving and Communication Systems) or the like as the first image data. The first image data may also be image data selected from three-dimensional volume data.

The comparison object acquisition function 152 acquires one or more image datasets containing multiple image data as a comparison object for the first image data specified by the image designation function 151. For example, the image datasets are multiple image data such as three-dimensional volume data captured by a medical image diagnostic device. More specifically, the comparison object acquisition function 152 acquires one or more image datasets from a PACS. In other words, the comparison object acquisition function 152 acquires image datasets from which a patient was previously captured.

The image data acquisition function 153 acquires information about the image data contained in the image dataset acquired by the comparison target acquisition function 152. More specifically, the image data acquisition function 153 acquires information about the image data from DICOM (Digital Imaging and Communications in Medicine) tags, radiology reports, electronic medical record information, etc.

Information related to image data includes, for example, the degree of match of the image data, the imaging conditions, and the series description. The degree of match is the degree of match between the first image data specified by the image specification function 151 and the second image data selected from the image dataset acquired by the comparison target acquisition function 152. The imaging conditions are the conditions for capturing the image data captured by the medical image diagnostic device. For example, the imaging conditions include the imaging range, tube voltage, and the presence or absence of contrast agent. The series description is information describing the contents of the image dataset written by a medical professional such as a technician operating the medical image diagnostic device.

The similarity calculation function 154 calculates the similarity between the first image data specified by the image specification function 151 and the image data contained in the image dataset acquired by the comparison object acquisition function 152. Here, it is meaningless for medical professionals to compare completely different image data, such as that of the head and abdomen. In other words, the medical image processing device 10 should compare similar image data. Therefore, the similarity calculation function 154 calculates the similarity between the image dataset containing the first image data and the image dataset acquired by the comparison object acquisition function 152. For example, the similarity calculation function 154 calculates the similarity based on the number of items that match between information about the image data of the first image data and information about the image data of the image dataset acquired by the comparison object acquisition function 152.

The comparison target identification function 155 selects one or more second image data from multiple image datasets containing multiple image data, which serve as the basis for a sample image showing the difference from the first image data specified by the image designation function 151. The comparison target identification function 155 is an example of a selection unit. A sample image is a difference image generated as a sample to make it easier for medical professionals to understand which image dataset to select when generating a formal difference image. Specifically, a sample image is a difference image showing the difference between a portion of the image data contained in the image dataset and the first image data. In this way, the medical image processing device 10 reduces the processing time required to generate a difference image by selecting only a portion of the image data contained in the image dataset, rather than all of the image data.

For example, the comparison target identification function 155 identifies an image dataset from which second image data will be selected, which will serve as the basis for generating a sample image, from among the multiple image datasets acquired by the comparison target acquisition function 152. For example, the comparison target identification function 155 identifies an image dataset based on the similarity between the first image data specified by the image designation function 151 and the image data contained in the multiple image datasets acquired by the comparison target acquisition function 152. The comparison target identification function 155 is an example of an identification unit. For example, the comparison target identification function 155 identifies an image dataset whose similarity is equal to or greater than a threshold. Alternatively, the comparison target identification function 155 identifies, from among the multiple image datasets acquired by the comparison target acquisition function 152, an image dataset from an examination undertaken by the same patient for the same purpose, i.e., the previous examination, as the image dataset from which second image data will be selected.

The comparison target identification function 155 also selects second image data that will serve as the basis for the sample image from the multiple image data included in the identified image dataset. In other words, the comparison target identification function 155 selects second image data that will serve as the basis for the difference image. For example, the comparison target identification function 155 selects image data that is located at a set position among the image data included in the image dataset as the second image data. Specifically, the comparison target identification function 155 selects image data that is located at a set position, such as the beginning, middle, or end of the image dataset, as the second image data.

Alternatively, the comparison target identification function 155 selects estimated important image data that is estimated to be important from the image data included in the image dataset as the second image data. For example, estimated important image data is image data included in an interpretation report that describes the test results, image data that describes the findings of a medical professional, image data to which information such as markings or annotations has been added, or image data referenced in electronic medical record information. Note that estimated important image data is not limited to the image data described above, and may be other image data. Furthermore, the comparison target identification function 155 may detect estimated important image data from an image dataset whose similarity is equal to or greater than a threshold, or may detect estimated important image data from an image dataset acquired by the comparison target acquisition function 152.

The part detection function 156 identifies the part of the patient included in the second image data selected by the comparison object identification function 155. For example, the part detection function 156 detects anatomical features included in the second image data. The part detection function 156 then identifies the part of the patient based on the detected anatomical features and the coordinates of the anatomical features in the second image data. The part of the patient is used to describe the second image data.

The difference extraction function 159 extracts the difference details by comparing the first image data specified by the image specification function 151 with the second image data selected by the comparison target identification function 155. For example, the difference extraction function 159 extracts the difference in the number of tumors contained in at least one of the first image data and the second image data, the difference in tumor size, the patient's weight, or the patient's age as the difference details. Note that the difference extraction function 159 may extract difference details regarding other content as the difference between the first image data and the second image data, in addition to the number of tumors, tumor size, patient weight, or patient age.

The sample image generation function 157 generates a difference image between the first image data specified by the image specification function 151 and one or more second image data selected by the comparison target identification function 155. The sample image generation function 157 is an example of a first generation unit. In this way, the sample image generation function 157 generates a difference image between the second image data selected from the image dataset and the first image data as a sample image.

The priority calculation function 158 calculates a priority for each sample image, indicating the degree to which it should be used as a basis for generating an official difference image. The priority calculation function 158 calculates the priority based on information about the image data acquired by the image data acquisition function 153, the parts detected by the part detection function 156, and the difference content extracted by the difference extraction function 159. For example, the priority calculation function 158 calculates the priority by calculating the degree of similarity between the information about the image data, the degree of similarity between the parts, and the degree of similarity between the difference content between the first image data and the second image data.

The display control function 160 causes the display 130 to display a sample list image G1 containing a list of sample images showing the differences between two pieces of image data generated by the sample image generation function 157. The display control function 160 is an example of a display control unit.

Figure 2 shows an example of a sample list image G1. The sample list image G1 is a list of sample images generated by the sample image generation function 157. More specifically, the sample list image G1 shows information about the sample image and the second image data on which the sample image is based, for each selection condition under which the comparison target identification function 155 selected the second image data and the date on which the second image data was captured.

The selection conditions are the conditions under which the second image data that served as the basis for the sample image were selected when generating the sample image. For example, the selection conditions may be image data from a specified position in an image dataset captured in a previous examination, image data to which information such as annotations or remarks has been added, or image data from which findings have been generated. The selection conditions may also be image data that is presumed to be important, even though a sample image, i.e., a difference image, has not been generated.

In the sample list image G1, the display control function 160 displays the sample images in association with information about the second image data on which the sample images are based. For example, the display control function 160 displays a summary description, body parts, differences, priority, etc. as information about the second image data on which the sample images are based. The summary description is information about the image data of the second image data on which the sample images are based, such as the degree of similarity between the first image data and the second image data, the shooting conditions, and a series description. In other words, the display control function 160 displays a summary description of the second image data on which the difference images are based in association with the difference images.

The body part is the patient's body part contained in the second image data on which the sample image detected by the body part detection function 156 is based. In other words, the display control function 160 displays the patient's body part contained in the second image data on which the sample image is based in association with the sample image.

The difference is information indicating the difference between the first image data and the second image data extracted by the difference extraction function 159. In other words, the display control function 160 displays the difference between the second image data on which the sample image is based in association with the sample image.

The priority is the order of priority calculated by the priority calculation function 158. In other words, the display control function 160 displays, in association with the sample image, the priority for selecting an image dataset having the second image data on which the sample image is based.

In the sample list image G1 shown in Figure 2, selection condition A shows a state in which sample images are presented on dates "2021,03,17," "2021,03,10," "2021,03,03," and "2021,02,24," along with a summary description, parts, differences, and priority. Selection condition B shows a state in which sample images are presented on dates "2021,03,17," and "2021,03,10," along with a summary description, parts, differences, and priority. Selection condition C shows a state in which a sample image is presented on date "2021,02,17," along with a summary description, parts, differences, and priority. In the sample list image G1 shown in Figure 2, "N/A" indicates that no relevant data exists.

Furthermore, when the second image data on which the sample image is based is estimated important image data, the display control function 160 adds a first estimated important mark G11 to the corresponding sample image. In other words, the display control function 160 displays the first estimated important mark G11, which indicates that the second image data on which the sample image is based is estimated to be important, in association with the sample image. The first estimated important mark G11 is an example of a first mark.

Furthermore, when the selection condition is that the image data is estimated to be important even though no sample image has been generated, the display control function 160 displays a sample list image G1 including image data that is estimated to be important but for which no sample image has been generated. The display control function 160 also adds a second importance estimation mark G12 to the corresponding image data. In other words, the display control function 160 displays the second importance estimation mark G12 in association with the image data, indicating that the image data is estimated to be important but for which no sample image has been generated. The second importance estimation mark G12 is an example of a second mark.

In the sample list image G1 shown in Figure 2, the first important estimation mark G11 corresponds to the sample images for the dates "2021,03,17" and "2021,03,03" under selection condition A and "2021,03,10" under selection condition B. The second important estimation mark G12 corresponds to the sample image for the date "2021,02,17" under selection condition C.

The operation control function 161 accepts an operation to select an image dataset for which a difference image is to be generated. For example, the operation control function 161 accepts an operation to select a sample image included in the sample list image G1. The operation control function 161 accepts the image dataset having the second image data that is the basis of the selected sample image as the image dataset for which a difference image is to be generated.

The difference image generation function 162 generates difference images between multiple image data of an image dataset having second image data that is the basis of a sample image selected from the sample list image G1 and the first image data specified by the image designation function 151. The difference image generation function 162 is an example of a second generation unit. In other words, the difference image generation function 162 generates a difference image when the operation control function 161 accepts an operation to select a sample image from the sample list image G1.

Next, we will explain the difference image generation process executed by the medical image processing apparatus 10. Figure 3 is a flowchart showing an example of the difference image generation process executed by the medical image processing apparatus 10 according to this embodiment.

The image designation function 151 designates the first image data, which is the image data to be compared (step S1).

The comparison target acquisition function 152 acquires an image dataset containing one or more image data (step S2).

The comparison target identification function 155 selects one or more second image data from the image dataset acquired by the comparison target acquisition function 152 (step S3).

The body part detection function 156 detects the body part of the patient contained in the second image data (step S4).

The sample image generation function 157 generates sample images showing the differences between the first image data and one or more second image data (step S5).

The priority calculation function 158 calculates a priority indicating the degree to which the generated official difference image should be based (step S6).

The difference extraction function 159 extracts the differences between the first image data and one or more second image data (step S7).

The display control function 160 displays a sample list image G1 showing sample images for each selection condition for selecting the second image data and the date on which the second image data was generated (step S8).

The operation control function 161 determines whether an operation to select a sample image included in the sample list image G1 has been received (step S9). If an operation to select a sample image has not been received (step S9; No), the operation control function 161 waits.

When an operation to select a sample image is received (step S9; Yes), the difference image generation function 162 generates official difference images between the first image data and the multiple image data contained in the image dataset corresponding to the sample image (step S10).

The medical image processing device 10 then completes the difference image generation process.

As described above, the image designation function 151 of the medical image processing apparatus 10 according to this embodiment designates first image data. Furthermore, the comparison target identification function 155 selects one or more second image data from one or more image datasets containing multiple image data. The sample image generation function 157 generates sample images showing the differences between the first image data and one or more second image data. The display control function 160 displays a sample list image G1 containing a list of sample images. When a sample image from the sample list image G1 is selected, the difference image generation function 162 generates difference images between the first image data and multiple image data from the image dataset containing the second image data on which the selected sample image was based. In this way, the list of sample images is displayed, making it easy for medical professionals to understand what kind of difference images will be generated. Therefore, the medical image processing apparatus 10 can assist in the selection of image data to be compared.

At least one of the embodiments described above can assist in the selection of images to be compared.

While several embodiments have been described, these embodiments are presented as examples and are not intended to limit the scope of the invention. These embodiments may be embodied in a variety of other forms, and various omissions, substitutions, modifications, and combinations of embodiments may be made without departing from the spirit of the invention. These embodiments and their variations are included within the scope and spirit of the invention, as well as within the scope of the invention and its equivalents as set forth in the claims.

10 Medical image processing device 110 NW (network) interface 120 Input interface 130 Display 140 Memory circuit 150 Processing circuit 151 Image designation function 152 Comparison object acquisition function 153 Image data acquisition function 154 Similarity calculation function 155 Comparison object identification function 156 Part detection function 157 Sample image generation function 158 Priority calculation function 159 Difference extraction function 160 Display control function 161 Operation control function 162 Difference image generation function G1 Sample list image G11 First important estimated mark G12 Second important estimated mark

Claims (9)

a designation unit for designating the first image data;
a selection unit that selects one or more second image data from a plurality of image data sets having a plurality of image data;
a first generation unit that generates a difference image between the first image data and one or more pieces of second image data;
a display control unit that displays a list image having a list of the difference images generated by the first generation unit;
a second generation unit that generates difference images between the first image data and the plurality of image data in the image data set having the second image data that is the basis of the difference image selected from the list of images;
A medical image processing device comprising: the display control unit causes the difference image and information about the second image data on which the difference image is based to be displayed in association with each other.
The medical image processing device according to claim 1 . a specifying unit for specifying the image data set from which the second image data is selected from the plurality of image data sets;
The selection unit selects the second image data from the plurality of image data included in the image data set identified by the identification unit.
3. The medical image processing device according to claim 1. the display control unit displays a first mark, which indicates that the second image data on which the difference image is based is estimated to be important, in association with the difference image.
The medical image processing device according to any one of claims 1 to 3. the display control unit displays a part of the patient included in the second image data on which the difference image is based in association with the difference image.
The medical image processing device according to any one of claims 1 to 4. the display control unit displays a difference between the first image data and the second image data on which the difference image is based, in association with the difference image.
The medical image processing device according to any one of claims 1 to 5. The display control unit displays, in association with the difference image, a priority order for selecting the image data set having the second image data on which the difference image is based.
The medical image processing device according to any one of claims 1 to 6. the display control unit displays the list image including the image data for which the difference image has not been generated and which is estimated to be important.
The medical image processing device according to any one of claims 1 to 7. the display control unit displays a second mark corresponding to the image data for which the differential image has not been generated and which indicates that the image data is estimated to be important.
The medical image processing device according to claim 8 .