JP7633033B2 - Medical image processing equipment - Google Patents

Description

The embodiments disclosed in this specification and the drawings relate to a medical image processing device.

For example, when a user such as an image interpreter wants to determine whether a tumor in a subject has metastasized to the bone, the user needs medical images taken in previous examinations to compare with medical images taken in the current examination. In this case, for example, the user performs a search using a patient ID or the like to obtain medical images of the subject taken in previous examinations from a server. At this time, the user visually determines whether the obtained medical images can be compared with the medical images of the subject taken in the current examination. In this case, time is spent searching for medical images to compare with before performing image interpretation, and the efficiency of image interpretation does not improve.

JP 2014-6130 A

One of the problems that the embodiments disclosed in this specification and the drawings aim to solve is to improve the efficiency of image interpretation. However, the problems solved by the embodiments disclosed in this specification and the drawings are not limited to the above problem. Problems corresponding to the effects of each configuration shown in the embodiments described below can also be positioned as other problems.

The medical image processing device according to this embodiment includes an acquisition unit, a control unit, and a presentation unit. The acquisition unit acquires a medical image of a subject. The control unit extracts a difference target image to be subtracted from the medical image from among past medical images of the subject taken at multiple different times prior to the medical image, based on first examination information, which is information on the examination in which the medical image was taken, and second examination information, which is information on the examination in which the past medical image was taken. The presentation unit presents the difference target image and the medical image together with the second examination information of the difference target image and the first examination information.

FIG. 1 is a diagram showing an example of the configuration of a medical image processing system including a medical image processing apparatus according to this embodiment. FIG. 2 is a diagram showing an example of the configuration of a medical image processing apparatus according to this embodiment. FIG. 3 is a diagram showing an example of conditions related to photography in this embodiment. FIG. 4 is a diagram showing an example of information relating to the purpose of the examination in this embodiment. FIG. 5 is a diagram for explaining an example of learning using AI in this embodiment. FIG. 6 is a diagram illustrating an example of a process using a trained model in this embodiment. FIG. 7 is a flowchart showing the procedure of processing by the medical image processing apparatus 100 according to this embodiment. FIG. 8 is a diagram for explaining the processing by the medical image processing apparatus 100 according to this embodiment. FIG. 9 is a diagram for explaining the processing by the medical image processing apparatus 100 according to this embodiment. FIG. 10 is a diagram for explaining the processing by the medical image processing apparatus 100 according to this embodiment.

Below, an embodiment of a medical image processing device will be described in detail with reference to the attached drawings. Note that the following description will be given taking as an example a medical image processing system including a medical image processing device. In the medical image processing system shown in Figure 1, one of each device is shown, but in reality, the system may include multiple devices.

The medical image processing system shown in FIG. 1 is a system including a hospital information system (HIS), a radiology information system (RIS), and a picture archiving and communication system (PACS). The medical image processing system includes a HIS server 10, a RIS server 20, a medical image diagnostic device 30, a PACS server 40, a terminal 50, an application server 60, and a medical image processing device 100.

The HIS server 10, RIS server 20, medical image diagnostic device 30, PACS server 40, terminal 50, medical image processing device 100, and application server 60 are connected to, for example, an in-hospital LAN (Local Area Network) installed in a hospital, and transmit information to a specified device and receive information transmitted from the specified device. Note that the HIS server 10 may be connected to an external network in addition to the in-hospital LAN.

For example, the terminal 50 is used by a radiologist who performs radiological interpretation using medical images and the like. For example, the terminal 50 includes a PC (Personal Computer), a tablet PC, a PDA (Personal Digital Assistant), a mobile terminal, and the like. A viewer (software) is installed on the terminal 50 for displaying images (medical images) and medical reports (radiological interpretation reports) based on medical image data on the terminal's own display.

HIS is a system that manages information generated within a hospital. Here, we will not describe the terminals (HIS terminals) that medical staff use to perform various tasks within the hospital.

In the HIS, the HIS server 10 shown in FIG. 1 manages information generated within the hospital. Information generated within the hospital includes patient information, test order information, etc. Each record included in the patient information has items such as a patient ID, patient name (first and last name), age (date of birth), sex, height, weight, and blood type. Each record included in the test order information has items such as a test ID that can identify the test, a patient ID, information indicating hospitalization or outpatient care, a test code, a medical department, a type of test, a part of the test, and a scheduled test date and time.

The examination ID is issued when the examination order information is input, and is an identifier for uniquely identifying the examination order information within, for example, one hospital. The patient ID is assigned to each patient, and is an identifier for uniquely identifying the patient within, for example, one hospital. The examination code is an identifier for uniquely identifying an examination, which is defined within, for example, one hospital. The medical department indicates, for example, the specialty division of medical treatment in medicine. Specifically, the medical department is internal medicine, surgery, etc. The examination type indicates an examination using medical images. For example, the examination types include X-ray examinations, CT (Computed Tomography) examinations, and MRI (Magnetic Resonance Imaging) examinations. The examination sites include the brain, kidneys, lungs, liver, and bones, etc.

For example, when test order information is input by a doctor requesting a test, the HIS server 10 transmits the input test order information and patient information identified by the test order information to the RIS. In this case, the HIS server 10 also transmits the patient information to the PACS.

The RIS is a system that manages examination reservation information related to radiological examination work. Here, we will omit the description of the terminals (RIS terminals) used by, for example, radiological technologists to carry out various tasks in the radiology department.

In the RIS, the RIS server 20 shown in FIG. 1 manages information related to radiological examination operations. For example, the RIS server 20 receives examination order information sent from the HIS server 10, adds various setting information to the received examination order information, accumulates it, and manages the accumulated information as examination reservation information. Specifically, when the RIS server 20 receives patient information and examination order information sent from the HIS server 10, it generates examination reservation information necessary to operate the medical image diagnostic device 30 based on the received patient information and examination order information. The examination reservation information includes information necessary to perform the examination, such as, for example, an examination ID, a patient ID, an examination type, and an examination site. The RIS server 20 transmits the generated examination reservation information to the medical image diagnostic device 30.

The medical image diagnostic device 30 shown in FIG. 1 is a device with which a clinical laboratory technician performs examinations by photographing a patient. The medical image diagnostic device 30 includes, for example, an X-ray diagnostic device, an X-ray computed tomography device (X-ray CT device), a magnetic resonance imaging device (MRI device), a nuclear medicine diagnostic device, and an ultrasound diagnostic device. The medical image diagnostic device 30 is also called a modality device.

The medical image diagnostic device 30 performs an examination based on examination reservation information transmitted from the RIS server 20, for example. The medical image diagnostic device 30 then generates examination implementation information indicating the implementation of the examination and transmits it to the RIS server 20. In this case, the RIS server 20 receives the examination implementation information from the medical image diagnostic device 30 and outputs the received examination implementation information to the HIS server 10 and the terminal 50 as the latest examination implementation information. For example, the HIS server 10 receives the latest examination implementation information and manages the received examination implementation information. For example, the image interpretation doctor's terminal 50 receives the latest examination implementation information and creates an image interpretation report, which is a medical report, according to the received examination implementation information. The examination implementation information includes examination reservation information (examination ID, patient ID, examination type, and examination area), and the date and time of the examination.

In addition, when performing an examination, the medical image diagnostic device 30 generates medical image data by having a clinical laboratory technician photograph a subject (patient). The medical image data is, for example, X-ray CT image data, X-ray image data, MRI image data, nuclear medicine image data, and ultrasound image data. The medical image diagnostic device 30 converts the generated medical image data into a format that complies with, for example, the DICOM (Digital Imaging and Communication in Medicine) standard. In other words, the medical image diagnostic device 30 generates medical image data to which a DICOM tag has been added as supplementary information.

The additional information includes, for example, a patient ID, an examination ID, an apparatus ID, an image series ID, conditions related to imaging, etc., and is standardized according to the DICOM standard. The apparatus ID is information for identifying the medical image diagnostic apparatus 30. The image series ID is information for identifying one imaging session by the medical image diagnostic apparatus 30, and includes, for example, the part of the subject (patient) imaged, the time of image generation, slice thickness, slice position, etc. For example, by performing a CT examination or an MRI examination, cross-sectional images at each of multiple slice positions are obtained as medical image data. The conditions related to imaging will be described later.

The medical image diagnostic device 30 transmits the generated medical image data to the PACS. The PACS is a system that manages various types of medical image data.

In the PACS, the PACS server 40 shown in FIG. 1 receives, for example, patient information transmitted from the HIS server 10 and manages the received patient information. The PACS server 40 is equipped with a memory circuit for managing patient information. For example, the PACS server 40 receives medical image data transmitted from the medical image diagnostic device 30, associates the received medical image data with patient information, and stores it in its own memory circuit. For example, the PACS server 40 reads medical image data in response to an acquisition request from the medical image processing device 100 from its own memory circuit and transmits it to the medical image processing device 100. In addition, the medical image data stored in the PACS server 40 is added with additional information such as a patient ID, an examination ID, an apparatus ID, and an image series ID. Therefore, a user can obtain the necessary patient information from the PACS server 40 by performing a search using a patient ID or the like. Here, the user includes medical professionals such as radiologists. Additionally, users can retrieve the required medical image data from the PACS server 40 by searching using a patient ID, examination ID, device ID, image series ID, etc.

The medical image processing device 100 shown in FIG. 1 is a workstation for displaying images (medical images) and medical reports (radiography reports) based on medical image data on the display of a terminal 50. A radiologist performs radiography interpretation using medical images and the like displayed on the display of the terminal 50 by the medical image processing device 100. Here, when performing radiography interpretation, the radiologist may compare the medical image data of the subject captured in the current examination with medical image data of the subject captured in a previous examination that precedes the current examination. For example, the radiologist determines whether or not the subject's tumor has metastasized to the bone by comparing the medical image data of the subject captured in the current examination with the medical image data of the subject captured in a previous examination.

The application server 60 shown in FIG. 1 is implemented with a bone subtraction application (program). The bone subtraction application can be read by the medical image processing device 100. The bone subtraction application is an application that supports the detection of bone metastases.

For example, when the medical image data is X-ray CT image data, the bone subtraction application performs binarization processing such as segmentation on the medical image data in the current examination and the medical image data in the previous examination to generate a bone extraction image in which the bone portion is extracted. The bone subtraction application also performs preprocessing such as contrast conversion on the medical image data in the current examination and the medical image data in the previous examination to generate a bone-enhanced image in which the bone portion is enhanced as a medical image based on the medical image data. The bone subtraction application then aligns the bone extraction images and reflects the result of the alignment in the alignment between the bone-enhanced images. The bone subtraction application then takes the difference between the aligned bone-enhanced image in the current examination and the bone-enhanced image in the previous examination to generate a difference image in which the area of bone metastasis is enhanced.

For example, in a bone subtraction application, a difference is taken between a medical image of a subject taken in a previous examination before a tumor developed and a medical image of the subject taken in a current examination after a tumor developed, in which the CT value of the bone has changed, and a difference image is generated that represents the difference. By referring to the difference image, a radiologist can determine the type of bone metastasis, such as whether the area depicted in the difference image is osteolytic metastasis, which has an area where bone has dissolved, osteoblastic metastasis, which has an area where bone has been created, or mixed metastasis, which is a combination of osteolytic and osteoblastic metastasis.

The medical image processing device 100 according to this embodiment will be described in detail below. FIG. 2 is a diagram showing an example of the configuration of the medical image processing device 100 according to this embodiment. As shown in FIG. 2, the medical image processing device 100 has an input interface 110, displays 120 and 123, a communication interface 130, a memory circuitry 140, and a processing circuitry 150.

The input interface 110 has a pointing device such as a mouse, a keyboard, etc., and receives input of various operations for the medical image processing device 100 from a user, and transfers the instructions and setting information received from the user to the processing circuitry 150. Here, the user refers to a medical professional, including an image interpretation doctor.

The display 120 is a monitor viewed by the user, and under the control of the processing circuit 150, displays image data to the user and displays a GUI (Graphical User Interface) for receiving various instructions and settings from the user via the input interface 110. The communication interface 130 is a NIC (Network Interface Card) or the like, and communicates with other devices.

The memory circuit 140 is, for example, a semiconductor memory element such as a random access memory (RAM) or a flash memory, or a storage device such as a hard disk or an optical disk. Here, the memory circuit 140 is an example of a memory unit.

The processing circuitry 150 controls the components of the medical image processing device 100. For example, the processing circuitry 150 executes a control function 151 and a display control function 152 as shown in FIG. 2. Here, for example, each processing function executed by the control function 151 and the display control function 152, which are components of the processing circuitry 150, is recorded in the memory circuitry 140 in the form of a program executable by a computer. The processing circuitry 150 is a processor that realizes the function corresponding to each program by reading and executing each program from the memory circuitry 140. In other words, the processing circuitry 150 in a state in which each program has been read has each function shown in the processing circuitry 150 in FIG. 2. The control function 151 is an example of an acquisition unit and a control unit, and the display control function 152 is an example of a presentation unit.

The term "processor" used in the above description means a circuit such as a central processing unit (CPU), a graphics processing unit (GPU), an application specific integrated circuit (ASIC), a programmable logic device (e.g., a simple programmable logic device (SPLD), a complex programmable logic device (CPLD), and a field programmable gate array (FPGA)). When the processor is a CPU, for example, the processor reads and executes a program stored in the memory circuit 140 to realize the function. On the other hand, when the processor is an ASIC, for example, instead of storing the program in the memory circuit 140, the program is directly incorporated into the circuit of the processor. Note that each processor in this embodiment is not limited to being configured as a single circuit for each processor, but may be configured as a single processor by combining multiple independent circuits to realize the function. Furthermore, the multiple components in FIG. 2 may be integrated into a single processor to realize the function.

The above describes the overall configuration of a medical image processing system including the medical image processing device 100 according to this embodiment. With this configuration, the medical image processing device 100 improves the efficiency of image interpretation.

For example, when a user such as an image interpreter determines whether a tumor in a subject has metastasized to bone, the user needs medical images taken in previous examinations to compare with medical images taken in the current examination. In this case, for example, the user performs a search using a patient ID or the like to obtain medical images of the subject taken in previous examinations from the PACS server 40. At this time, the user visually determines whether the obtained medical images are to be used as a difference target for the medical images of the subject taken in the current examination.

The reason for this is that the medical images of the subject taken in the current examination and the medical images of the subject taken in the previous examination must satisfy the imaging conditions. Examples of imaging conditions include whether the medical images taken in the current examination and the medical images taken in the previous examination are taken in the same position. For example, if the medical images taken in the current examination are medical images taken with the subject's arms raised, the medical images taken in the previous examination must also be medical images taken with the subject's arms raised. Examples of imaging conditions include whether the medical images taken in the current examination and the medical images taken in the previous examination include areas including landmarks for facilitating alignment, such as the scapula and pelvis. For example, if the medical images taken in the current examination are medical images of an area including the subject's scapula, it is preferable for alignment that the medical images taken in the previous examination are also medical images of an area including the subject's scapula. In either case, the user visually determines whether the medical image obtained from the PACS server 40 and the medical image captured during the current examination meet the imaging conditions.

Another reason is that the purpose of the examination in which the subject's medical image was taken in a previous examination is important. For example, if the user finds that the subject's tumor may have metastasized to the bone after referring to the medical image of the subject taken in the current examination, the user would like to refer to the medical image of the subject taken in the previous examination before the bone metastasis occurred. The medical image before the bone metastasis occurs is, for example, a medical image of the subject whose primary lesion was photographed in a previous examination. In addition, for example, the user would like to check the prognosis of the treatment for a subject who has already been treated for bone metastasis, so as a result of the runner referring to the medical image taken after the treatment of bone metastasis as the medical image of the subject taken in the current examination, the user would like to refer to the medical image taken before the treatment of bone metastasis. In either case, the user visually judges for what examination purpose the medical image of the subject obtained from the PACS server 40 was taken.

In this way, the user visually determines whether or not the medical image acquired from the PACS server 40 is the medical image to be subtracted from the medical image of the subject captured in the current examination. In this case, time is wasted searching for the medical image to be subtracted before interpretation, and the efficiency of interpretation is not improved.

Therefore, in the medical image processing apparatus 100 according to this embodiment, the following process is performed to improve the efficiency of interpretation. First, in the medical image processing apparatus 100 according to this embodiment, the control function 151 acquires a medical image of a subject. The control function 151 extracts a difference target image to be a difference target with the acquired medical image from past medical images taken of the subject at multiple different times before the medical image, based on first examination information, which is information about the examination in which the medical image was taken, and second examination information, which is information about the examination in which the past medical image was taken. The display control function 152 causes the terminal 50 to present the difference target image and the medical image together with the second examination information and the first examination information of the difference target image. The past medical image is an example of a medical image of the subject taken in a past examination. The acquired medical image is an example of a medical image of the subject taken in the current examination. The difference target image is an example of a medical image to be a difference target of the medical image of the subject taken in the current examination.

Here, the medical images to be subtracted are automatically extracted, for example, by learning or machine learning using AI (Artificial Intelligence) based on a neural network.

For example, in this embodiment, the conditions for photographing medical images taken in an examination are learned using a model described later. FIG. 3 is a diagram showing an example of the conditions for photographing in this embodiment. The conditions for photographing include information required for alignment between images. The conditions for photographing include, for example, reconstruction conditions, photographing conditions, and photographing range. The reconstruction conditions include, for example, a condition as to whether the medical image of the subject photographed in the current examination and the medical image of the subject photographed in the past examination are created using the same reconstruction function. Examples of the reconstruction function include a reconstruction function for soft tissue and a reconstruction function for lung fields. The photographing conditions include, for example, a condition as to whether the medical image of the subject photographed in the current examination and the medical image of the subject photographed in the past examination are photographed with the same parameters. Examples of parameters include tube voltage and tube current. The photographing conditions also include, for example, a condition as to whether the medical image of the subject photographed in the current examination and the medical image of the subject photographed in the past examination are photographed in the same position. The body position may be the subject's arms raised or lowered, or prone position. The imaging range includes conditions such as whether areas including landmarks such as the scapula and pelvis are captured in the medical images of the subject captured in the current examination and in the medical images of the subject captured in the previous examination. The reconstruction conditions, imaging conditions, imaging range, etc. described above are constraints for aligning the images.

In addition, for example, in this embodiment, information on the purpose of the examination for which the medical image was taken is learned using a model described later. FIG. 4 is a diagram showing an example of information on the purpose of the examination in this embodiment. As shown in FIG. 4, for example, the examination is performed in chronological order, such as "initial examination", "preoperative plan", "radiotherapy", "follow-up 1", "radiotherapy", "follow-up 2", and "present". For example, if the subject is suspected of having a tumor, the medical image of the subject is taken by the medical image diagnostic device 30 at the "initial examination" as the purpose of the examination. Then, it is assumed that "preoperative plan" is implemented and "radiotherapy" is performed on the subject. In this case, since it is desired to confirm the prognosis of the treatment, the medical image of the subject is taken by the medical image diagnostic device 30 at the "follow-up" of the first radiation therapy as the purpose of the examination. Then, it is assumed that "radiotherapy" is performed on the subject again. In this case, since it is desired to confirm the prognosis of the treatment, the medical image of the subject is taken by the medical image diagnostic device 30 at the "follow-up" of the second radiation therapy as the purpose of the examination. Then, at the "present", medical images of the subject are taken by the medical image diagnostic device 30 so that the image interpreting doctor can determine whether the subject's tumor has metastasized to the bone.

Note that the flow of the examination is not limited to the example shown in FIG. 4. In FIG. 4, "radiotherapy" is performed on the subject after "preoperative planning" is carried out, but "radiotherapy" may also be performed after surgery on the subject after "preoperative planning". Also, in a "follow-up" of the second radiation therapy, it may be found that the subject's tumor has metastasized to the bone, so the next examination may be performed "currently".

In addition, for example, in this embodiment, the time when the medical image was taken is also learned using a model described below. For example, even if the medical image of the subject was taken before bone metastasis occurred, if the medical image was taken when the subject was in a traffic accident, the medical image may not be useful for the user in interpreting the image, and so is excluded from the extraction target. For example, if the medical image of the subject was taken more than 10 years ago, it may not be useful for the user in interpreting the image, and so is excluded from the extraction target.

Figure 5 is a diagram for explaining an example of learning using AI in this embodiment. Figure 5 shows a block diagram assuming deep learning using AI. Figure 6 is a diagram for explaining an example of processing using a trained model in this embodiment.

In this embodiment, the trained model is generated, for example, by the control function 151 of the medical image processing device 100 and stored in the memory circuitry 140 in the medical image processing device 100. Here, the trained model may be realized by an in-hospital device or an external device.

In the process of generating a trained model, as shown in FIG. 5, first, the control function 151 inputs a training dataset 501 obtained when an examination was performed in the past. The training dataset 501 includes data that associates multiple medical images taken in past examinations with information related to the examinations in which the multiple medical images were taken. The data is stored in the memory circuitry 140. For example, the multiple medical images include medical images of a subject taken in each of multiple examinations performed on the subject at different times. Note that, since algorithms such as AI learn from experience, the datasets used for learning do not have to be from the same subject.

Here, the information about the examination includes the examination ID, the patient ID, the examination type, the examination site, the date and time of the examination, the conditions for taking the medical image taken in the examination, and information about the purpose of the examination for which the medical image was taken. As described above, the examination type indicates an examination using a medical image. For example, the examination type includes X-ray examination, CT examination, and MRI examination. The examination site includes the brain, kidney, lung, and liver. The information about the examination also includes information about findings, etc., and the findings include information about the type of tumor, the location of the primary lesion, and age. The conditions about the imaging include information required for positioning between images, and include, for example, the above-mentioned reconstruction conditions, imaging conditions, imaging range, and the like. The information about the purpose of the examination for which the medical image was taken includes, for example, information about whether the medical image of the subject taken in the past examination was a medical image taken at the "initial consultation", a medical image taken as a "follow-up" to the first radiation therapy, or a medical image taken as a "follow-up" to the second radiation therapy. In addition, information regarding the purpose of the examination for which the medical image was taken includes, for example, information as to whether the medical image of the subject was taken when the subject was in a traffic accident, or whether the image was taken more than 10 years ago.

The control function 151 has a learning program 502 as an AI algorithm. Using the learning program 502, the control function 151 learns from the learning dataset 501 a plurality of medical images taken in past examinations and information on the examinations in which the plurality of medical images were taken, and generates a trained model 503 that is functionalized to receive input data described below and output medical images extracted from the plurality of medical images and information on the examinations of the medical images as output data. For example, the generated trained model 503 is stored in the memory circuitry 140. Here, the learning method may be "supervised learning" in which the learning data is trained with a correct answer given, or "unsupervised learning" in which the learning data is trained without a correct answer given.

In a process that uses the trained model 503, as shown in FIG. 6, for example, the control function 151 inputs medical images of the subject captured in the current examination from the terminal 50 as input data 504. At this time, the control function 151 reads out the trained model 503 from the memory circuitry 140.

Then, the control function 151 inputs the medical image of the subject taken in the current examination and information related to the examination of the medical image as input data 504 from the terminal 50 to the trained model 503 read out from the memory circuitry 140, thereby extracting, from the multiple medical images taken in past examinations, difference target images to be subtracted from the medical image of the subject taken in the current examination. The control function 151 then outputs the extracted medical images and information related to the examination of the medical images to the terminal 50 as output data 505. For example, the display control function 447 displays the output data 505 on the display of the terminal 50 together with the medical image of the subject taken in the current examination and information related to the examination of the medical image.

In this way, the medical image processing device 100 uses the learning program 502 to learn from multiple medical images taken in past examinations together with information about the examinations, and generates a trained model 503. As a result, the medical image processing device 100 can input medical images of the subject taken in the current examination and information about the examination of the medical images to the trained model 503, and automatically extract difference target images that are to be used as difference targets with the medical image of the subject taken in the current examination, from among multiple medical images taken in past examinations.

The following describes each function of the control function 151 and the display control function 152 with reference to Figs. 7 to 10. Fig. 7 is a flowchart showing the procedure of processing by the medical image processing device 100 according to this embodiment. Figs. 8 to 10 are diagrams for explaining the processing by the medical image processing device 100 according to this embodiment.

First, the case of extracting one subtraction target image to be used as the subtraction target for the medical image of the subject captured in the current examination will be explained with reference to Figures 7 and 8.

In step S101 of FIG. 7, the control function 151 executes the extraction process.

First, the control function 151 acquires the medical image of the subject taken in the current examination and information related to the examination of the medical image. For example, the control function 151 acquires the medical image of the subject taken in the current examination and information related to the examination of the medical image from the PACS server 40. Then, the control function 151 inputs the medical image of the subject taken in the current examination and information related to the examination of the medical image as input data 504 from the terminal 50 to the trained model 503 read from the memory circuitry 140, thereby extracting one difference target image that is to be the difference target of the medical image of the subject taken in the current examination from multiple medical images taken in past examinations. At this time, the control function 151 outputs the extracted medical image and information related to the examination of the medical image as output data 505.

Next, in step S102 of FIG. 7, the display control function 152 executes the presentation process.

For example, the display control function 152 displays on the display of the terminal 50 a medical image extracted as output data 505, which is a subtraction target image that is a subtraction target of the medical image of the subject captured in the current examination, and information related to the examination of the medical image, together with the medical image of the subject captured in the current examination and information related to the examination of the medical image. For example, the display control function 152 displays a screen 200 shown in FIG. 8 on the display of the terminal 50.

The screen 200 shown in FIG. 8 displays a medical image 230 of the subject captured in the current examination, and a subtraction target image 220 that is the subtraction target of the medical image 230. The screen 200 shown in FIG. 8 also includes display fields 210 and 240, and the display field 240 displays information related to the examination of the subtraction target image 220 and information related to the examination of the medical image 230. The display field 210 displays the examination corresponding to the extracted subtraction target image 220 in an identifiable manner in a schematic diagram showing multiple examinations performed on the subject in chronological order.

For example, if a subject is suspected of having a tumor, the subject is assumed to have been photographed by the medical image diagnostic device 30 at the "initial consultation." Then, after the "preoperative plan" is implemented and the subject is given "radiotherapy," the subject is assumed to have been photographed by the medical image diagnostic device 30 as "follow-up 1" of the first radiation therapy. After the subject is given "radiotherapy" again, the subject is assumed to have been photographed by the medical image diagnostic device 30 as "follow-up 2" of the second radiation therapy. Then, at the "present," the subject is assumed to have been photographed by the medical image diagnostic device 30 at the current examination so that the radiologist can determine whether the subject's tumor has metastasized to the bone. Here, the subtraction target image 220 that is the subtraction target of the medical image 230 is assumed to be the medical image of the subject photographed at the "initial consultation" examination.

In this case, the display control function 152 displays the examination "initial examination" corresponding to the extracted difference target image 220 in the display field 210 in a identifiable manner in a schematic diagram showing a plurality of examinations "initial examination", "preoperative plan", "radiotherapy", "follow-up 1", "radiotherapy", "follow-up 2", and "current" performed on the subject in chronological order. Here, the display control function 152 controls the display form of the examination "initial examination" corresponding to the difference target image 220 and the examination "current" corresponding to the medical image 230 in the display field 210. For example, as a control of the display form, the examination "initial examination" corresponding to the difference target image 220 and the examination "current" corresponding to the medical image 230 are displayed by hatching. Alternatively, as a control of the display form, control such as changing the color or blinking may be performed. In addition, the display control function 152 displays information on the examination "initial examination" of the difference target image 220 and information on the examination "current" of the medical image 230 in the display field 240.

Also, the screen 200 shown in FIG. 8 displays a difference image 250 between the difference target image 220 and the medical image 230. For example, when one difference target image 220 is extracted, the control function 151 reads out a bone subtraction application implemented in the application server 60, and generates a difference image 250 between the difference target image 220 and the medical image 230 using the read out bone subtraction application. The display control function 152 displays the difference image 250 on the screen 200. Here, by referring to the difference image 250, the user can determine the type of bone metastasis, such as whether the area depicted in the difference image 250 is osteolytic metastasis, osteoblastic metastasis, or mixed metastasis. Furthermore, by referring to information on the examination "initial examination" of the difference target image 220 displayed in the display field 240 and information on the examination "current" of the medical image 230, the user can determine whether the comparison between the difference target image 220 and the medical image 230 is appropriate.

In step S103 of FIG. 7, the control function 151 executes the update process.

For example, the screen 200 shown in FIG. 8 includes an acceptance button "Accept" 260 and an acceptance button "Dismiss" 270. The acceptance button "Accept" 260 is operated by the user when the difference image 250 displayed on the screen 200 is used for interpretation. When the acceptance button "Accept" 260 is operated, the control function 151 accepts information indicating that the difference image 250 displayed on the screen 200 is used for interpretation. The acceptance button "Dismiss" 270 is operated by the user when the difference image 250 displayed on the screen 200 is not used for interpretation. When the acceptance button "Dismiss" 270 is operated, the control function 151 accepts information indicating that the difference image 250 displayed on the screen 200 is not used for interpretation. When the control function 151 accepts information indicating whether or not the difference image 250 displayed on the screen 200 is used for interpretation, the control function 151 updates the trained model 503 based on the information. In other words, information indicating whether or not the difference image 250 is to be used for interpretation is fed back to the trained model 503.

In this way, the medical image processing device 100 according to this embodiment inputs the medical image 230 of the subject taken in the current examination and information related to the examination of the medical image 230 to the trained model 503, and automatically extracts the difference target image 220, which is to be the difference target with the medical image 230 of the subject taken in the current examination, from among multiple medical images taken in past examinations, thereby improving the efficiency of interpretation.

Next, the case where multiple subtraction target images are extracted to be subtracted from the medical images of the subject captured in the current examination will be explained using Figures 7, 9, and 10.

First, in step S101 of FIG. 7, the control function 151 executes the extraction process.

For example, the control function 151 obtains the medical image of the subject taken in the current examination and information on the examination of the medical image from the PACS server 40. Then, the control function 151 inputs the medical image of the subject taken in the current examination and information on the examination of the medical image as input data 504 from the terminal 50 to the trained model 503 read from the memory circuitry 140, thereby extracting multiple subtraction target images to be subtracted from the medical image of the subject taken in the current examination from multiple medical images taken in past examinations. Here, the trained model 503 associates accuracy with the subtraction target images to be subtracted from the medical image of the subject taken in the current examination based on the conditions related to the shooting and information on the purpose of the examination, and scores them according to the accuracy. Then, the trained model 503 extracts the subtraction target images to be subtracted from the medical image of the subject taken in the current examination, for example, in descending order of accuracy. In FIG. 9, three medical images are extracted. At this time, the control function 151 outputs the extracted medical images and information related to the examination of the medical images as output data 505.

Next, in step S102 of FIG. 7, the display control function 152 executes the presentation process.

For example, the display control function 152 displays on the display of the terminal 50 a medical image extracted as output data 505, which is a plurality of subtraction target images to be subtracted from the medical image of the subject captured in the current examination, and information related to the examination of the plurality of subtraction target images, together with the medical image of the subject captured in the current examination and information related to the examination of the medical images. For example, the display control function 152 displays a screen 200 shown in FIG. 9 on the display of the terminal 50.

The screen 200 shown in FIG. 9 displays a medical image 230 of the subject captured in the current examination, and a subtraction target image 221 that is the subtraction target of the medical image 230. The screen 200 shown in FIG. 9 also includes display fields 210 and 240, an acceptance button "Accept" 260, an acceptance button "Dismiss" 270, and an acceptance button 280 "Subtraction". The acceptance button 280 "Subtraction" is operated by the user when generating a subtraction image between the subtraction target image 221 and the medical image 230. The operation of the acceptance button 280 "Subtraction" will be described later.

Here, if multiple subtraction target images that are to be subtracted from the medical images of the subject captured in the current examination are extracted, the subtraction images are not displayed on the screen 200 shown in FIG. 9. For example, if subtraction images were to be generated for all extracted medical images, it would take time to execute a bone subtraction application. Therefore, in this embodiment, a subtraction image is generated for a selected subtraction target image from among the multiple extracted subtraction target images. Note that subtraction images may be generated and displayed for all extracted subtraction target images.

For example, in the display field 210, in a schematic diagram showing a plurality of examinations performed on the subject in chronological order, the examinations corresponding to the plurality of extracted difference target images 220 are displayed in an identifiable manner. Here, the plurality of extracted difference target images are assumed to be medical images of the subject taken in the examinations "initial examination", "follow-up 1", and "follow-up 2". In this case, the display control function 152 causes the display field 210 to identifiablely display the examinations "initial examination", "follow-up 1", and "follow-up 2" corresponding to the plurality of extracted difference target images in a schematic diagram showing a plurality of examinations performed on the subject in chronological order: "initial examination", "preoperative planning", "radiotherapy", "follow-up 1", "radiotherapy", "follow-up 2", and "present".

For example, if the medical image of the subject taken in the examination "follow-up 1" is the medical image with the highest accuracy, the display control function 152 displays the medical image of the subject taken in the examination "follow-up 1" on the screen 200 as the subtraction target image 221. In this case, the display control function 152 controls the display form of the examination "follow-up 1" corresponding to the subtraction target image 221 and the examination "current" corresponding to the medical image 230 in the display field 210.

The display control function 152 also displays information about the examination "follow-up 1" of the difference target image 221 and information about the examination "current" of the medical image 230 in the display field 240. The display control function 152 may also display information about the examinations of the multiple extracted difference target images in the display field 240 in descending order of accuracy.

Here, the display control function 152 accepts a display switch for the multiple extracted difference target images. For example, when the user refers to the difference target image 221 displayed on the screen 200, the medical image 230, the information related to the examination in the display field 240, and the schematic diagram in the display field 210 and determines that the medical image of the subject taken in the examination "follow-up 2" is more suitable as a difference target image to be used as a difference target for the medical image of the subject taken in the current examination than the medical image of the subject taken in the examination "follow-up 1", the user switches the display.

Specifically, the display control function 152 accepts the selection of an examination corresponding to one of the extracted subtraction target images that is to be the subtraction target of the medical image of the subject captured in the current examination in the display field 210. For example, the examination "follow-up 2" is selected by the user's operation in the display field 210. At this time, the display control function 152 accepts the selection of the examination "follow-up 2" and displays the screen 200 shown in FIG. 10 on the display of the terminal 50. In this case, the display control function 152 displays the medical image of the subject captured in the examination "follow-up 2" as the subtraction target image 221 on the screen 200, and controls the display form of the examination "follow-up 2" corresponding to the subtraction target image 221 and the examination "current" corresponding to the medical image 230 in the display field 210. In addition, the display control function 152 displays information on the examination "follow-up 2" of the subtraction target image 221 and information on the examination "current" of the medical image 230 in the display field 240.

In step S103 of FIG. 7, the control function 151 executes the update process.

For example, when the examination "follow-up 2" is selected by the user's operation in the display field 210, the control function 151 accepts the selection of the examination "follow-up 2". Upon accepting information representing the selection of the examination "follow-up 2", the control function 151 updates the trained model 503 based on the selected information. In other words, the control function 151 feeds back information representing the user's selection of a medical image to the trained model 503.

Here, the user's selection of medical images is reflected in the trained model 503. For example, if a medical image of the subject is captured in an examination following the current examination, the display control function 152 may display information about the examination of multiple extracted difference target images in the display area 240 in order of the user's tendency to select medical images.

In addition, when the user operates the reception button 280 "Subtraction" after selecting the medical image of the subject taken in the examination "follow-up 2" on the screen 200 shown in FIG. 10, step S102 in FIG. 7 is executed. For example, when the reception button 280 "Subtraction" is operated, the control function 151 reads out the bone subtraction application implemented in the application server 60, and generates a difference image between the subtraction target image 221 and the medical image 230 using the read bone subtraction application. The display control function 152 displays the difference image on the screen 200. Here, the user can determine the type of bone metastasis, such as whether the area depicted in the difference image is osteolytic metastasis, osteoblastic metastasis, or mixed metastasis, by referring to the difference image. Furthermore, the user can determine whether the comparison between the difference target image 221 and the medical image 230 is appropriate by referring to the information about the examination "Follow-up 2" of the difference target image 221 displayed in the display field 240 and the information about the examination "Current" of the medical image 230.

Here, when the acceptance button "Accept" 260 or the acceptance button "Dismiss" 270 is operated, step S103 in FIG. 7 is executed. For example, when the acceptance button "Accept" 260 is operated, the control function 151 accepts information indicating that the difference image displayed on the screen 200 will be used for interpretation. When the acceptance button "Dismiss" 270 is operated, the control function 151 accepts information indicating that the difference image displayed on the screen 200 will not be used for interpretation. When the control function 151 accepts information indicating whether or not the difference image displayed on the screen 200 will be used for interpretation, the control function 151 updates the trained model 503 based on the information. In other words, the information indicating whether or not the difference image will be used for interpretation is fed back to the trained model 503.

In this way, the medical image processing apparatus 100 according to this embodiment inputs the medical image 230 of the subject taken in the current examination and information related to the examination of the medical image 230 to the trained model 503, and automatically extracts multiple difference target images to be used as difference targets for the medical image 230 of the subject taken in the current examination from multiple medical images taken in past examinations, so that the user can select a medical image suitable for interpretation from the multiple extracted difference target images. Therefore, the medical image processing apparatus 100 according to this embodiment can improve the efficiency of interpretation.

Other Embodiments
Although the embodiment has been described above, the present invention may be embodied in various different forms other than the above-described embodiment.

In the above-described embodiment, when the control function 151 uses the trained model 503 to extract multiple subtraction target images to be subtracted from the medical image of the subject captured in the current examination, the user selects a medical image suitable for interpretation from the multiple extracted subtraction target images, but this is not limited to this. For example, the user may select a medical image that has not been extracted as a subtraction target for the medical image 230 of the subject captured in the current examination, as a subtraction target for the medical image 230 of the subject captured in the current examination.

For example, it is assumed that the multiple extracted subtraction target images are medical images of the subject taken in the examinations "follow-up 1" and "follow-up 2", and no medical images of the subject taken in the examination "initial examination" have been extracted. Here, for example, in the display field 210, the examination "initial examination" is selected by the user's operation. At this time, the display control function 152 accepts the selection of the examination "initial examination". In this case, the display control function 152 displays the medical image of the subject taken in the examination "initial examination" on the screen 200 as, for example, the subtraction target image 220 shown in FIG. 8, and controls the display form of the examination "initial examination" corresponding to the subtraction target image 220 and the examination "current" corresponding to the medical image 230 in the display field 210. In addition, the display control function 152 displays information on the examination "initial examination" of the subtraction target image 220 and information on the examination "current" of the medical image 230 in the display field 240.

Furthermore, when the examination "initial examination" is selected by the user's operation in the display field 210, the control function 151 accepts the selection of the examination "initial examination". When the control function 151 accepts information representing the selection of the examination "initial examination", it updates the trained model 503 based on the information. That is, it feeds back information representing the selection of a medical image by the user to the trained model 503. In this way, when a medical image that has not been extracted as a difference target for the medical image 230 of the subject captured in the current examination is selected as a difference target for the medical image 230 of the subject captured in the current examination, the control function 151 updates the trained model 503 based on the selected medical image.

The above-described embodiment is an example of determining whether a tumor in a subject has metastasized to a bone. That is, the medical image is an image of a bone. However, the present invention is not limited to this. For example, the above-described embodiment is applicable to any tissue for which it is easy to align two images and to subtract the two images. As an example, the above-described embodiment is applicable to the brain, which is surrounded by bone. That is, the medical image may be an image of the brain.

For example, when the subject's brain is photographed in the current examination, the control function 151 inputs the medical image of the subject photographed in the current examination and information related to the examination of the medical image to the trained model 503, and extracts a difference target image that is a difference target of the medical image of the subject photographed in the current examination from multiple medical images photographed in past examinations. Even in this case, the display control function 152 displays the extracted medical image and information related to the examination of the medical image on the display of the terminal 50 together with the medical image of the subject photographed in the current examination and information related to the examination of the medical image. Then, the user refers to the difference image between the extracted medical image and the medical image of the subject photographed in the current examination to determine, for example, whether a tumor in an organ has metastasized to the brain.

In the above-described embodiment, the control function 151 uses the trained model 503 to extract a difference target image that is to be the difference target of the medical image of the subject captured in the current examination, but is not limited to this. For example, when the medical image of the subject captured in the current examination and information related to the examination of the medical image are input from the terminal 50 as input parameters, the control function 151 may refer to the PACS server 40 and extract a medical image corresponding to the input parameters from multiple medical images captured in past examinations as a difference target image that is to be the difference target of the medical image of the subject captured in the current examination.

Note that the components of each device illustrated in this embodiment are conceptual and functional, and are not necessarily required to be physically configured as illustrated. In other words, the specific form of distribution and integration of each device is not limited to that illustrated, and all or part of the devices can be functionally or physically distributed and integrated in any unit depending on various loads and usage conditions. Furthermore, all or any part of the processing functions performed by each device can be realized by a CPU and a program analyzed and executed by the CPU, or can be realized as hardware using wired logic.

The method described in this embodiment can be realized by executing a prepared program on a computer such as a personal computer or a workstation. This program can be distributed over a network such as the Internet. This program can also be recorded on a non-transitory computer-readable recording medium such as a hard disk, flexible disk (FD), CD-ROM, MO, or DVD, and executed by being read from the recording medium by a computer.

According to at least one of the embodiments described above, the efficiency of image interpretation can be improved.

Although several embodiments have been described, these embodiments are presented as examples and are not intended to limit the scope of the invention. These embodiments can be implemented in various other forms, and various omissions, substitutions, modifications, and combinations of embodiments can be made without departing from the spirit of the invention. These embodiments and their modifications are within the scope of the invention and its equivalents as set forth in the claims, as well as the scope and spirit of the invention.

100 Medical image processing device 151 Control function 152 Display control function

Claims (9)

an acquisition unit for acquiring medical images of a subject;
a control unit that extracts a subtraction target image to be subtracted from the medical image from among past medical images taken of the subject at multiple different times prior to the medical image, based on first examination information, which is information regarding the examination in which the medical image was taken, and second examination information, which is information regarding the examination in which the past medical image was taken;
a presentation unit that presents the subtraction target image , the medical image , second examination information of the subtraction target image, and the first examination information, and presents the examination corresponding to the subtraction target image in a identifiable manner in a schematic diagram showing a plurality of examinations performed on the subject in chronological order ;
A medical image processing device comprising: an acquisition unit for acquiring medical images of a subject;
a control unit that extracts a subtraction target image to be subtracted from the medical image from among past medical images taken of the subject at multiple different times prior to the medical image, based on first examination information, which is information regarding the examination in which the medical image was taken, and second examination information, which is information regarding the examination in which the past medical image was taken;
a presentation unit that presents the subtraction target image and the medical image together with second examination information of the subtraction target image and the first examination information;
Equipped with
The control unit inputs the medical images and the first examination information as input data to a trained model using a plurality of medical images and information on the examinations in which the plurality of medical images were taken, and outputs the difference target image extracted from the plurality of past medical images taken at different times and second examination information of the difference target image as output data .
Medical imaging equipment. The information regarding the examination includes conditions for taking medical images taken during the examination and information regarding the purpose of the examination;
The photographing conditions include information required for image registration.
The medical image processing device according to claim 2 . The presentation unit causes a difference image between the difference target image and the medical image to be presented,
When the control unit receives information indicating whether the presented difference image is to be used for image interpretation, the control unit updates the trained model based on the information.
The medical image processing device according to claim 2 or 3. when a plurality of difference targets are extracted, the presentation unit presents tests corresponding to the plurality of extracted difference targets in a identifiable manner in a schematic diagram illustrating a plurality of tests performed on the subject in a chronological order;
When the control unit receives, from the schematic diagram, a selection of an examination corresponding to one difference target to be a difference target of the medical image from among the plurality of extracted difference targets, the control unit updates the trained model based on the selected information.
The medical image processing device according to any one of claims 2 to 4. the presentation unit, after the one difference target is selected, receives a request to perform a difference between the one difference target image, which is the medical image selected as the one difference target, and the medical image, and presents a difference image between the one difference target image and the medical image;
When the control unit receives information indicating whether the presented difference image is to be used for image interpretation, the control unit updates the trained model based on the information.
The medical image processing device according to claim 5 . When the past medical image that has not been extracted as the subtraction target image is selected as the subtraction target of the medical image, the control unit updates the trained model based on the selected image.
The medical image processing device according to any one of claims 2 to 6. The medical image is an image of a bone.
The medical image processing device according to any one of claims 1 to 7. The medical image is an image of the brain.
The medical image processing device according to any one of claims 1 to 7.

Images