JP7794430B2 - Computer program, information processing device, and information processing method - Google Patents

Description

The present invention relates to a computer program, an information processing device, and an information processing method.

As the population ages in recent years, the number of dementia patients and those at risk of developing dementia (mild cognitive impairment) is increasing. The three major diseases that cause dementia are Alzheimer's disease (AD), dementia with Lewy bodies (DLB), and vascular dementia. Other diseases, such as multiple sclerosis (MS), also cause a variety of neurological symptoms, including dementia. While the cause of Alzheimer's disease has yet to be elucidated, specific lesions are observed in the brain as the disease progresses. For example, the deposition of senile plaques caused by amyloid beta is known to occur outside of nerve cells. To visualize amyloid beta in the brain, a drug that binds to amyloid beta in brain tissue is injected into the subject, and PET (Positron Emission Tomography) images are used to show the concentration distribution of the drug on cross-sections across the brain.

In order to visually evaluate the distribution of amyloid-β in the brain by interpreting PET images, it is necessary to manually adjust the brightness so that the signal value of a specified reference region becomes the reference point on the color scale of the entire image. To identify the reference region, a technique such as that described in Patent Document 1 can be used, in which a mask image is used to perform segmentation on an MRI image taken of the entire head to obtain an image of the part corresponding to the brain.

JP 2009-247534 A

However, in clinical settings, structural images such as MRI images are sometimes unavailable, and interpretation is often performed using only functional images such as PET images, which can make it difficult to adjust the brightness of functional images.

The present invention was made in consideration of these circumstances, and aims to provide a computer program, information processing device, and information processing method that can adjust brightness using only functional images.

The present application includes multiple means for solving the above problems. One example is a computer program that causes a computer to execute the following process: acquire a functional image by detecting radiation emitted from a radiopharmaceutical; accept a selection of the radiopharmaceutical used in the acquired functional image; and adjust and display the brightness of the functional image according to the selected radiopharmaceutical.

According to the present invention, brightness can be adjusted using only the functional image.

1 is a diagram illustrating an example of a configuration of an information processing system according to an embodiment of the present invention. FIG. 1 illustrates an example of a configuration of an information processing device. FIG. 10 is a diagram illustrating an example of a method for generating a learning model. FIG. 10 is a diagram illustrating an example of brightness adjustment. FIG. 10 is a diagram showing an example of a display screen before brightness adjustment. FIG. 10 is a diagram showing an example of a display screen after brightness adjustment. FIG. 10 is a diagram illustrating an example of a processing procedure performed by an information processing device.

Embodiments of the present invention will now be described with reference to the drawings. Figure 1 shows an example of the configuration of an information processing system according to this embodiment. The information processing system includes an information processing device 50. A client device 40 is connected to the information processing device 50 via a communication network 1. The client device 40 may be configured, for example, as a personal computer, and has an image viewer or the like installed. An image server 30 is connected to the client device 40. The image server 30 receives medical images from a PET (Positron Emission Tomography) device 10 and stores them in a database. The client device 40 acquires medical images from the image server 30 and transmits the acquired medical images to the information processing device 50. The information processing device 50 can perform predetermined processing on the medical images and transmit the processing results to the client device 40. The client device 40 can display the processing results. Details of the predetermined processing will be described later.

The PET device 10 can obtain PET images by intravenously injecting a positron-emitting diagnostic agent and detecting gamma rays emitted when the positrons, released from cells that have taken up the diagnostic agent, are annihilated. PET devices 10 include not only dedicated PET devices, but also devices incorporating X-ray CT (Computed Tomography) devices or SPECT (Single Photon Emission CT) devices. In other words, PET images, PET-CT images, and SPECT images can be obtained from the PET device 10. The PET device 10 is installed, for example, in a medical institution such as a hospital. Various images obtained by the PET device 10 are stored in the image server 30. PET images and SPECT images are also referred to as functional images.

PET and SPECT images are generated by administering a radioactive drug to a subject via intravenous injection or other means and capturing the radiation emitted by the drug within the body. Images using the drug allow physicians to understand not only the morphology of various parts of the body, but also how the administered drug is distributed within the body and the accumulation of substances in the body that react with the drug, thereby contributing to improved accuracy in diagnosing diseases. For example, PET images can be captured using what is commonly known as Pittsburgh compound B as a PET radioactive drug (tracer). Measuring the level of amyloid beta protein accumulation in the brain based on the captured PET images can be useful for differential diagnosis or early diagnosis of Alzheimer's disease. Furthermore, SPECT images can also be used to examine various areas, such as cerebral blood flow tests for cerebrovascular disorders and dementia, depending on the type of drug.

In this specification, medical images include PET images, PET-CT images, SPECT images, CT images, MRI (Magnetic Resonance Imaging) images, etc. Furthermore, medical images include not only medical images of the brain, but also medical images of parts other than the brain.

The image server 30 records medical images (brain function images and brain structure images) for each patient. For example, for each patient, the date the medical image was taken, the imaging conditions, whether or not medication was administered at the time of imaging or the number of times medication was administered, and the name and amount of the therapeutic drug administered are recorded in association with the medical image.

Figure 2 is a diagram showing an example of the configuration of an information processing device 50. The information processing device 50 can be configured as a computer, and includes a control unit 51 that controls the entire information processing device 50, a communication unit 52, a memory 53, a preprocessing unit 54, a display control unit 55, a brightness adjustment unit 56, and a storage unit 57. The storage unit 57 stores a computer program 58 and a learning model 59.

The control unit 51 can be configured with a CPU (Central Processing Unit), MPU (Micro-Processing Unit), GPU (Graphics Processing Unit), etc. The control unit 51 can execute processing defined by the computer program 58. In other words, the processing by the control unit 51 is also processing by the computer program 58.

The communication unit 52, for example, includes a communication module and has the ability to communicate with the client device 40 via the communication network 1. The communication unit 52 can acquire (receive) medical images (e.g., functional images of a patient) from the client device 40.

Memory 53 can be composed of semiconductor memory such as SRAM (Static Random Access Memory), DRAM (Dynamic Random Access Memory), or flash memory. Computer program 58 can be loaded into memory 53, and control unit 51 can execute computer program 58.

The memory unit 57 may be configured, for example, as a hard disk or semiconductor memory, and may store necessary information in addition to the computer program 58 and learning model 59.

The pre-processing unit 54 aligns the functional image (e.g., PET image or SPECT image) acquired via the communication unit 52 with the template image. Specifically, the pre-processing unit 54 performs rigid body transformation (a transformation that allows only translation and rotation of coordinates) using a standard brain image (template image) prepared in advance, and performs processing to align the tilt and position of the brain in the functional image. This improves the accuracy of the segmentation processing by the learning model 59. It also makes it possible to automatically adjust the tilt of the functional image in accordance with interpretation guidelines.

The control unit 51 inputs the functional images aligned by the preprocessing unit 54 into the learning model 59.

The learning model 59 is generated to output a specific reference region when a functional image is input. The reference region is a region on the functional image used to adjust the brightness of the entire functional image, and includes, for example, the cerebellum, pons, and the whole brain (regions excluding non-brain tissue such as salivary glands and bones in the PET image). Specific reference regions are defined in interpretation handbooks and guidelines according to the type of radiopharmaceutical used in the PET examination. The control unit 51 can input a functional image into the learning model 59 to obtain a specific reference region on the functional image. A separate learning model 59 may be used for each specific reference region. For example, when a functional image is input, a learning model 59 that outputs the cerebellum region and a learning model 59 that outputs the pons region may be separate models. The learning model 59 may use semantic segmentation such as U-Net or SegNet, or may use instance segmentation such as Mask R-CNN, DeepMask, or FCIS.

Figure 3 shows an example of a method for generating a learning model 59. A required number of mask images with identified reference regions are prepared in advance as training data for generating the learning model 59. Mask images can be generated by performing segmentation processing on structural images such as MRI images.

The control unit 51 acquires training data including functional images as learning input data and mask images as teacher data. The training data may be collected and stored, for example, in the image server 30 or another data server (not shown), and acquired from that server. When a functional image is input based on the training data, the control unit 51 generates a learning model 59 so that it outputs a reference region identified by a mask image included in the training data. Specifically, the parameters of the learning model 59 may be adjusted so that the value of a loss function based on the output data (reference region) output by the learning model 59 and the teacher data (reference region on the mask image) is minimized.

The learning model 59 may be generated by the information processing device 50, or by a device separate from the information processing device 50. By using the learning model 59, it is possible to segment the reference region using only functional images, even when structural images are not available.

The brightness adjustment unit 56 adjusts the brightness of the functional image based on the signal value of a specific reference region on the functional image.

The display control unit 55 can display the functional image, the brightness of which has been adjusted by the brightness adjustment unit 56, on the display screen of the client device 40. The display control unit 55 can perform display control to display required information on the display screen of the client device 40.

Figure 4 shows an example of brightness adjustment. In functional images such as PET images, a color scale is defined for each radiopharmaceutical to achieve optimal imaging for visual interpretation of the amyloid-β accumulation state. Color scales include, for example, the Sokoloff scale, rainbow scale, and spectral scale. The color scale may also be a grayscale. As shown in Figure 4, information indicating the correspondence between drugs (radiopharmaceuticals) and color scales can be stored, for example, in memory unit 57. In the example of Figure 4, drugs are represented by symbols D1, D2, D3, etc., and the corresponding color scales are represented by CS1, CS2, CS3, etc.

Before adjusting the brightness, the control unit 51 receives a drug selection from the client device 40. The drug selection is intended to identify the drug used in the functional image. In the example of FIG. 4, drug D2 is selected, and color scale CS2 is used. The brightness adjustment unit 56 calculates the statistical value of the signal values of the reference region output by the learning model 59. The statistical value of the signal values may be the mean, median, or percentile value. The brightness adjustment unit 56 adjusts the brightness of the functional image based on a predetermined value of the color scale CS2 and the statistical value of the signal values of the reference region. Specifically, for example, the minimum brightness value can be set to 0, and the maximum brightness value can be calculated so that the statistical value of the signal values is equal to the predetermined value of the color scale CS2. If the predetermined value is 0.9 (90%) of the maximum value, the maximum brightness value can be calculated as follows: Maximum value = (Statistical value of the signal values of the reference region) / 0.9. Note that the method for calculating the maximum brightness is not limited to this; it can be calculated according to the requirements of the guidelines corresponding to the drug used.

Next, we will explain an example of what will appear on the display screen of the client device 40.

Figure 5 shows an example of the display screen 100 before brightness adjustment. As shown in Figure 5, the display screen displays attribute information related to the patient, such as the patient ID (which may include the name), date of birth, gender, date of medical image capture (date of capture at the time of medication), medication, and medication history. The patient ID may be selectable from among multiple patients. Furthermore, if there are multiple imaging dates, the imaging date may be selectable.

The image area 101 can display the desired slice images from multiple tomographic images (for example, brain functional images such as PET images) in the form of sagittal, coronal, and axial slices. The functional images displayed in the image area 101 are before brightness adjustment and use the minimum and maximum values for the entire image, which are the default values for many image viewers. This may result in the degree of accumulation of amyloid beta in the brain becoming unclear.

PET images include, for example, distribution information (e.g., SUVR values for each voxel) that visualizes the distribution of amyloid beta in the brain. SUVR (Standardized Uptake Value Ratio) can be calculated by summing the SUV (Standardized Uptake Value: amyloid beta protein accumulation level) of four regions of the cerebral gray matter (prefrontal cortex, anterior and posterior cingulate cortex, parietal lobe, and lateral temporal lobe) and dividing it by the SUV of a specific reference region (e.g., the cerebellum). The accumulation level (SUV) of amyloid beta can be calculated, for example, by counting the number of voxels whose brightness values for each voxel constituting a given region are equal to or greater than a predetermined threshold. Accumulation (e.g., 00%) can be calculated based on the ratio of the count value to the total number of voxels in the given region.

A user such as a doctor can select the drug used in the functional image in the drug selection field 102 to adjust the brightness of the functional image. This allows the control unit 51 of the information processing device 50 to accept the selection of the radioactive drug used in the acquired functional image. The selected drug can be deselected by operating the "deselect" icon 103.

In interpretation handbooks and guidelines, specific reference regions are defined according to the type of radiopharmaceutical used in PET scans, so the reference region is automatically selected when the drug is selected in the drug selection field 102. However, users such as doctors can also select the desired reference region by operating the "reference region selection" icon 104. Specifically, they can specify the region they want to identify as the reference region in any of the sagittal, coronal, or axial slice images and operate the "reference region selection" icon 104. Alternatively, they can perform a specific operation on the image to select the reference region. The selected reference region can be deselected by operating the "deselect" icon 105.

The control unit 51 (display control unit 55) can adjust the brightness of the functional image and display it according to the selected radiopharmaceutical. The control unit 51 (display control unit 55) may also accept the selection of a reference region on the functional image, and adjust the brightness of the functional image and display it based on the signal value of the selected reference region.

Figure 6 shows an example of the display on the display screen 110 after brightness adjustment. In the image area 101, functional images with adjusted brightness are displayed in the form of sagittal, coronal, and axial slices. By adjusting the brightness, the degree of accumulation of amyloid beta in the brain becomes clearer, enabling accurate interpretation of the intrusion into white matter and gray matter within and between cases.

Information about the reference region can be displayed in the reference region information field 111. The control unit 51 (display control unit 55) displays information about the identified reference region. For example, if the pons (Pons) region of the brain is used as the reference region to adjust the brightness of the functional image, information such as "The brightness of the pons (Pons) has been adjusted so that it has the highest brightness of 00%" may be displayed.

By operating the "Reference Area Selection" icon 104, the desired reference area can be selected, the brightness can be adjusted based on the selected reference area, and the display of the functional image can be updated. By operating the "Predetermined Reference Area" icon 112, a default reference area (e.g., a reference area determined by a drug, a reference area predetermined by the user, etc.) can be selected, the brightness can be adjusted based on the default reference area, and the display of the functional image can be updated.

Figure 7 is a diagram showing an example of a processing procedure performed by the information processing device 50. For convenience, the following description will be given with the control unit 51 as the main actor in the processing. The control unit 51 acquires a functional image (S11) and aligns the acquired functional image with a template image (S12). The control unit 51 accepts the selection of the radiopharmaceutical used in the functional image (S13), and identifies a reference region based on the acquired functional image (S14).

The control unit 51 identifies a color scale corresponding to the selected radiopharmaceutical (S15) and calculates minimum and maximum brightness values based on the statistical values of the signal values in the reference region and the predetermined values of the identified color scale (S16). The control unit 51 adjusts the brightness of the functional image based on the calculated minimum and maximum brightness values (S17), and then ends the process.

According to this embodiment, by performing reference region segmentation using only functional images such as PET images, the preprocessing of functional images required to start interpretation, including brightness adjustment, which was previously performed manually, can be performed fully automatically and seamlessly in accordance with interpretation guidelines. Brightness can now be adjusted automatically, eliminating the need for doctors and other professionals to make manual adjustments.

The computer program of this embodiment causes a computer to execute the following processes: acquire a functional image by detecting radiation emitted from a radiopharmaceutical; accept a selection of the radiopharmaceutical used in the acquired functional image; and adjust and display the brightness of the functional image according to the selected radiopharmaceutical.

The computer program of this embodiment causes a computer to execute processing to adjust and display the brightness of an acquired functional image based on the signal value of a specific reference region on the functional image.

The computer program of this embodiment causes the computer to execute processing to adjust and display the brightness of the functional image based on a predetermined color scale or grayscale value for the selected radiopharmaceutical and the signal value.

The computer program of this embodiment causes a computer to execute a process in which, when a functional image is input, the acquired functional image is input into a learning model that outputs a specific reference area, and a specific reference area on the functional image is acquired.

The computer program of this embodiment causes a computer to acquire training data including specific reference regions on functional images and structural images, and, based on the training data, generate the learning model so that, when the functional image is input, the specific reference region included in the training data is output.

The computer program of this embodiment causes the computer to execute processing to display information about the specific reference region.

The computer program of this embodiment causes the computer to execute a process that accepts the selection of a reference area on an acquired functional image, and adjusts and displays the brightness of the functional image based on the signal value of the selected reference area.

The information processing device of this embodiment includes an acquisition unit that detects radiation emitted from a radiopharmaceutical and acquires an imaged functional image, a reception unit that receives the selection of the radiopharmaceutical used in the acquired functional image, and a display unit that adjusts and displays the brightness of the functional image according to the selected radiopharmaceutical.

The information processing method of this embodiment detects radiation emitted from a radiopharmaceutical, acquires a functional image, accepts a selection of the radiopharmaceutical used in the acquired functional image, and adjusts and displays the brightness of the functional image according to the selected radiopharmaceutical.

REFERENCE SIGNS LIST 1 Communication network 10 PET device 30 Image server 40 Client device 50 Information processing device 51 Control unit 52 Communication unit 53 Memory 54 Preprocessing unit 55 Display control unit 56 Brightness adjustment unit 57 Storage unit 58 Computer program 59 Learning model

Claims (7)

On the computer,
A functional image is obtained by detecting radiation emitted from the radioactive agent;
Accepting the selection of the radiopharmaceutical ;
adjusting and displaying the brightness of the functional image based on the selected radiopharmaceutical and the signal value of a specific reference region on the acquired functional image ;
A computer program that executes a process. On the computer,
adjusting and displaying the brightness of the functional image based on a predetermined value of a predetermined color scale or a predetermined value of a predetermined gray scale for the selected radiopharmaceutical and the signal value;
The computer program product of claim 1 , which executes a process. On the computer,
When a functional image is input, the acquired functional image is input to a learning model that outputs a specific reference region, and a specific reference region on the functional image is acquired.
3. The computer program according to claim 1 , which executes a process. On the computer,
obtaining training data including functional images and specific reference regions on the functional images;
generating the learning model based on the training data so that, when the functional image is input, a specific reference region included in the training data is output;
4. A computer program according to claim 3 , which causes a process to be executed. On the computer,
Displaying information about the specific reference region;
5. A computer program product according to claim 1 , which causes a process to be executed. an acquisition unit that detects radiation emitted from the radiopharmaceutical and acquires a functional image;
a reception unit that receives the selection of the radiopharmaceutical ;
a display unit that adjusts and displays the brightness of the functional image based on the selected radiopharmaceutical and the signal value of a specific reference region on the acquired functional image .
Information processing device. A functional image is obtained by detecting radiation emitted from the radioactive agent;
Accepting the selection of the radiopharmaceutical ;
adjusting and displaying the brightness of the functional image based on the selected radiopharmaceutical and the signal value of a specific reference region on the acquired functional image ;
An information processing method that causes a computer to execute a process .