JP7764037B2 - System, control method and program - Google Patents

Description

The present invention relates to a system that provides information to assist in image diagnosis.

The image processing device disclosed in Document 1 comprises an input unit that inputs functional images of the subject's brain; an anatomical standardization unit that anatomically standardizes the functional images of the subject; an ROI candidate presentation unit that reads data on anatomical regions from a standard brain data storage unit that stores data on anatomical regions assigned to a standard brain and presents the data on the anatomical regions as ROI candidates; an ROI setting unit that accepts the selection of an anatomical region and sets an ROI on the anatomically standardized brain image of the subject based on the selected anatomical region or regions; an evaluation value calculation unit that calculates an evaluation value based on pixel values within the ROI; and a display unit that displays information related to the calculated evaluation value.

Japanese Patent Application Laid-Open No. 2019-074343

It is hoped that the use of deep learning in diagnostic assistance using medical images will improve the accuracy of differentiation. However, how the differentiation results and evaluations obtained using deep learning are calculated is a black box, making it difficult for medical professionals who handle the results to interpret and explain them.

One aspect of the present invention is a system comprising: a first access unit accessible to a first image evaluation system that statistically evaluates a first type of medical image that includes at least a portion of a subject's body as a target region; a second access unit accessible to a second image evaluation system that determines the subject's disease state using a first model trained through machine learning to evaluate a first disease based on the first type of medical image; and a support unit that provides, via a common image evaluation environment, at least one of input of the evaluation target and output of the evaluation results to the first and second image evaluation systems. This system enables evaluation of the machine-learned first model and evaluation of the statistical processing via a common environment, thereby providing support to medical professionals and others in interpreting the machine-learned discrimination results of medical images. The support unit may use a common image evaluation environment in which the subject's first type of medical image includes evaluation images standardized to images that can be input and output to the first and second image evaluation systems.

Another aspect of the present invention is a control method for an assistance system having the first access unit, the second access unit, and an assistance unit, the method including at least one of the following steps:
i) The support unit outputs a first result regarding the evaluation of the medical image obtained from the first image evaluation system and a second result regarding the evaluation of the medical image of the subject obtained from the second image evaluation system via a common image evaluation environment.
ii) The support unit outputs the results of re-evaluating the first results regarding the evaluation of the medical images obtained from the first image evaluation system based on the disease state judgment of the second image evaluation system via a common image evaluation environment.
iii) The support unit outputs, via a common image evaluation environment, a first area that is considered important based on a first result regarding the evaluation of the medical image obtained from the first image evaluation system and a second area that is considered important for determining the disease state of the second image evaluation system.
iv) The support unit selects an image area including a first area that is considered important by a first result regarding the evaluation of the medical image acquired from the first image evaluation system as an evaluation target for the second image evaluation system via a common image evaluation environment.
v) The support unit selects an image area including a second area that is considered important for determining the disease state of the second image evaluation system as an evaluation target for the first image evaluation system via a common image evaluation environment.
vi) The support unit controls the output of a first result regarding the evaluation of the medical image acquired from the first image evaluation system or a second result regarding the evaluation of the medical image of the subject acquired from the second image evaluation system using the common image evaluation environment based on the reliability of the mapping of the medical image of the subject to the common image evaluation environment.

Another aspect of the present invention is a program for evaluating medical images by a computer. The program (program product) includes instructions for the computer to access a first image evaluation system that statistically evaluates a first type of medical image that includes at least a portion of a subject's body as a target region; access a second image evaluation system that determines the subject's disease state using a first model trained by machine learning to evaluate a first disease based on the first type of medical image; provide at least one of input of the evaluation target and output of the evaluation result to the first image evaluation system and the second image evaluation system via a common image evaluation environment; and execute at least one of steps i) to vi) above. The program may be provided recorded on a computer-readable recording medium.

FIG. 1 is a diagram showing an overview of an information provision system for supporting image diagnosis. FIG. 1 is a diagram showing an overview of the processing of an evaluation support system. Diagram showing an overview of the anatomical standardization process. A diagram showing the standardization process. FIG. 10 is a diagram showing an example of a region of interest when an anatomically standardized image is used as input for deep learning. 1A-1C show different examples of anatomical standardization processing. FIG. 10 is a diagram showing an example of input processing. FIG. 10 is a diagram showing an example of input processing involving part extraction. FIG. 10 is a diagram showing the attributes of subjects in an experimental example. FIG. 10 is a diagram showing a pulse sequence of an MRI scanner in an experimental example. FIG. 10 shows an example of the results of ROIs subjected to ANOVA analysis. FIG. 1 shows the ROIs of each group. Graph showing the progression of atrophy in each group. Diagram showing a deep learning model. A graph showing changes in sensitivity and specificity. FIG. 10 shows the reassessment of validation subjects. FIG. 1 shows the ROI of a deep learning model. FIG. 10 is a diagram showing ROIs obtained by statistical processing. A diagram showing the ROI of the deep learning model and the ROI of the statistical processing superimposed.

Figure 1 shows an overview of a system 1 that provides information for supporting image diagnosis. This system 1 provides information related to supporting diagnosis using medical images that include the brain or a portion of the brain as a target region for image diagnosis or evaluation as part of the subject's body. System 1 includes an image database 52 that stores brain images 53 of the subject as a first type of medical image that includes at least a portion of the subject's body as a target region; a first image evaluation system 60 that statistically evaluates the brain images 53; a second image evaluation system 70 that determines the subject's disease state using a first model that has been machine-learned to evaluate a first disease based on the brain images 53; and an image evaluation support system 10. The image evaluation support system (support system) 10 has a first access unit (interface) 11 accessible to a first image evaluation system 60, a second access unit (interface) 12 accessible to a second image evaluation system 70, and a support unit 30 that provides at least one of inputs 62 and 72 of evaluation objects and outputs 63 and 73 of evaluation results to the first image evaluation system 60 and the second image evaluation system 70 via a common image evaluation environment.

Various types of tomography devices (modalities), such as CT (Computed Tomography), MRI (Magnetic Resonance Imaging), PET (Positron Emission Tomography), SPECT (Single Photon Emission Computed Tomography), and PET-CT, are known as devices for diagnosing the form and function of a subject's (test subject, examinee's) body or parts thereof, and these modality images (medical images) are used to diagnose a variety of diseases. In particular, modality images (medical images) 53 that include the subject's brain as the target region for diagnosis or evaluation are used to obtain data related to the physical state of the subject's brain and are used to diagnose diseases such as dementia and Parkinson's disease.

CT and MRI are examples of types of medical images, which can accurately reflect morphological information. MRI images include, for example, T1-weighted images, T2-weighted images, diffusion-weighted images, flare images, diffusion tensor images, QSM images, pseudo-PET images, and pseudo-SPECT images. QSM images (Quantitative Susceptibility Mapping) stand for quantitative susceptibility mapping. Other examples of types of medical images are PET and SPECT, which are generated by administering a radioactive drug to a subject via intravenous injection or other means and capturing the radiation emitted from the drug within the body. Drug-based images allow physicians to understand not only the morphology of various parts of the body, but also how the administered drug is distributed within the body and how substances react with the drug accumulate within the body, thereby contributing to improved accuracy in disease diagnosis. For example, a PET image can be captured using what is commonly known as Pittsburgh compound B as a PET radiopharmaceutical (tracer), and the degree of accumulation of amyloid beta protein in the brain can be measured based on the captured PET image, which can be useful for differential diagnosis or early diagnosis of Alzheimer's dementia. The pseudo PET image is a term used to distinguish it from an actual PET image, and is an image that estimates the actual PET image. The pseudo PET image may be generated based on, for example, an MRI image. Similarly, the pseudo SPECT image is an image that estimates the actual SPECT image.

One example of SPECT imaging is a SPECT scan using the radiopharmaceutical 123I-ioflupane, called DatSCAN (Dopamine transporter SCAN), which visualizes the distribution of dopamine transporters (DAT). The purposes of this imaging include early diagnosis of Parkinson's syndrome (PS) in Parkinson's disease (PD), aiding in the diagnosis of dementia with Lewy bodies (DLB), and determining the appropriate medication for treatment in cases of striatal dopaminergic neuronal loss, known as levodopa.

The system 1 includes a mapping system 55 that standardizes a first type of medical image 53 of the subject into an image (standardized image) 50 that can be input and output to a first image evaluation system 60 and a second image evaluation system 70. The support system 10 may include a third access unit (interface) 13 that can access the mapping system 55. An example of the standardized image 50 is an anatomical standardized image, and the mapping system 55 may function as an anatomical standardization processing unit. The mapping system 55 outputs a third result 56 regarding the evaluation of the standardization processing of the brain image 53, including the reliability of the anatomical standardization processing at each voxel. The support unit 30 may acquire the result 56 via the access unit 13. As described below, the support unit 30 includes a common image evaluation environment that provides an evaluation of the image based on the standardized image 50. The evaluation during mapping or the third result 56 regarding the evaluation can be output in the same environment as the mapping system 55 or used as input for processing in the support unit 30.

The assistance system 10, including the assistance unit 30, may provide users, such as medical professionals, with information for evaluating images via a common image evaluation environment (common evaluation environment, user interface module, U/I module) 15, including standardized evaluation images. Users may access the assistance system 10 using access devices such as a display 16a and touch panel 16b attached to the assistance system 10, or may access the assistance system 10 via the cloud (Internet) 17.

This system for providing information for diagnostic imaging support (diagnostic imaging support information provision system) 1 may be provided in a standalone configuration by a device (system) equipped with computer resources including memory and a CPU, such as a server, or the configuration area 8 including the image database 52, first image evaluation system 60, and second image evaluation system 70 may be provided via the cloud (Internet). Furthermore, the configuration area 9 including the mapping system 55, storage of standardized images 50, input 62 and evaluation output 63 of the first image evaluation system 60, and input 72 and evaluation output 73 of the second image evaluation system 70 may be provided via the cloud.

The first image evaluation system 60 for statistically evaluating medical images may be provided as a system equipped with computer resources and may include a processor 61 for performing statistical processing and a database 65 storing libraries and programs for executing the statistical processing. The first image evaluation system 60 may output a first result 66 regarding the evaluation of the medical image to be processed, i.e., the statistical evaluation. The evaluation result 66 may be output (displayed) based on the standardized image 50 by an evaluation output function (display unit) 63 of the evaluation system 60. The support unit 30 may acquire the result 66 via the access unit 11. The support unit 30 may use a common image evaluation environment (common evaluation environment) 15 that provides an evaluation of the image based on the standardized image 50 to output the evaluation of the statistical processing or the first result 66 regarding the evaluation in the same environment as the first image evaluation system 60, or use the result 66 as input for processing in the support unit 30. The first result 66 may include information about a first region (region of interest, ROI) that is considered important in the statistical processing.

The first image evaluation system 60 may have an input function (selection unit) 62 for selecting an image or an area within an image to be subjected to statistical processing based on the standardized image 50. Control of the analysis target (input control information) 67, including the selection of the area to be subjected to statistical processing, may be provided by the support unit 30. The support unit 30 can use the common evaluation environment 15 to input or select information to be subjected to statistical processing based on the standardized image 50 via the same environment as the first image evaluation system 60. Processing such as applying correction for covariates such as age or gender (or various biomarker values) as further input to each voxel data of the standardized processed image 50 may also be performed via the common evaluation environment 15.

The second image evaluation system 70 for assessing disease status based on medical images may be provided as a system equipped with computer resources, and may include a processor 71 for processing using a learning model, and a database 75 storing a first model 74 trained by machine learning to assess disease status based on medical images, as well as a library. The second image evaluation system 70 may output a second result 76 regarding the evaluation of the medical image to be processed, i.e., the assessment of the subject's disease status using the first model 74 trained by machine learning to assess a first disease based on medical images. The evaluation result 76 may be output (displayed) based on the standardized image 50 by the evaluation output function (display unit) 73 of the evaluation system 70. The support unit 30 may acquire the result 76 via the access unit 12. The support unit 30 may use the common evaluation environment 15 to output the evaluation regarding the assessment of disease status or the second result 76 regarding the evaluation based on the standardized image 50 via the same environment as the second image evaluation system 70, or use it as input for processing in the support unit 30.

When dementia is the target, a second image evaluation system 70 can be employed that uses a learning model 74 that has been machine-learned to differentiate between the causative diseases AD (Alzheimer's Disease) and DLB (Dementia with Lewy Bodies). The learning model 74 can be used to output a second result 76 regarding the subject's disease status. The second result 76 may include information on the presence or absence and progression of the first disease of interest, e.g., AD or DLB, as well as information on a second region (ROI) that is considered important in determining the disease status. Using a deep learning method such as GradCAM, it is possible to visualize the region of interest (ROI) used in deep learning differentiation. Using standardized images 50 via a common evaluation environment 15, the support unit 30 can evaluate the ROI in a shared environment with the second evaluation system 70.

The second image evaluation system 70 may be equipped with an input function (selection unit) 72 that selects an image or an area within the image to be subjected to disease state assessment by the learning model 74 based on the standardized image 50. Control (input control information) 77 of the diagnostic target (differentiation target), including the selection of the area to be processed, may be provided by the support unit 30. The support unit 30 can use the common evaluation environment 15 to input or select information to be subjected to the differentiation process of the learning model 74 based on the standardized image 50 in the same environment as the second image evaluation system 70. In this case, the common evaluation environment 15 may also be used to perform processing such as applying correction for covariates such as age or gender (or various biomarker values) as further input to each voxel data of the standardized processed image 50.

The assistance system 10 may be provided as a device equipped with computer resources such as a server accessible to the cloud, and may include a database 18 storing libraries required for various processes and a program 19 containing instructions for executing processes as the assistance system. Services to users using the common evaluation environment 15 may be provided as a cloud-based service (Software as a Service (SaaS)). As described above, the assistance system 10 provides the common evaluation environment 15 using the standardized processed image 50, and the assistance unit 30 can use the common evaluation environment 15 to seamlessly provide the user with the evaluation results 66 from the first image evaluation system 60 and the evaluation results 76 from the second image evaluation system 70 in a cross-referenceable state. The standardization results 56 can also be provided to the user.

Deep learning is expected to improve the accuracy of diagnostic support using medical images. For neurodegenerative diseases such as Alzheimer's disease, a method is used to calculate regions of interest (ROIs) for each imaging modality (e.g., morphological MRI, SPECT perfusion, PET images) that statistically show significant differences in brain volume, blood flow (glucose metabolism), and the amount of disease-causing substances such as amyloid beta between healthy and diseased groups, and then evaluate values such as Z-scores and SUV within those regions. Many methods using deep learning for neurodegenerative diseases have been reported to improve diagnostic accuracy, but the way in which the diagnostic results and evaluations are calculated remains a black box, making them difficult for medical professionals to interpret and explain. There are two main reasons for this: 1) Neurodegenerative diseases require identifying which parts of the brain are altered.
(For example, Alzheimer's disease may be suspected due to atrophy of the parahippocampal gyrus.)
2) The output values of deep learning (including intermediate layers) are processed into information that cannot be interpreted by humans, such as brain volume or blood flow.

The support system 10 is capable of solving the two problems mentioned above using the results obtained using deep learning, and can provide interpretations and explanations of the results to medical professionals who handle the results of deep learning.

Regarding issue 1) above, using a deep learning method such as GradCAM makes it possible to visualize the regions of interest used in deep learning discrimination. Furthermore, if the input for deep learning discrimination is a brain image mapped to an anatomical standard brain 50 provided in the common image environment 15 using the support system 10, it is possible to visually compare it with the statistically calculated ROI on the same image (by switching the display or overlaying).

Regarding issue 2) above, it is possible to specify the area of interest for deep learning using the common image environment 15, perform statistical processing within that area, calculate values for brain volume and blood flow, and present human-interpretable index values for areas that are effective for differentiation.

This support system 10 can also serve as a tool function (system-wise) that can be considered for other business purposes. For example, it can be applied to responses based on analysis results, allowing for the selection of intervention methods, and if possible, the selection of medication prescriptions and guidance on additional testing. In the case of DLB, it can provide support such as recommending DatSCAN/MIBG myocardial scintigraphy tests and recommending testing institutions. By linking to a research paper database, it is also possible to provide support for research, diagnosis, and treatment by displaying links to related papers when the mouse is hovered over the ROI area displayed in the common image environment 15.

In the support system 10, the support unit 30 can provide several functions using the common image environment 15. One function is the input support function (input support unit) 37, which can provide the following functions by supplying input control information 67 and 77 to the first image evaluation system 60 and/or the second image evaluation control system 70 via the support unit 30:

After the brain image 53 is subjected to anatomical standardization processing in the mapping system 55, the anatomically standardized image 50 is input to the deep learning model 74 of the second image evaluation system 70, and the second image evaluation system 70 is controlled to determine whether the image is a brain-related disease (assuming a disease that can be diagnosed from brain images), or to classify the type, progression, etc.

Each voxel data of the anatomically standardized processed image 50 is further corrected for covariates such as age or gender (or various biomarker values) as input, and then selected as input for the deep learning model 74.

When predicting the class of the deep learning model 74, the model is given the option to display a value between 0 and 1 for each class ultimately evaluated by the softmax function. Classes can be classified according to disease; for example, in the case of dementia, a type of brain disease, classes such as NC (Normal Control), AD, and DLB can be envisioned.

In the input support unit 37, if the first disease to be differentiated includes Alzheimer's disease (AD) and dementia with Lewy bodies (DLB), and if the first type of medical image 53 is an MR image, the input control information 67 and 77 may be set to include at least one of the hippocampus, parahippocampal gyrus, dorsal brainstem, middle temporal pole, and basal ganglia (putamen, caudate nucleus, entorhinal cortex, parahippocampal gyrus, amygdala, etc.) as the target region. If the first type of medical image 53 is a SPECT image, the input control information 67 and 77 may be set to include at least one of the precuneus, occipital lobe, and dorsolateral prefrontal cortex of the brain as the target region.

The support unit 30 may include a function (individual/comparative evaluation unit) 31 that outputs a first result 66 regarding the evaluation of a medical image obtained from a first image evaluation system 60 and a second result 76 regarding the evaluation of a medical image of a subject obtained from a second image evaluation system 70 via a common image evaluation environment 15. Using the common image environment 15, the evaluation (first result) 66 obtained by statistically processing the medical image and the evaluation (second result) 76 predicted from the medical image by a deep learning model 74 may be output individually or in a comparable state, for example, in parallel or in a switched manner, via a standardized image 50.

The support unit 30 may include a function (re-statistical processing request unit) 32 that outputs the results of re-evaluating the first results 66 regarding the evaluation of the medical image acquired from the first image evaluation system 60 based on the disease state determination by the second image evaluation system 70 via the common image evaluation environment 15. Even if the results of the preceding statistical processing do not reveal a region of interest (ROI) related to disease, it can be re-recognized as a disease ROI by re-evaluating the subtle differences in the statistical processing of the image of the subject determined to have the disease based on the disease state determination by the deep learning model 74.

The support unit 30 may include a function (overlay unit) 33 that outputs a first region of interest (ROI) that is considered important in the first result 66 regarding the evaluation of the medical image obtained from the first image evaluation system 60 and a second region of interest (ROI) that is considered important for the disease state determination by the second image evaluation system 70, overlaid on the common image evaluation environment 15. Information regarding the ROI may be acquired as part of the information 66 and 76 regarding the evaluation results from each system 60 and 70. When the deep learning model 74 predicts the disease class, the region that the model 74 considered important in the evaluation can be displayed on the anatomically standardized brain image 50. By overlaying the ROI that the deep learning model 74 focused on for the disease state determination and the ROI used in statistical processing on the standardized image 50, interpretations and explanations of the results of the deep learning model 74 can be provided to medical professionals handling the results.

The support unit 30 may include a function (model input selection unit) 34 for selecting an image region including a first region of interest (ROI) that is considered important by a first result 66 regarding the evaluation of the medical image acquired from the first image evaluation system 60 as an evaluation target for the second image evaluation system 70 via the common image evaluation environment 15. One or more anatomical regions may be further input as regions of interest (ROI) using the standardized image 50, and an image of the anatomically standardized image 50 filtered by the anatomical region may be selected as input to the deep learning model 74. When the anatomically standardized image 50 is used, anatomical regions are defined on the standardized brain coordinate system, and a "region of interest," which is a set of coordinates with the same name, can be selected for the first image evaluation system 60 and the second image evaluation system 70.

The support unit 30 may include a function (statistical processing input selection unit) 35 that selects an image area including a second region (ROI) that is considered important for the second image evaluation system 70's assessment of the disease state as an evaluation target for the first image evaluation system 60 via the common image evaluation environment 15. By having the deep learning 74 select an ROI that focuses on assessing the disease state and perform statistical processing, interpretations and explanations of the results of the deep learning 74 can be provided to medical professionals handling the results.

The support unit 30 may include a function (mapping evaluation unit) 36 that controls the output of a first result 66 regarding the evaluation of a medical image acquired from a first image evaluation system 60 or a second result 76 regarding the evaluation of a medical image of a subject acquired from a second image evaluation system 70 to a standardized image 50 using the common image evaluation environment 15, based on the reliability of mapping the medical image of the subject to a common image evaluation environment, in this example, a standardized image 50. When a deep learning model 74 using an anatomically standardized processed image 50 as input predicts a disease class, the model 74 can display the results of reliability correction of each voxel value of the region of interest (ROI) that the model 74 used in the evaluation. Problems with the accuracy of mapping to the standard brain image 50 may affect the reliability of analysis results using regions with low mapping accuracy as ROIs. By quantifying the mapping accuracy before analyzing the brain image (extracting ROIs and performing differentiation), an ROI filter can be applied to obtain an evaluation that does not use image regions with low reliability.

The assistance system 10 may further include a unit 20 that verifies the assessment of a first disease, e.g., AD or DLB, by the learning model (first model) 74 based on the output of these assessment results using the common image environment 15 of the assistance unit 30.

Figure 2 shows a flowchart of an evaluation support method using the image evaluation support system 10. The evaluation support system 10 can be provided as an information processing device equipped with computer resources including memory and a CPU, and this support method can be provided as a control method for the system 10 or as a program having instructions executable on a computer. The program (program product) may be provided by being recorded on a computer-readable recording medium, or may be provided in a downloadable form from the Internet, etc.

In step 81, the mapping system 55 maps the brain image 53 of the subject (user, examinee) to a standardized image (anatomical standardized image) 50. An overview of the anatomical standardization process is shown in Figure 3. Figure 4 also shows the standardization process.

Anatomical standardization involves mapping an individual's functional images onto a standard template, either linearly or nonlinearly. Anatomical standardization has become a standard method for aligning the positions of brain regions between subjects when analyzing medical images, and even brain images. This technology was originally required for creating activation maps from PET imaging, but the basic idea is to set a standard brain and then linearly or nonlinearly align input images to this standard brain.

Voxel-based morphometry (VBM) is a tool that provides one form of this processing for PET, fMRI, and MRI, but it is also implemented in other tools such as 3D-SSP. It is now common to perform nonlinear transformations when analyzing MRI images, but originally a method based on diffeomorphic mapping called LDDMM was devised, and the DARTEL method, which was devised as an improved method due to the long calculation time, is often used.

In analyses after such anatomical standardization, it can be assumed that the data represent the same location on the brain, allowing for comparisons of coordinate values (volume values in the case of MRI) between groups. Another benefit is that the analysis can absorb differences in the imaging conditions of the data. Furthermore, the ability to refine the complexity of the information contained in the data to the granularity required for analysis offers the advantage of being a reasonable preprocessing step in situations such as clinical research, where large amounts of data are involved and deep learning cannot be trained to handle all data processing processes.

In step 811 of Figure 3, horizontal axis adjustment (ACPC transformation) is performed on all images (Figure 4(a)). Furthermore, in step 812, each subject's image is segmented into gray matter (GM), white matter (WM), and cerebrospinal fluid (CSF) (Figure 4(b)). In step 813, DARTEL processing is performed on the images segmented into gray matter images using a template created from a standard brain only (Figure 4(c)). In step 814, standardization to MNI space is further performed (Figure 4(d)). After a series of nonlinear registration steps to MNI space coordinates, the gray matter images may be further subjected to modulation processing to shape the volume information and smoothing processing using an 8-mm wide Gaussian kernel. Note that the smoothing width is not limited to 8 mm.

Deep learning makes it possible to display the region of interest on the input image for specific models when performing differential evaluation. On the other hand, if anatomical standardization is performed, the same coordinate space is used for statistical analysis, which has the advantage of making it easy to draw the images side by side.

Figure 5 shows an example of a visualization technique called SHAP, where anatomically standardized morphological MRI images were used as input for deep learning. For example, in addition to visualizing the regions heavily used in deep learning as described above, we can also consider a method of using statistical analysis to extract regions of interest from validation samples that have been identified to improve accuracy (or sensitivity or specificity) when conducting accuracy comparison experiments with conventional methods for determining statistical cutoff values.

Returning to Figure 2, in step 82, a result 56 related to the reliability evaluation of mapping to the standardized image is obtained. Figure 6 shows the process of performing anatomical standardization using a deep learning model in the mapping system 55. Instead of steps 811 to 814, in step 815, the anatomical standardization process is performed using a deep learning model. Deep learning may be applied after anatomical standardization, or it may be applied to the anatomical standardization process. Anatomical standardization may use LDDMM for nonlinear transformation, but this may result in long processing times. This process can be trained using a neural network, introducing a method that is faster and more accurate than DARTEL. Furthermore, in step 816, the reliability of the mapping may be calculated using a Bayesian neural network while also increasing speed. This reliability evaluation result 56 may be used to correct the regions of interest (ROIs) in the first image evaluation system 60 and the second image evaluation system 70.

Returning to FIG. 2, in step 83, the support unit 30 may use the common image environment 15 to provide input control support for the first image evaluation system 60 and the second image evaluation system 70. Attributes and biomarkers such as age and gender may be further input into the anatomically standardized processed image 50, and the corrected anatomically standardized processed image may be used for subsequent processing. FIG. 7 shows the processing flow when attributes and biomarkers are further input and used.

In the input/output support of step 83, the common image environment 15 may be used to provide input control support for the second image evaluation system 70. In the second image evaluation system 70, a choice may be made between directly assessing the deep learning model and performing filtering by anatomical region before assessing the deep learning model. Figure 8 shows the processing flow when extracting regions.

Returning to Figure 2, in step 84, when the support unit 30 needs to obtain the evaluation results of the statistical processing, in step 85 it obtains the first evaluation results 66 regarding the statistical processing of the image of the first image evaluation system 60 via the first access unit (interface) 11 that is accessible to the first image evaluation system 60.

After anatomical standardization, intergroup comparisons are performed using statistical tests to consider coordinate positions with significant differences. In reality, clinical research often involves biases in age ranges and gender between groups, so the coordinate position values are analyzed after correcting for these as covariates. Generalized linear models are often used. This processing can be performed on anatomically standardized data before inputting it into deep learning, which motivates the system to have an input format for attribute values and biomarkers.

Statistical processing for evaluating medical images involves statistically comparing brain images of subjects with those of healthy individuals. One method for assessing brain atrophy using brain images is voxel-based morphometry (VBM), which processes brain images acquired by imaging the subject's head in units of voxels, which are three-dimensional pixels. A typical statistical processing technique is to generate a Z-score map.

Taking MR images as an example, the average and standard deviation images are created by calculating the average and standard deviation for each voxel from the MR images of a normal case that has undergone brain morphology standardization processing, and then substituting the values of the data (normal standard brain) and the values of the subject's image data (processed image) into the following formula, which calculates the Z score.
z=(M(x,y,z)-I(x,y,z))/SD(x,y,z)
M and SD represent the mean and standard deviation images of a normal control brain, and I represents the processed image. Using the Z-score map, it is possible to quantitatively analyze which regions of a processed image are undergoing changes compared to a normal control brain. For example, voxels with positive Z-score values indicate areas of atrophy compared to a normal control brain, and the larger the value, the greater the statistical deviation. For example, a Z-score of "2" indicates a region exceeding two standard deviations from the mean, which is evaluated as having a statistically significant difference with a risk of approximately 5%. To quantitatively evaluate atrophy within a region, M, SD, and I are calculated for the region of interest, respectively, and the average of all positive Z-scores can be calculated.

Various statistical methods have been proposed, including comparing the volume or area of various brain regions and using a t-test with a general linear model (GLM). Using the so-called Pittsburgh compound B as a PET radiopharmaceutical (tracer), measuring the level of amyloid beta protein accumulation in the brain based on PET images can be useful for the differential diagnosis or early diagnosis of Alzheimer's disease. These PET images can be used for statistical processing to obtain the SUVR value (Standardized Uptake Value Ratio, or cerebellar ratio SUVR), which indicates the ratio of the sum of the amyloid beta protein accumulation levels (SUV) in the cerebral gray matter of a portion of the brain to the amyloid beta protein accumulation level (SUV) in the cerebellum. SUVR can be defined by the following formula:

The numerator of this equation represents the sum of the SUVs of four cerebral gray matter regions, namely the cortical regions of the cerebral cortex (prefrontal cortex, anterior and posterior cingulate cortex, parietal lobe, and lateral temporal lobe), and the denominator represents the SUV of the cerebellum.

For statistical processing of DatSCAN using SPECT images, BR (Binding Ratio) can be used as an evaluation (index value), and is expressed by the following formula.

In the formula, C is the average value of DAT within each region of interest, Cspecific represents the average value for the putamen and caudate nucleus in the brain, and Cnonspecific represents the average value for the occipital cortex in the brain.

Visualization of regions of interest (ROIs) that show significant differences using statistical tests on anatomically standardized brain images is also performed in individual image evaluation systems, and tools such as SPM (Statistical Parametric Mapping) are known.

As an example, we will describe a study in which 208 subjects participated (101 in the DLB group, 69 in the AD group, and 38 in the healthy control group). Each disease group was diagnosed by two certified specialists from the Japanese Society of Dementia Research as having DLB or AD as the primary neurological disorder according to the DSM-5 diagnostic criteria. To exclude vascular disorders, subjects were recruited under the condition of excluding subjects with progressive and acute white matter lesions. This group excluded subjects from the healthy control group who were not diagnosed with dementia and were suspected of having a central nervous system disorder. This study was approved by the ethics committee and conducted in accordance with the guidelines of the participating institutions.

Figure 9 shows the demographics of the subjects. The DLB group consisted of 50 women and 51 men, with a mean and standard deviation of age of 73.25 ± 8.05 years. The AD group consisted of 36 women and 33 men (also age 71.58 ± 6.33 years). The NC group consisted of 28 women and 10 men (also age 71.03 ± 6.28 years). There were no significant differences in age or gender between any of the subject groups. All subjects took the MMSE test, and the scores for the DLB, AD, and NC groups were 22.21 ± 4.86, 21.32 ± 3.95, and 28.21 ± 1.26, respectively, indicating no significant difference between the DLB and AD groups.

The MRI data of the subjects was acquired using a total of 11 different scanners. The images were 3D T1-weighted images acquired using sagittal gapless imaging, and the pulse sequence of each MRI scanner is shown in Figure 10. Each MRI image was converted into an anatomical standardized image 50 using the processing described above.

Statistical analysis using SPM was performed on the three groups (NC, DLB, and AD) using ANOVA analysis normalized for ICV (intracranial volume). Figure 11 shows the MNI (Montreal Neurological Institute) coordinates where significant differences in gray matter volume were determined to exist between groups. The area names for the MNI coordinate locations are based on WFU Pickatlas (Department of Radiology of Wake Forest University School of Medicine, Winston-Salem, North Carolina; fmri.wfubmc.edu).

Figure 12 shows the ROIs for the three groups after ICV normalization. Figure 12 shows the results of ICV normalization. Significant differences in gray matter volume were observed over a relatively wide area among the three groups, with the most significant difference occurring in the region extending from the parahippocampal gyrus to the brainstem.

Figure 13 shows the results of evaluating the effect within the ROI of each group after ICV normalization using 90% CI. No significant differences were observed between the DLB and AD groups.

Returning to Figure 2, in step 86, when the support unit 30 needs to obtain a second result 76 regarding the evaluation of the medical image from a second image evaluation system 70 that determines the subject's disease state using a first model (deep learning model) 74 that has been machine-learned to evaluate a first disease, e.g., AD or DLB, based on the medical image, the support unit 30 obtains the second evaluation result 76 of the second image evaluation system 70 via a second access unit (interface) 13 that is accessible to the second image evaluation system 70 in step 87.

Diseases are being differentiated from medical images of subjects using machine learning models (learning models) based on information from medical images. Iizuka, Tomomichi et al., "Deep-learning-based imaging-classification identified cingulate island sign in dementia with Lewy bodies" (Scientific Reports 9.1 (2019): 1-9.), report that an even higher accuracy of 89.32% was achieved in an experiment using a convolutional neural network on perfusion SPECT images, and that deep learning focuses on blood flow findings in the occipital lobe, which have traditionally been used in image interpretation, for the purpose of differentiation. Litjens, Geert et al., "A survey on deep learning in medical image analysis" (Medical image analysis 42 (2017): 60-88.) and Wen, Junhao et al., "Convolutional Neural Networks for Classification of Alzheimer's Disease: Overview and Reproducible Evaluation" (CoRR abs/1904.07773 (2019)) report that the application of recent deep learning technology has demonstrated high accuracy in the diagnosis of AD.

However, no research has focused on whether deep learning can differentiate between DLB and AD, and there are no known studies that have applied deep learning to the differentiation of DLB and AD using morphological MRI images. Therefore, as part of the above research, we investigated a differentiation approach using deep learning on morphological MRI images.

Figure 14 shows an overview of the model used. In this study, we adopted a ResNet-type neural network model adapted for 3D data. ResNet is a type of convolutional neural network model, characterized by a model structure that prevents the loss of signal source features compared to conventional convolutional neural networks. Furthermore, by adding a mechanism called "skip connection" to the convolutional layer, ResNet can adopt a mechanism that transmits the output of the convolution to the next layer by combining the convolution output and the input of the current layer. This prevents information loss in the training data even when the model layers are deep, achieving high accuracy in many image classification tasks. In this study, we adopted an implementation using the Python framework. We previously tested network depths of 18, 34, 50, and 101 layers, and finally adopted 34 layers. The training process was performed using one NVIDIA Tesla K80 GPU, and the optimization process used stochastic gradient descent (SGD) with an inertia term.

Figure 15 shows an ROC plot of the changes in sensitivity and specificity for the model with the highest validation accuracy for each 5-fold, while varying the softmax disease detection threshold in the output layer. When balanced using YoudenIndex, the sensitivity and specificity were 81.54 ± 10.43% and 76.77%, respectively, and the accuracy was 79.15 ± 5.22% (sensitivity and accuracy are 5-fold mean ± SD when specificity is fixed). Although limited, this experiment confirmed that a deep learning model using the same gray matter volume data was capable of distinguishing between the DLB and AD groups, where no significant difference could be confirmed using conventional SPM statistical testing.

The results of the preliminary validation experiment showed that the areas with significant differences in gray matter between the three groups (healthy control, DLB, and AD) were the right middle temporal pole and left parahippocampal gyrus. There are precedents for detecting brainstem regions in analysis using ICV, and other studies have confirmed significant differences between DLB and AD groups, albeit in white matter dorsal to the brainstem. However, in our preliminary validation, the difference in the degree of atrophy of the gray matter, from the hippocampus to the brainstem, between the DLB and AD groups was small compared to the intracranial volume.

Analysis using ICV revealed a significant difference in the parahippocampal gyrus. Hippocampal atrophy is typically seen as a sign of AD progression, but this study demonstrated atrophy in both the DLB and AD groups, suggesting that atrophy was more advanced in the AD group than in the DLB group. Other reports have used a cascade approach, first evaluating atrophy in the parahippocampal gyrus to differentiate between healthy and dementia groups, and then evaluating atrophy in the dorsal brainstem to differentiate between DLB and AD. Our preliminary validation results suggest that it may be more effective to differentiate between DLB and AD groups using the parahippocampal gyrus. Therefore, we performed a t-test on two independent groups, the DLB group and the AD group, but no significant differences were detected in either the hippocampus or the dorsal brainstem. Based on these findings, it is thought that the pattern of gray matter atrophy between the DLB and AD groups is so subtle that it cannot be detected using conventional methods based on statistically significant differences, compared to differentiating between healthy and diseased groups.

In contrast, our experimental results, applying deep learning to VBM-based data, demonstrated the ability to differentiate DLB and AD groups with a certain degree of accuracy. For example, when using the deep learning model of this embodiment, we achieved a 35.79% improvement in sensitivity and a 15.70% improvement in accuracy compared to the VBM-based method. This demonstrates that deep learning models such as this embodiment can be used to differentiate DLB and AD groups. Furthermore, in the VBM preprocessing step we adopted, the deep learning model was able to evaluate minute volumetric differences more sensitively than statistical analysis under standard significance conditions. However, considering clinical applications, applying such very small differences to diagnosis carries risks. Therefore, comprehensive judgments, including test data other than morphological MRI, are required. Furthermore, it is necessary to further evaluate the generalization performance of this method by increasing the number of validation facilities.

For example, while the atrophy patterns of the DLB and AD groups can be subtle, the deep learning model demonstrates a certain level of discrimination performance. These results suggest that deep learning may be able to handle subtle features that are effective for discrimination, but it is important to analyze and evaluate areas of interest that are effective for discrimination, for example by visualizing the features evaluated for deep learning discrimination. The support unit 30 provides a support environment for image evaluation in such cases.

Returning to FIG. 2, when, in step 88a, a request is made to display the evaluation results of each system 60 and 70, in step 88b, the display and comparison unit 31 of the support unit 30 provides the user with a first result 66 regarding the evaluation of the medical image obtained from the first image evaluation system 60 and a second result 76 regarding the evaluation of the subject's medical image obtained from the second image evaluation system 70 via the common image evaluation environment 15.

In step 89a, when a review of the statistical processing is requested based on the discrimination result of the learning model 74, in step 89b, the re-statistical processing request unit 32 of the support unit 30 requests the first image evaluation system 60 via the input control information 67 to re-evaluate the first result 66 regarding the evaluation of the medical image obtained from the first image evaluation system 60 based on the judgment of the disease state of the second image evaluation system 70, and outputs the result 66 via the common image evaluation environment 15.

For example, in the previous experiment, there were five cases of DLB where the existing method gave an incorrect diagnosis but the deep learning model 74 gave a correct diagnosis. For these cases, a test was conducted using age and gender as covariates, including correction using a general linear model, and a predetermined or required number of healthy subjects as the control group. No significant differences were confirmed at p<0.05 using the FWE multiple comparison correction, but further testing without multiple comparison correction at p<0.001 revealed some significant differences.

Figure 16 shows the SPM statistical test results for five DLB verification subjects, the areas with the smallest p-values, and the softmax output values (0-1) that served as the basis for the deep learning model 74's judgment for that subject. The gray matter areas with the highest p-values for each subject were the putamen, caudate nucleus, entorhinal cortex, parahippocampal gyrus, and amygdala, respectively. The striatum, such as the putamen and caudate nucleus, is a region where dopaminergic neuronal degeneration is observed in dementia with Lewy bodies and Parkinson's disease. Atrophy around the hippocampus was observed in subjects (c) and (d). Atrophy around the hippocampus is often observed in AD cases, but in these cases, the evaluation using the softmax function output values of the proposed method provided relatively low evidence for DLB diagnosis, although the difference was slight. Atrophy of the amygdala was observed in subject (e). The amygdala is also a region where α-synuclein accumulation has been reported in cortical Lewy bodies. While existing methods define an ROI limited to the dorsal brainstem region and perform an evaluation, the method using learning model 74 is able to capture the necessary features better than existing methods in Lewy body dementia, where pathological mechanisms affect a wide range of areas, and this may have contributed to improved accuracy.

In the case of differentiating AD/DLB using morphological MRI images, it was necessary to capture very subtle changes between the AD/DLB groups, and statistical processing was unable to extract ROIs with significant differences. In contrast, deep learning model 74 achieved a certain level of discrimination accuracy, and although the volume values within the region of interest at this time were not significantly different, they can be considered an effective indicator for differentiating between groups. In particular, since VBM only leaves volume information, if deep learning model 74 does not consider the interrelationships with distant areas, this UI can provide clinical significance by displaying as an indicator these subtle changes that are discarded and not presented as results in conventional statistical analysis tools.

Returning to Figure 2, in step 90a, when there is a request to overlay the ROI of the statistical processing and the ROI of the deep learning model 74, in step 90b, the overlay display unit 33 of the support unit 30 outputs, via the common image evaluation environment 15, a first region (ROI) that is considered important by the first result 66 regarding the evaluation of the medical image obtained from the first image evaluation system 60, and a second region (ROI) that is considered important for determining the disease state by the second image evaluation system 70.

Figure 17 shows an example of using GradCAM with a deep learning model 74 to output a region of interest (ROI) used in the deep learning model 74's discrimination, and displaying it on an anatomical standardized image 101. This screen 100 displays the GradCAM output using a sagittal section 102, coronal section 103, and horizontal section 104 of the anatomically standardized brain. Screen 100 also shows the volumes 105 of GM (gray matter), WM (white matter), TBV, and ICV, along with the average values for healthy subjects (shown in parentheses). Clinical information 108 and DLB confidence level 106 of the subject are also displayed.

In this example, the only check item for "Select Area" is "Gray Matter," but multiple areas can be selected, not just gray matter. It may also be possible to select more specific areas within the gray matter.

Figure 18 shows an example of the results of statistical processing of the brain image 53 of the same subject. In this display 110, the region of interest (ROI) of the Z-score is shown using the sagittal section 112, coronal section 113, and horizontal section 114 of the anatomical standardized image 111.

Figure 19 shows how a first evaluation result, including a region of interest in the statistical evaluation of the medical image of a first subject obtained from a first image evaluation system, and a second evaluation result, including a region of importance captured by a second image evaluation system when determining the disease state of the first subject's medical image, are overlaid and output in a common image evaluation environment. For example, the ROI of the deep learning model 74 obtained by GradCAM and the ROI of the Z-score are overlaid. In this overlay display 120, the ROIs of both models are overlaid on the sagittal section 122, coronal section 123, and horizontal section 124 of the anatomical standardized image 121. In this way, if the input for differentiation by the deep learning model 74 is a brain image mapped to an anatomical standard brain, it becomes possible to visually compare and contrast the ROIs with conventional statistically calculated ROIs on the same image.

Returning to Figure 2, in step 91a, when processing of the learning model 74 is requested based on the ROI of statistical processing, in step 91b, the model input selection unit 34 of the support unit 30 selects an image area including a first area (ROI) that is considered important by the first result 66 regarding the evaluation of the medical image obtained from the first image evaluation system 60 via the common image evaluation environment 15, and provides it as an evaluation target for the second image evaluation system 70 via input control information 77.

In step 92a, if it is selected to perform statistical processing based on the ROI of the learning model 74, in step 92b, the statistical processing input selection unit 35 of the support unit 30 selects an image region including a second region (ROI) that is considered important for the second image evaluation system 70's assessment of the disease state via the common image evaluation environment 15, and provides it as an evaluation target for the first image evaluation system 60 via input control information 67. The first image evaluation system 60 calculates values of brain volume and blood flow within the region of interest of the deep learning model, and is able to present human-interpretable index values for regions that are effective for differentiation. Statistical processing is not limited to Z scores, and may also include volume values, volume density values, blood flow, glucose metabolism, and accumulation amounts of tracer reactants.

If, in step 93a, it is requested that an evaluation of the reliability of the mapping be taken into account, in step 93b, the mapping evaluation unit 36 of the support unit 30 controls the output of a first result 66 regarding the evaluation of the medical image obtained from the first image evaluation system 60 or a second result 76 regarding the evaluation of the medical image of the subject obtained from the second image evaluation system 70 using a common image evaluation environment 15 based on the reliability of the mapping to the common image evaluation environment of the subject's medical image, in this example, the standardized image 50.

Once these image evaluation support programs are completed, in step 94, medical personnel may evaluate the discrimination results of the deep learning model 74 based on various information provided via the common image evaluation environment 15.

While this example primarily describes AD, DLB, and healthy individuals, the present invention is not limited to AD and DLB. The system, control method, and program of this embodiment can also be applied to brain disorders (including brain diseases). Brain disorders include dementia, as well as higher-level brain disorders such as attention disorders, memory disorders, executive dysfunction, social behavior disorders, aphasia, apraxia, and apraxia. Dementia includes AD (Alzheimer's Disease), DLB (Dementia with Lewy Bodies), and other degenerative dementias, such as frontotemporal dementia, progressive supranuclear palsy, corticobasal degeneration, and argyrophilic grain dementia. The state of brain damage includes various aspects related to the brain damage of the subject (examinee, patient, user), such as the presence or absence of brain damage, its progression, the presence and differentiation of causative diseases (causative illnesses) of brain damage such as dementia, and the progression of single or multiple causative illnesses. Brain diseases also include dementia (including AD, DLB, frontotemporal lobar degeneration (FTLD), normal pressure hydrocephalus (NPH), etc.), brain tumors, mental disorders (also known as mental illnesses, including schizophrenia, epilepsy, mood disorders, addiction disorders, higher brain dysfunction, etc.), Parkinson's disease, Asperger's syndrome, attention deficit hyperactivity disorder (ADHD), sleep disorders, childhood diseases, ischemic brain disorders, mood disorders (including depression, etc.), etc. Brain disorders include brain-related diseases such as dementia and multiple sclerosis, and amyloid-β-related diseases include, for example, mild cognitive impairment (MCI), mild cognitive impairment due to Alzheimer's disease (MCI due to AD), prodromal Alzheimer's disease, preclinical AD (the pre-symptomatic stage of Alzheimer's disease), Parkinson's disease, multiple sclerosis, insomnia, sleep disorders, cognitive decline, cognitive impairment, and amyloid-positive/negative related neurodegenerative diseases.

In the above, the present invention has been described using an example in which the target region included in the medical image to be evaluated is the brain or a part of the brain, but the target region is not limited to the brain and may be any other part of the subject's body. Furthermore, the disease to be evaluated is not limited to dementia, and may be any disease related to any other part of the body as long as it is a disease that is the target of image diagnosis.

1 Image diagnosis support information provision system 8, 9 Configuration range 10 Support system 11 First access unit 12 Second access unit 13 Access unit 15 Image evaluation environment (common evaluation environment)
16a Display 16b Touch panel 17 Cloud 18 Database 19 Program 20 Unit 30 Support unit 31 Individual and comparative evaluation unit 32 Re-statistical processing request unit 33 Overlay display (overlay) unit 34 Model input selection unit 35 Statistical processing input selection unit 36 Mapping evaluation unit 37 Input support function (input support unit)
50 Standardized image 52 Image database 53 Brain image (first type of medical image)
55 Mapping system 56 Third result 60 First image evaluation system 61 Processor 62, 72 Input of evaluation target 63, 73 Output of evaluation result 65 Database 66 First result 67, 77 Input control information 70 Second image evaluation system 71 Processor 74 Deep learning model (learning model, first model)
75 Database 76 Second result

Claims (15)

a first access unit accessible to a first image evaluation system for statistically evaluating a first type of medical image having as a region of interest at least a portion of a subject's body;
a second access unit accessible to a second image evaluation system that determines the disease state of the subject using a first model trained by machine learning to evaluate a first disease based on the first type of medical image;
a support unit that provides at least one of input of an evaluation target and output of an evaluation result to the first image evaluation system and the second image evaluation system via a common image evaluation environment;
The support unit is one of the following:
i) The support unit includes a unit that selects an image area including a first area that is considered important by a first result regarding the evaluation of the medical image obtained from the first image evaluation system as an evaluation target for the second image evaluation system via the common image evaluation environment.
ii) The support unit includes a unit that selects an image area including a second area that is important for determining the disease state of the second image evaluation system as an evaluation target for the first image evaluation system via the common image evaluation environment.
iii) The assistance unit includes a unit for controlling the output of a first result regarding the evaluation of the medical image obtained from the first image evaluation system or a second result regarding the evaluation of the medical image of the subject obtained from the second image evaluation system using the common image evaluation environment based on the reliability of mapping of the medical image of the subject to the common image evaluation environment. In claim 1,
The support unit uses the common image evaluation environment, in which the first type of medical image of the subject includes evaluation images that are standardized to images that can be input and output to the first image evaluation system and the second image evaluation system. In claim 1 or 2,
The system includes a unit for outputting, via the common image evaluation environment, a first result regarding the evaluation of the medical image obtained from the first image evaluation system and a second result regarding the evaluation of the medical image of the subject obtained from the second image evaluation system. In claim 3,
The support unit includes a unit that outputs the first result and the second result in the common image evaluation environment in an overlaid manner. In any one of claims 1 to 4,
The system includes a unit for outputting, via the common image evaluation environment, a result of re-evaluating a first result regarding the evaluation of the medical image acquired from the first image evaluation system based on the judgment of the disease state of the second image evaluation system. In any one of claims 1 to 5,
The support unit includes a unit that outputs, via the common image evaluation environment, a first area that is considered important by a first result regarding the evaluation of the medical image obtained from the first image evaluation system and a second area that is considered important for determining the disease state of the second image evaluation system. In claim 6,
The support unit includes a unit that outputs the first area and the second area in the common image evaluation environment in an overlaid manner. In any one of claims 1 to 7,
The system, wherein the first disease includes dementia, the first type of medical image is an MR image, and the target region includes at least one of the parahippocampal gyrus, dorsal brainstem, and portions of the basal ganglia (putamen, caudate nucleus, entorhinal cortex, parahippocampal gyrus, amygdala) of the brain. In any one of claims 1 to 7,
the first disease comprises dementia;
the first type of medical image is a SPECT image;
The target region includes at least one of the precuneus, the occipital lobe, and the dorsolateral prefrontal cortex of the brain;
system. In any one of claims 1 to 9,
The system further comprises a unit for verifying an assessment of the first disease according to the first model based on an output of the support unit. A control method for an assistance system, comprising:
The assistance system includes:
a first access unit accessible to a first image evaluation system for statistically evaluating a first type of medical image having as a region of interest at least a portion of a subject's body;
a second access unit accessible to a second image evaluation system that determines the disease state of the subject using a first model trained by machine learning to evaluate a first disease based on the first type of medical image;
a support unit that provides at least one of input of an evaluation target and output of an evaluation result to the first image evaluation system and the second image evaluation system via a common image evaluation environment;
The control method includes at least one of the following steps:
i) The support unit selects an image area including a first area that is considered important by a first result regarding the evaluation of the medical image obtained from the first image evaluation system as an evaluation target for the second image evaluation system via the common image evaluation environment.
ii) The support unit selects an image area including a second area that is important for the second image evaluation system to determine the disease state as an evaluation target for the first image evaluation system via the common image evaluation environment.
iii) The assistance unit controls the output of a first result regarding the evaluation of the medical image obtained from the first image evaluation system or a second result regarding the evaluation of the medical image of the subject obtained from the second image evaluation system using the common image evaluation environment based on the reliability of mapping of the medical image of the subject to the common image evaluation environment. In claim 11,
A control method, wherein the first disease includes dementia, the first type of medical image is an MR image, and the target region includes at least one of the hippocampus, parahippocampal gyrus, dorsal brainstem, middle temporal pole, and parts of the basal ganglia (putamen, caudate nucleus, entorhinal cortex, parahippocampal gyrus, amygdala) of the brain. In claim 11,
the first disease includes dementia, the first type of medical image is a SPECT image, and the region of interest includes at least one of the precuneus, the occipital lobe, and the dorsolateral prefrontal cortex of the brain;
Control method. In any one of claims 11 to 13,
The control method further comprises verifying the assessment of the first disease by the first model based on the output of the assistance unit. A program for evaluating medical images by a computer, comprising:
accessing, by the computer, a first image evaluation system for statistically evaluating a first type of medical image having at least a portion of a subject's body as a region of interest;
accessing a second image assessment system that determines the disease state of the subject using a first model trained on machine learning to assess a first disease based on the first type of medical image;
providing at least one of input of an evaluation target and output of an evaluation result to the first image evaluation system and the second image evaluation system via a common image evaluation environment;
A program having instructions for performing at least one of the following steps:
i) Selecting an image area including a first area that is considered important by a first result regarding the evaluation of the medical image obtained from the first image evaluation system as an evaluation target for the second image evaluation system via the common image evaluation environment.
ii) Selecting an image area including a second area that is considered important for determining the disease state by the second image evaluation system as an evaluation target for the first image evaluation system via the common image evaluation environment.
iii) Based on the reliability of mapping of the medical image of the subject to the common image evaluation environment, controlling the output of a first result regarding the evaluation of the medical image obtained from the first image evaluation system or a second result regarding the evaluation of the medical image of the subject obtained from the second image evaluation system using the common image evaluation environment.