JP7833309B2 - Medical information processing device and medical information processing method - Google Patents

Description

Embodiments disclosed herein and in the drawings relate to a medical information processing device and a medical information processing method.

By systematizing disease classifications and relationships as medical knowledge and constructing databases, it is expected that this can be utilized for understanding complex diseases, discovering research hypotheses, and supporting clinical practice. For example, in the medical field, medical ontologs such as ICD-10 (a disease classification system) and SNOMED-CT (which describes semantic relationships between medical terms) have been developed. Furthermore, in recent years, data analysis methods using graph structures that can represent the relationships between multiple clinical events have attracted attention.

Diseases are subject to fluctuations. That is, there isn't always a single, clear criterion for a disease; even the same disease can have various pathological conditions, its definition may change over time, and diagnoses can vary even among different physicians.

Japanese Patent Publication No. 2018-60529

Y Li et al., “Graph Neural Network-Based Diagnosis Prediction”, Big Data, 8(5):379-390, 2020.

One of the problems that the embodiments disclosed in this specification and drawings aim to solve is to improve the accuracy of medical judgments. However, the problems that the embodiments disclosed in this specification and drawings aim to solve are not limited to the above problem. Problems corresponding to the effects of each configuration shown in the embodiments described later can also be positioned as other problems.

The medical information processing device according to this embodiment includes an acquisition unit, a mapping unit, and an estimation unit. The acquisition unit acquires medical information relating to the medical events of a target patient. The mapping unit maps the medical information onto a first graph having nodes corresponding to medical events and edges representing the relationships between those nodes, thereby generating a second graph relating to the target patient. The estimation unit estimates medical judgment information for the target patient based on the second graph relating to the target patient.

Figure 1 shows an example of the configuration of a medical information system according to this embodiment. Figure 2 is a conceptual diagram of the medical knowledge graph. Figure 3 shows an example of the configuration of the medical information processing device shown in Figure 1. Figure 4 shows the flow of medical information processing using the medical information processing device shown in Figure 3. Figure 5 shows an example of the mapping process related to step SA2 in Figure 4. Figure 6 shows an example of the estimation process related to step SA3 in Figure 4. Figure 7 shows an example of the display screen shown in step SA4 of Figure 4. Figure 8 shows an example of a patient individual circumstances database used in step SA5 of Figure 4. Figure 9 shows an example of the configuration of the medical information learning device shown in Figure 1. Figure 10 shows the flow of machine learning processing by the medical information learning device shown in Figure 9. Figure 11 is a schematic diagram showing the update process related to step SB3 in Figure 10. Figure 12 shows an example of the configuration of the medical information display device shown in Figure 1. Figure 13 is a diagram showing the flow of medical knowledge graph display processing by the medical information display device shown in Figure 12. Figure 14 shows an example of the initial screen displayed in step SC1 of Figure 13. Figure 15 shows an example of the display screen shown in step SC4 of Figure 13. Figure 16 shows an example of a visualization graph displaying information for the entire patient population. Figure 17 shows an example of a visualization graph displaying information about the disease as a whole. Figure 18 is a conceptual diagram of the assignment of graph features (occurrence order, occurrence frequency, and occurrence severity) related to Application Example 1. Figure 19 schematically shows the input and output of two machine learning models related to Application Example 2. Figure 20 is a schematic diagram showing the convolution process using a graph convolution layer related to Application Example 3. Figure 21 shows an example of the input and output of a linear coupling layer related to Application Example 4. Figure 22 is a schematic diagram showing the machine learning model related to Application Example 5. Figure 23 is a diagram showing an overview of the graph features related to application example 6. Figure 24 is a diagram showing an overview of the graph features related to application example 7. Figure 25 shows an example of the estimation process for medical judgment information related to Application Example 7. Figure 26 schematically shows the relationship between the patient graph and spatial information related to application example 9. Figure 27 shows the concept of a patient graph network related to application example 9. Figure 28 is a diagram showing an overview of the graph features related to application example 10.

The embodiments of the medical information processing device, medical information learning device, medical information display device, and medical information processing method will be described in detail below with reference to the drawings.

Figure 1 shows an example configuration of the medical information system 100 according to this embodiment. The medical information system 100 is a computer network system comprising a medical information storage device 1, a medical knowledge graph storage device 2, a medical information processing device 3, a medical information learning device 4, and a medical information display device 5. The medical information storage device 1, the medical knowledge graph storage device 2, the medical information processing device 3, the medical information learning device 4, and the medical information display device 5 are connected to the network in a manner that allows them to communicate with each other.

The medical information storage device 1 is a computer with a storage device for storing medical information of multiple patients. The medical knowledge graph storage device 2 is a computer with a storage device for storing medical knowledge graphs. The medical knowledge graph is represented by a graph structure generated based on medical knowledge such as the medical information and medical ontology of multiple patients. Hereafter, the medical knowledge graph is assumed to be a graph. The medical information processing device 3 is a computer that estimates medical judgment information for a target patient using the target patient's medical information and medical knowledge graph. The medical information processing device 3 can also accumulate various types of medical knowledge in the medical knowledge graph. The medical information learning device 4 is a computer that trains machine learning models used for estimating medical judgment information. The medical information display device 5 is a computer that extracts and displays desired medical knowledge from the medical knowledge graph.

Figure 2 is a conceptual diagram of the medical knowledge graph 20. The medical knowledge graph 20 is a graph that integrates medical knowledge with the medical information of multiple patients. Medical information is information generated through the medical treatment of a particular patient. Medical information is classified into information related to medical events (hereinafter referred to as medical event information) and information related to diseases (hereinafter referred to as disease information). Medical events are specific matters related to medical treatment. The types of medical events can be divided into four categories, for example: symptoms, findings, treatment, and response. Symptoms are physical and mental changes caused by a disease. Specifically, types of medical events related to symptoms include, for example, edema, palpitations, and shortness of breath. Findings are the doctor's judgment regarding the symptoms. Findings may be further divided into categories of physical findings and laboratory findings. Treatment is a medical act to cure or alleviate symptoms. Types of medical events related to treatment include, for example, administration of cardiac stimulants and CRT implantation surgery. Responses are the physical and mental reactions of the patient to the treatment. The responses may be further divided into categories of therapeutic responses and side effects. Examples of clinical events related to responses include, for instance, diuresis exceeding 40 mL/h or electrolyte abnormalities in response to diuretic administration. Disease information refers to information regarding the name or symbol of the disease diagnosed in the patient.

Medical information is collected through various hospital information systems, including the Hospital Information System (HIS), the Radiology Information System (RIS), and the Picture Archiving and Communication System (PACS). Each medical event and disease information included in the medical information is associated with the date of occurrence of that information.

As shown in Figure 2, the medical knowledge graph 20 is a graph constructed from multiple nodes 21 and multiple edges 22. Each node 21 corresponds to a medical event. The edges 22 indicate the relationships between the nodes 21. That is, the edges 22 indicate the relationships between the medical events corresponding to the connected nodes 21. The relationships between medical events refer to causal relationships and correlations. Causal relationships refer to cause-and-effect relationships between medical events, while correlations refer to relationships without causality. That is, edges 22 representing causal relationships are directed, and edges 22 representing correlations are undirected. Edges 22 can also connect medical events belonging to different categories. The medical events used in the medical knowledge graph 20 are selected from the medical information stored in the medical information storage device 1 based on medical knowledge, medical ontology, etc. The relationship between one medical event and another is analyzed based on medical knowledge, medical ontology, etc., and if a relationship is recognized, the two nodes 21 corresponding to those two medical events are connected by an edge 22.

Even within the same disease, the presence or strength of relationships between medical events varies depending on the individual circumstances of each patient. Estimating a patient's disease without considering their individual circumstances does not necessarily yield accurate results.

The medical knowledge graph 20 does not have nodes corresponding to clinical events such as diseases. A specific disease is extracted from a set of clinical events—symptoms, findings, treatment, and response—that occurred in the target patient. In other words, the medical knowledge graph 20 represents individual diseases as a chain of relationships between a series of clinical events: symptoms, findings, treatment, and response. The medical knowledge graph 20 can also be described as a graph representing a disease concept, describing a chain of relationships between a series of clinical events: symptoms, findings, treatment, and response. The medical information system 100 constructs a medical knowledge graph (patient graph) adapted to the context of the patient's individual circumstances to perform accurate medical judgments, such as disease estimation.

Figure 3 shows an example configuration of the medical information processing device 3 according to this embodiment. As shown in Figure 3, the medical information processing device 3 includes a processing circuit 31, a memory 32, an input interface 33, a communication interface 34, and a display 35.

The processing circuit 31 includes processors such as a CPU (Central Processing Unit) and a GPU (Graphics Processing Unit). The processing circuit 31 executes a medical information processing program to estimate and present medical judgment information, thereby realizing functions such as acquisition (311), mapping (312), estimation (313), visualization graph generation (314), storage (315), and display control (316). Note that each of the functions 311-316 is not limited to being realized by a single processing circuit. Multiple independent processors may be combined to form a processing circuit, and each processor may execute a program to realize each of the functions 311-316. Furthermore, each of the functions 311-316 may be a modularized program constituting the medical information processing program, or it may be an individual program. These programs are stored in memory 32.

By implementing the acquisition function 311, the processing circuit 31 acquires various types of information. For example, the processing circuit 31 acquires medical event information, which is medical information related to the medical events of the target patient, from the medical information storage device 1. The processing circuit 31 also acquires a medical knowledge graph from the medical knowledge graph storage device 2.

By implementing the mapping function 312, the processing circuit 31 generates a patient graph for the target patient, which maps the patient's medical event information onto a medical knowledge graph having nodes corresponding to medical events and edges representing the relationships between those nodes.

By implementing the estimation function 313, the processing circuit 31 estimates medical judgment information for the target patient based on the patient graph. The medical judgment information is information related to at least one of the following medical judgments: disease classification information, prognosis prediction information, and severity classification information.

By implementing the visualization graph generation function 314, the processing circuit 31 generates a patient graph for display (hereinafter referred to as the visualization graph) for the target patient based on the medical knowledge graph and the patient's medical information.

By implementing the storage function 315, the processing circuit 31 stores individual patient information in the medical knowledge graph. Individual patient information will be described later.

The display control function 316 enables the processing circuit 31 to display various information on the display 35. For example, the processing circuit 31 displays medical judgment information for the target patient and visualization graphs of the target patient.

Memory 32 is a storage device such as ROM (Read Only Memory), RAM (Random Access Memory), HDD (Hard Disk Drive), SSD (Solid State Drive), or integrated circuit memory that stores various types of information. Memory 32 may also be a drive device that reads and writes various types of information to portable storage media such as CDs (Compact Discs), DVDs (Digital Versatile Discs), or flash memory, or to semiconductor memory elements. Furthermore, memory 32 may be located in another computer connected to the medical information processing device 3 via a network. For example, memory 32 stores medical knowledge graphs acquired by the acquisition function 311.

The input interface 33 receives various input operations from the operator, converts the received input operations into electrical signals, and outputs them to the processing circuit 31. Specifically, the input interface 33 is connected to input devices such as a mouse, keyboard, trackball, switch, button, joystick, touchpad, and touch panel display. The input interface 33 outputs electrical signals corresponding to the input operations to the input device to the processing circuit 31. Furthermore, the input device connected to the input interface 33 may also be an input device located on another computer connected via a network or the like.

The communication interface 34 is an interface for sending and receiving various types of information between the medical information system 100 and other computers, such as the medical information storage device 1, the medical knowledge graph storage device 2, the medical information learning device 4, and the medical information display device 5.

The display 35 displays various information according to the display control function 316 of the processing circuit 31. As the display 35, for example, a liquid crystal display (LCD), a cathode ray tube (CRT) display, an organic electroluminescent display (OLED), a plasma display, or any other suitable display can be used. Furthermore, a projector may be provided instead of or in conjunction with the display 35.

Next, we will describe the medical information processing performed by the processing circuit 31 according to the medical information processing program. In the following embodiment, the medical judgment information is assumed to be disease classification information.

Figure 4 shows the flow of medical information processing. As shown in Figure 4, the processing circuit 31 acquires the patient's medical event information and medical knowledge graph by implementing the acquisition function 311 (step SA1). The medical knowledge graph is acquired from the medical knowledge graph storage device 2. The medical event information is information that has a combination of the type of medical event and a value indicating the degree to which the patient is relevant to that medical event. The degree of relevance is typically represented by two values, "relevant" or "not relevant," but it may also be defined by three or more discrete or continuous values indicating the degree of relevance. In the following embodiment, the degree of relevance in the medical event information acquired in step SA1 is represented by a binary value. The medical event information is acquired, for example, as follows:

First, the processing circuit 31 obtains the patient's medical information history from the medical information storage device 1. Next, the processing circuit 31 obtains medical event information from the medical information history. For example, the processing circuit 31 performs information processing, such as document retrieval using medical events registered in the medical knowledge graph as search keys, on the patient's medical information history to determine whether the medical events registered in the medical knowledge graph are included in the medical information history. Then, the processing circuit 31 generates medical event information, assigning a value to each medical event indicating its inclusion in the history (i.e., a value of "1" indicating it is relevant) or a value indicating it is not relevant (i.e., a value of "0" indicating it is not relevant). Note that the medical event information may be stored in the medical information storage device 1, and the medical event information may be obtained from the medical information storage device 1 in step SA1. Note that in step SA1, medical event information for the period corresponding to the period to be considered when estimating disease classification information is obtained. Furthermore, the medical event information obtained in step SA1 does not necessarily include medical event information for all categories, nor does it necessarily omit medical event information for some categories.

Once step SA1 is performed, the processing circuit 31, by implementing the mapping function 312, maps the medical event information acquired in step SA1 onto a medical knowledge graph to generate a patient graph for the target patient (step SA2).

Figure 5 shows an example of the mapping process related to step SA2. As shown in Figure 5, the medical event information includes information indicating whether each medical event is applicable or not. The medical events indicated as applicable or not in the medical event information 11 correspond to each node that constructs the medical knowledge graph. If the medical event occurs in the target patient, a value of "1" is assigned to indicate "applicable," and if the medical event does not occur in the target patient, a value of "0" is assigned to indicate "not applicable." For example, the medical event information 11 is given in the format that "cardiac dilation" is applicable ("1"), "edema" is applicable ("1"), and "palpitations" is not applicable ("0").

As described above, the medical knowledge graph 20A is composed of multiple nodes 21 and edges 22 connecting the nodes 21. Nodes 21 and edges 22 are also called graph components. Each node 21 corresponds to a predefined medical event. Medical events are classified into four categories: symptoms, findings, treatment, and response, and each node 21 is assigned a specific medical event. The edges 22 represent the relationships between medical events. If two medical events are related, the two nodes 21 corresponding to those two medical events are connected by an edge 22. If two medical events are not related, the two nodes 21 corresponding to those two medical events are not connected by an edge 22.

Edge 22 can be described by the adjacency matrix A shown in equation (1) below. Each row and column corresponds to each node 21. For example, the matrix element at row number "1" and column number "0" is "0," indicating the existence of an edge connecting the first and second nodes. Diagonal elements with equal row and column numbers represent edges between identical nodes and are therefore assigned "0."

In this embodiment, edges are represented by an adjacency matrix, but the representation is not limited to this. Other representations, such as adjacency lists or kernel matrices (Gaussian kernels, linear kernels) that represent the similarity between nodes as edge strength, may also be used. Furthermore, while the medical knowledge graph in this embodiment is assumed to be an undirected graph where edge arrows do not have a direction, i.e., where causal relationships between medical events do not exist, it may also be a directed graph where edge arrows do have a direction, i.e., where causal relationships between medical events exist.

As shown in Figure 5, the processing circuit 31 maps the medical event information 11 onto the medical knowledge graph 20A to generate the patient graph 20B. Specifically, each node is assigned a feature (hereinafter referred to as a graph feature). A graph feature has a binary value, for example, indicating whether the corresponding medical event is applicable or not. For example, as shown in Figure 5, the medical event "pleural effusion" is applicable ("1"), so the graph feature of the node corresponding to the medical event "pleural effusion" is assigned "1". The medical event "syncope" is not applicable ("0"), so the graph feature of the node corresponding to the medical event "syncope" is assigned "0". In Figure 5, among the nodes 21 included in the patient graph 20B, nodes assigned a graph feature "1" are represented by colored circles, and nodes assigned a feature "0" are represented by white circles.

The graph feature X of each node can be represented as a matrix, as shown in equation (2) below. Each element in the column of the graph feature X corresponds to a node. Each element is assigned a value such as "1" (applicable) or "0" (inapplicable). As mentioned above, the values of the graph feature X are not limited to binary values. The graph feature X can also be referred to as the matrix representation of the medical event information 11.

Furthermore, as will be discussed later, graph features are not limited to whether a medical event is applicable or not; they may also include values indicating the number of occurrences, order of occurrence, and severity of occurrence of medical events.

In the above embodiment, it was assumed that the medical knowledge graph was common to multiple patients, but this embodiment is not limited to this. For example, the medical knowledge graph storage device 2 may store multiple medical knowledge graphs with different connection relationships to edge nodes, that is, with different combinations of numerical values for each element included in the adjacency matrix. In this case, the processing circuit 31 selects the medical knowledge graph to be used for the target patient from the multiple medical knowledge graphs. The selection of the medical knowledge graph may be performed manually by the user via the input interface 33, or it may be selected automatically according to an arbitrary algorithm.

When step SA2 is performed, the processing circuit 31, by implementing the estimation function 313, estimates disease classification information based on the patient graph generated in step SA2 and the trained model (step SA3). The trained model is a machine learning model whose learning parameters have been trained by the medical information learning device 4. The learning parameters are trainable parameters such as weights and biases.

Figure 6 shows an example of the estimation process related to step SA3. As shown in Figure 6, the processing circuit 31 applies the patient graph 20B to the trained model 60 to estimate disease classification information 71. The trained model 60 is a machine learning model whose learning parameters have been trained to take the patient graph 20B as input and output disease classification information 71.

As shown in Figure 6, the trained model 60 has a graph convolutional layer 61, a readout layer 62, and a linear combination layer 63. The graph convolutional layer 61 is a graph convolutional network (GCN) that takes a patient graph 20B before convolution as input and outputs a patient graph 20C after convolution. The graph convolutional layer 61 performs convolution on each node. Specifically, the graph convolutional layer 61 performs a convolution operation based on the graph features of the target node before convolution, the graph features of nodes connected to the target node via edges (hereinafter referred to as adjacent nodes), and the weights for the edges connecting the target node and adjacent nodes, calculating the graph features of the target node after convolution. The weights for the edges are weight parameters included in the weight matrix of the graph convolutional layer 61. Hereafter, the weights for the edges will be referred to as GCN weights.

The graph feature vector X' of the target node after convolution can be expressed as a function Conv based on the graph feature vector X of the target node before convolution, the adjacency matrix A, and the GCN weights W _C, as shown in equation (3) below. The GCN weights W _C are learning parameters trained by the medical information learning device 4. The graph convolution layer 61 can have one or more layers. As will be described later, the graph convolution layer 61 may be divided into filters for each medical event category, each causal direction, and each causal/correlation strength. The convolution process shown in equation (3), etc., is repeated for the number of layers. By performing the above convolution operation on each node included in the patient graph 20B before convolution, the patient graph 20C after convolution is generated. Note that I is the identity matrix and Λ is the order matrix of the graph.

As shown in Figure 6, the graph feature X' after convolution, as a result of the convolution operation, will not have a binary value of "1" or "0," but rather a continuous value representing the degree, such as "0.2" or "0.4."

The readout layer 62 is a network layer that converts the convolutional patient graph 20C into a feature vector 20D. The feature vector 20D is a single column vector with the same number of dimensions as the number of nodes in the convolutional patient graph 20C. That is, the readout layer 62 reads the graph features of each node in the convolutional patient graph 20C and converts them into the feature vector 20D.

The linearly connected layer 63 is a network layer that converts feature vectors 20D into disease classification information 71. The linearly connected layer 63 is also called a multilayer perceptron (MLP). The linearly connected layer 63 performs a classification task or a regression task. The classification task may be a two-class classification that determines the presence or absence of the relevant disease, or a multi-class classification that identifies one disease from multiple disease candidates. It may also be a multi-label classification that allows multiple labels for the same dataset. The regression task outputs a numerical value indicating the probability of each of the relevant one or more diseases being present.

In this embodiment, the linear coupling layer 63 has a classifier that performs multi-class classification, classifying the feature vector 20D into multiple classes corresponding to multiple diseases. In multi-class classification, a softmax function is calculated, outputting the probability that the feature vector 20D belongs to each disease (class). This probability of belonging to each disease is output as disease classification information 71. For example, as shown in Figure 6, the probability of belonging to the disease "heart failure," the probability of belonging to the disease "renal failure," the probability of belonging to the disease "COPD," etc., are output as disease classification information 71. In this case, the disease classification information 71 can be represented in matrix notation Y, as shown in equation (4) below.

The probability Y of each disease among the disease classification information 71 is calculated based on the convolutional graph feature X' and disease weight W _L , as shown in equation (5) below. The disease weight W _L is a learning parameter trained by the medical information learning device 4. The disease weight W _L is set for each disease class for each element (node) of the feature vector 20D. In other words, the disease weight W _L can also be said to be a parameter that represents the degree of influence of each node on the disease in question. The probability Y of each disease is obtained by applying the activation function σ to the weighted addition operation Linear of the graph feature X' by the disease weight W _L. The activation function σ is realized by the softmax function or the like, as described above.

When step SA3 is performed, the processing circuit 31 displays the disease classification information estimated in step SA3 (step SA4) by implementing the display control function 316. In step SA4, the processing circuit 31 displays the disease classification information on a display screen with a predetermined layout.

Figure 7 shows an example of the disease classification information display screen I1. As shown in Figure 7, the display screen I1 has a disease classification information display field I11, a patient selection field I12, a disease selection field I13, a medical event category selection field I14, and a patient graph display field I15. The disease classification information estimated in step SA3 is displayed in the disease classification information display field I11. The patient selection field I12 displays the identifier of the target patient (hereinafter referred to as the patient identifier). The patient identifier can be the patient ID or patient name, etc. The patient identifier displayed in the selection field I12 can be selected by the operator via the input interface 33, etc. Information corresponding to the selected patient identifier is displayed on the display screen I1.

The disease selection field I13 displays the identifier of the target disease (hereinafter referred to as the disease identifier). The disease identifier may be the name or symbol of the disease. The disease identifier displayed in selection field I13 can be selected by the operator via the input interface 33, etc. The probability of the selected disease identifier being present is displayed in display field I11.

The selection field I14 for the medical event category displays a list of the medical event categories to be displayed. The displayed medical event category may be its name or symbol, or it may be a simulated or reduced image of a patient graph related to that medical event category.

The display area I15 of the patient graph displays a visualization graph 81 based on either the patient graph before or after convolution. The generation of the visualization graph 81 by the visualization graph generation function 314 and the display of the visualization graph 81 by the display control function 316 will be explained in detail below. When no particular distinction is made between the patient graph before and after convolution, they will simply be referred to as the patient graph.

First, the processing circuit 31 identifies the medical event category selected in the selection field I14. If no medical event category is selected in the selection field I14, or if the visualization graph 81 is initially displayed, all medical event categories are identified as the selected medical event category. Next, the processing circuit 31 extracts a graph (hereinafter referred to as a partial patient graph) from the patient graph, consisting of nodes belonging to the identified medical event category and the edges connecting those nodes. For example, in Figure 7, since the symptom category is selected in the selection field I14, a partial patient graph consisting of nodes belonging to the symptom category and the edges connecting those nodes is extracted from the patient graph.

Next, the processing circuit 31 simultaneously or sequentially determines patient features and disease impact for each node belonging to the identified medical event category. Patient features are scalar values of the graph features of the target patient for each node. If the graph features are scalar values, the patient features are determined to be those scalar values or values corrected from those scalar values. If the graph features are vectors, first, the graph feature vector is converted into scalar values such as statistical values of the element values of the vector. These statistical values can be, for example, the mean, median, maximum, minimum, or arbitrary quantile of the element values of the graph feature vector. The patient features are determined to be scalar values based on the graph feature vector or values corrected from those scalar values. A machine learning model that calculates scalar values from graph feature vectors may be used to obtain scalar values. Note that patient features may also be determined based on the graph features X before convolution.

The disease impact is numerical information that quantifies the degree of influence of the target disease on each node. The disease impact is determined based on the disease weight _WL of each node. As described above, the disease weight _WL is calculated for each disease class for each element (node) of the feature vector. If the linear connected layer 63 is a single layer, the disease impact is determined to a value that matches the disease weight _WL or a value based on the disease weight _WL . If the linear connected layer 63 is multi-layered, first, the contribution rate of each node is calculated based on the disease weight _WL of each layer. For example, the contribution rate can be calculated by learning the contribution rate for explanation using a global or local explainable model based on the linear connected layer 63. The disease impact is determined to a value that matches the said contribution rate or a value based on the said contribution rate.

Next, the processing circuit 31 determines the display format of the node based on the patient characteristics and disease impact. An example of the display format is described below. As shown in Figure 7, the display color of the node is determined according to the patient characteristics. The display color changes from light red to dark red as the patient characteristics increase. Auxiliary information I16 is displayed to the operator, showing the visual relationship between the patient characteristics and the display color. The display size of the node is determined according to the disease impact. The display size changes from small to large as the disease impact increases. Auxiliary information I17 is displayed to the operator, showing the visual relationship between the disease impact and the display size.

The processing circuit 31 then displays the probability of disease occurrence for the target disease in the display area I11, and displays a visualization graph 81, which visualizes a partial patient graph, in the display area I15. The visualization graph 81 represents a partial patient graph relating to the target patient specified in the selection area I12, the target disease selected in the selection area I13, and the medical event belonging to the medical event category selected in the selection area I14. For example, in Figure 7, the visualization graph 81 relating to the target patient "0001", the target disease "heart failure", and the medical event category "symptoms" is displayed. The processing circuit 31 displays the nodes 82 included in the visualization graph 81 in a display format corresponding to the patient features assigned to the nodes included in the patient graph. The processing circuit 31 also displays the nodes 82 included in the visualization graph 81 in a display format corresponding to the disease impact assigned to the nodes included in the patient graph. The patient features and disease impact are visualized in the node 82. Edges 83 connecting the two related nodes 82 are drawn. Furthermore, each node 82 should be labeled with the name or symbol of the medical event corresponding to that node 82. This will make it possible to understand the meaning of each node.

A visualization graph is a display object that visualizes a patient graph, representing a series of medical events that occurred in a target patient using nodes connected by edges according to their relationships. Each node is highlighted with patient characteristics and disease impact specific to the target patient. By observing such a visualization graph, users can visually understand which medical events were involved in the onset of the target disease. For example, users can understand the medical events related to the "symptoms" of the target disease "heart failure" and the relationships between these medical events, as well as the degree of patient characteristics and disease impact of each medical event. Furthermore, by observing the visualization graph, users can grasp the essential characteristics of the disease and promote an understanding of the disease mechanism.

The user can, at any time, instruct the selection field I14 to change the category of medical events to be displayed. If a change in the category of medical events is instructed, the processing circuit 31 reconstructs and displays a visualization graph based on the nodes of medical events belonging to the changed category and the partial patient graphs connecting those nodes, in the same manner as described above. If a change in the target disease is instructed in the selection field I13, the processing circuit 31 recalculates the disease impact based on the disease weight _WL of the changed target disease, in the same manner as described above, and displays the node 82 in a display format corresponding to the disease impact.

Once step SA4 is performed, the processing circuit 31, through the implementation of the storage function 315, stores the individual circumstances information of the target patient in the medical knowledge graph (step SA5). This individual circumstances information includes, for example, the graph features before and after convolution for each node, disease weights, and disease impact scores. For example, the individual circumstances information is recorded in a database such as a Look Up Table (LUT). Hereinafter, this database will be referred to as the patient individual circumstances database.

Figure 8 shows an example of a patient-specific circumstances database. As shown in Figure 8, the patient-specific circumstances database associates individual circumstances information such as graph features before convolution, graph features after convolution, patient features, disease weight, and disease impact with each node included in the medical knowledge graph in a searchable manner. The patient-specific circumstances database also associates the disease applicability probability with the entire patient-specific information of the target patient. Individual circumstances information is calculated by the processing circuit 31 as described above. Note that disease weight and disease impact are calculated for each node for multiple disease classes. For example, node number "1" records graph feature "X1", graph feature after convolution "X1'", patient feature "F1", disease weight "W _L 11" for the first disease label, disease weight "W _L 12" for the second disease label, ..., disease impact "I11" for the first disease label, disease impact "I12" for the second disease label, ... etc. Furthermore, the probability of disease applicability for the first disease label "Y11", the probability of disease applicability for the second disease label "Y12", etc., are recorded for the entirety of this individual circumstances information. Note that the disease label represents a numerical value indicating applicability to each disease class.

A patient-specific circumstances database (DB) is created for each patient. The patient-specific circumstances database is stored along with the medical knowledge graph by the medical knowledge graph storage device 2. The individual circumstances information recorded in the patient-specific circumstances database may consist of only some of the information described above, or it may include other information as well.

Once step SA5 is completed, the medical information processing by the processing circuit 31 is finished.

As described above, the medical information processing device 3 according to this embodiment has a processing circuit 31. The processing circuit 31 acquires medical event information of the target patient. The processing circuit 31 maps the medical event information to a medical knowledge graph having nodes corresponding to the medical event information and edges representing the relationships between those nodes, thereby generating a patient graph related to the target patient. Based on the patient graph related to the target patient, the processing circuit 31 estimates medical judgment information for the target patient.

The patient graph is a medical knowledge graph that maps the clinical event information of the target patient, and therefore reflects the individual circumstances of each patient. By using such a patient graph, it is possible to estimate medical judgment information, such as the probability of disease occurrence, while considering the individual circumstances of each patient. Furthermore, it becomes possible to provide robust support for medical judgment that is not influenced by specific symptoms or similar patients.

Next, the medical information learning device 4 according to this embodiment will be described.

Figure 9 shows an example of the configuration of the medical information learning device 4. As shown in Figure 9, the medical information learning device 4 includes a processing circuit 41, a memory 42, an input interface 43, a communication interface 44, and a display 45.

The processing circuit 41 includes a processor such as a CPU and a GPU. The processing circuit 41 executes a learning program to generate a pre-trained model used by the medical information processing device 3, thereby realizing the acquisition function 411, mapping function 412, learning function 413, and display control function 414, etc. Note that each function 411-414 is not limited to being realized by a single processing circuit. Multiple independent processors may be combined to form a processing circuit, and each processor may execute a program to realize each function 411-414. Furthermore, each function 411-414 may be a modularized program constituting the learning program, or it may be an individual program. These programs are stored in memory 42.

The processing circuit 41 acquires various types of information through the implementation of the acquisition function 411. For example, the processing circuit 41 acquires multiple learning samples related to multiple patients from the medical information storage device 1. The learning samples correspond to medical information and include medical event information and medical judgment information. The medical event information is used as input samples for machine learning, and the medical judgment information is used as output samples (teaching samples) for machine learning. Note that the medical judgment information included in the learning samples is not medical judgment information estimated by the medical information processing device 3, but rather information regarding medical judgments made by doctors or other medical professionals for the patient in question. Furthermore, the processing circuit 41 acquires a medical knowledge graph from the medical knowledge graph storage device 2.

By implementing the mapping function 412, the processing circuit 41 maps the medical information of the learning sample onto a medical knowledge graph having nodes corresponding to medical events and edges representing the relationships between those nodes, thereby generating a patient graph related to the learning sample.

By implementing the learning function 413, the processing circuit 41 trains a machine learning model based on the patient graph and medical judgment information, and generates a trained model that estimates medical judgment information from the patient graph.

The display control function 414 enables the processing circuit 41 to display various information on the display 45. For example, the processing circuit 41 can display a machine learning settings screen, etc.

Memory 42 is a storage device such as ROM, RAM, HDD, SSD, or integrated circuit memory that stores various types of information. Memory 42 may also be a drive device that reads and writes various types of information to portable storage media such as CDs, DVDs, or flash memory, or to semiconductor memory elements. Furthermore, memory 42 may be located in another computer connected to the medical information learning device 4 via a network. For example, memory 42 stores the medical knowledge graph acquired by the acquisition function 411.

The input interface 43 receives various input operations from the operator, converts the received input operations into electrical signals, and outputs them to the processing circuit 41. Specifically, the input interface 43 is connected to input devices such as a mouse, keyboard, trackball, switch, button, joystick, touchpad, and touch panel display. The input interface 43 outputs electrical signals corresponding to the input operations to the input device to the processing circuit 41. Furthermore, the input device connected to the input interface 43 may also be an input device located on another computer connected via a network or the like.

The communication interface 44 is an interface for sending and receiving various types of information between the medical information system 100 and other computers, such as the medical information storage device 1, the medical knowledge graph storage device 2, the medical information processing device 3, and the medical information display device 5.

The display 45 displays various information according to the display control function 414 of the processing circuit 41. For example, a liquid crystal display, CRT display, organic EL display, plasma display, or any other suitable display can be used as the display 45. A projector may also be provided instead of or in conjunction with the display 45.

Next, we will describe the machine learning process executed by the processing circuit 41 according to the learning program. In the following embodiment, the medical judgment information is assumed to be disease classification information.

Figure 10 shows the flow of machine learning processing. As shown in Figure 10, the processing circuit 41 acquires training samples by implementing the acquisition function 411 (step SB1). The training samples include a combination of medical event information and disease information for a single patient. The medical event information, as described above, is medical information relating to medical events that occurred in the patient. The disease information is information about the disease the patient is suffering from and corresponds to the medical event information for that patient. The disease information has a number of dimensions corresponding to the number of disease classes, and each disease class has a value indicating applicability or non-applicability. The disease information is disease classification information to which values have been artificially assigned, and is a type of disease classification information. The value indicating applicability for each disease class is also called a disease label. For the medical event information for a single patient, a disease label for one disease may be assigned, or multiple disease labels for two or more diseases may be assigned. For the medical event information for a single patient, two or more disease labels may be assigned according to the hierarchical structure in disease classification. The processing circuit 41 may identify disease information based on at least one of the medical event information and the medical ontology.

Once step SB1 is performed, the processing circuit 41 maps the medical event information acquired in step SB1 onto a medical knowledge graph to generate a patient graph (step SB2). Step SB2 is the same as step SA2.

When step SB2 is performed, the processing circuit 41 updates the learning parameters of the machine learning model based on the patient graph generated in step SB2 and the disease information acquired in step SB1 (step SB3).

Figure 11 is a schematic diagram illustrating the update process. As shown in Figure 11, the machine learning model 65 is trained based on supervised learning, using the patient graph 20B as an input sample and disease information 72 as a teaching sample. The machine learning model 65 has a graph convolutional layer 66, a readout layer 67, and a linear combination layer 68. The graph convolutional layer 66 corresponds to the graph convolutional layer 61 of the trained model 60 and is a network layer that performs convolution on the patient graph 20B. The readout layer 67 corresponds to the readout layer 62 of the trained model 60 and is a network layer that converts the patient graph into feature vectors. The linear combination layer 68 corresponds to the linear combination layer 63 of the trained model 60 and is a network layer that converts the feature vectors into disease classification information. Initially, the learning parameters included in the machine learning model 65 are set to arbitrary initial values. The learning parameters include the GCN weights W _C of the graph convolutional layer 66 and the disease weights W _L of the linear combination layer 68.

In the update process, the processing circuit 41 calculates a loss function. The loss function is a function that evaluates the error between the disease classification information, which is calculated by sequentially propagating the patient graph to the graph convolutional layer 66, the readout layer 67, and the linear combination layer 68, and the disease information, which is the teaching sample. The processing circuit 41 updates the learning parameters of the machine learning model 65 so that the loss function is minimized according to an arbitrary optimization method. Specifically, the GCN weights W _C of the graph convolutional layer 66 and the disease weights W _L of the linear combination layer 68 are updated as learning parameters. Any optimization method such as stochastic gradient descent or Adam (adaptive moment estimation) can be used.

When step SB3 is performed, the processing circuit 41 determines whether the stop condition is met (step SB4). The stop condition may be set to, for example, that the number of learning parameter updates has reached a predetermined number, or that the amount of learning parameter updates is below a threshold. If it is determined that the stop condition is not met (step SB4: NO), the processing circuit 41 acquires another learning sample (step SB1). Then, for the other learning samples, the processing circuit 41 sequentially performs patient graph generation (step SB2), learning parameter updates (step SB3), and determination of whether the stop condition is met (step SB4). In this way, the processing circuit 41 repeats patient graph generation (step SB2), learning parameter updates (step SB3), and determination of whether the stop condition is met (step SB4) for each learning sample until the stop condition is met.

Then, if it is determined in step SB4 that the stop condition is met (step SB4: YES), the processing circuit 41 outputs the machine learning model at the time the stop condition was met as the trained model (step SB5). The trained model is transmitted to and stored in the medical information processing device 3, etc.

Once step SB5 is completed, the machine learning processing by the processing circuit 41 is finished.

In the learning process shown in Figure 10, the learning parameters of the machine learning model are updated for each training sample. However, this embodiment is not limited to this. For example, mini-batch learning, in which the learning parameters of the machine learning model are updated for each of multiple training samples, may be performed. Furthermore, if the stopping condition is that the amount of learning parameter updates falls below a threshold, step SB4 may be executed after step SB2, and if it is determined that the stopping condition is not met, step SB3 may be executed.

As described above, the medical information learning device 4 according to this embodiment has a processing circuit 41. The processing circuit 41 acquires a medical knowledge graph having nodes corresponding to medical events and edges representing the relationships between those nodes, as well as medical event information. It generates a patient graph by mapping the medical event information onto the medical knowledge graph, and trains a machine learning model for estimating medical judgment information from the patient graph based on the patient graph and medical judgment information corresponding to the medical event information.

According to the above configuration, it is possible to generate a machine learning model for estimating medical decision-making information from patient graphs, thereby enabling the estimation of highly accurate medical decision-making information.

Next, the medical information display device 5 according to this embodiment will be described.

Figure 12 shows an example configuration of the medical information display device 5. As shown in Figure 12, the medical information display device 5 includes a processing circuit 51, a memory 52, an input interface 53, a communication interface 54, and a display 55.

The processing circuit 51 has a processor such as a CPU and a GPU. The processing circuit 51 executes a display program to display a medical knowledge graph, thereby realizing the acquisition function 511, selection function 512, visualization graph generation function 513, and display control function 514, etc. Note that each function 511 to 514 is not limited to being realized by a single processing circuit. Multiple independent processors may be combined to form a processing circuit, and each processor may execute a program to realize each function 511 to 514. Furthermore, each function 511 to 514 may be a modularized program that constitutes the display program, or it may be an individual program. These programs are stored in memory 52.

The processing circuit 51 acquires various types of information through the implementation of the acquisition function 511. For example, the processing circuit 51 acquires the medical knowledge graph and the patient individual circumstances database from the medical knowledge graph storage device 2. As described above, the patient individual circumstances database is a database that associates individual circumstances information such as graph features, patient features, disease weights, and disease impact for each of multiple nodes.

The selection function 512 enables the processing circuit 51 to select the patient and/or disease to be displayed. The processing circuit 51 may select only the patient, only the disease, or both the patient and the disease. It may also select one patient or multiple patients. Similarly, it may select one disease or multiple diseases.

By implementing the visualization graph generation function 513, the processing circuit 51 generates a visualization graph that visualizes the patient graph based on the medical knowledge graph, according to the patient characteristics and/or disease impact of the patient and/or disease being displayed.

The display control function 514 enables the processing circuit 51 to display various information on the display 55. For example, the processing circuit 51 displays a visualization graph that visualizes the patient graph according to the patient characteristics and/or disease impact of the patient and/or disease being displayed. When a specific patient is selected as the display target, the processing circuit 51 displays the nodes included in the visualization graph in a display format corresponding to the patient characteristics of the specific patient corresponding to that node. When a specific disease is identified as the display target, the processing circuit 51 displays the nodes included in the visualization graph in a display format corresponding to the disease impact of the specific disease corresponding to that node. When a specific combination of patient and disease is identified as the display target, the processing circuit 51 displays the nodes included in the visualization graph in a first display format corresponding to the patient characteristics of the combination corresponding to that node and a second display format corresponding to the disease impact of the combination.

Memory 52 is a storage device such as ROM, RAM, HDD, SSD, or integrated circuit memory that stores various types of information. Memory 52 may also be a drive device that reads and writes various types of information to portable storage media such as CDs, DVDs, or flash memory, or to semiconductor memory elements. Furthermore, memory 52 may be located in another computer connected to the medical information display device 5 via a network. For example, memory 52 stores the medical knowledge graph acquired by the acquisition function 511 and the patient's individual circumstances database.

The input interface 53 receives various input operations from the operator, converts the received input operations into electrical signals, and outputs them to the processing circuit 51. Specifically, the input interface 53 is connected to input devices such as a mouse, keyboard, trackball, switch, button, joystick, touchpad, and touch panel display. The input interface 53 outputs electrical signals corresponding to the input operations to the input device to the processing circuit 51. Furthermore, the input device connected to the input interface 53 may be an input device located on another computer connected via a network or the like.

The communication interface 54 is an interface for sending and receiving various types of information between the medical information system 100 and other computers, such as the medical information storage device 1, the medical knowledge graph storage device 2, the medical information processing device 3, and the medical information learning device 4.

The display 55 displays various information according to the display control function 514 of the processing circuit 51. For example, a liquid crystal display, CRT display, organic EL display, plasma display, or any other display can be used as the display 55. A projector may also be provided instead of or in conjunction with the display 55.

Next, the medical knowledge graph display process, executed by the processing circuit 51 according to the display program, will be explained. The medical knowledge graph display process is a process for viewing individual circumstances information stored in the medical knowledge graph by the medical information processing device 3, along with the medical knowledge graph itself.

Figure 13 shows the flow of the medical knowledge graph display process. Note that at the start of Figure 13, the processing circuit 51 has already acquired the medical knowledge graph and the patient individual circumstances database for multiple patients from the medical knowledge graph storage device 2. The medical knowledge graph and the patient individual circumstances database for multiple patients are stored in memory 52.

As shown in Figure 13, the processing circuit 51 displays the initial screen (step SC1) by implementing the display control function 514. The initial screen is displayed on the display 55.

As shown in Figure 14, the initial screen I2 has a patient selection field I12, a disease selection field I13, a medical event category selection field I14, and a patient graph display field I15. The selection field I12 displays selectable patient identifiers for multiple patients whose individual circumstances information is stored in the medical knowledge graph. The selection field I12 may also display selectable patient identifiers representing statistical groups of patients. As statistical groups of patients, various classifications can be used as appropriate, such as the entire group of patients whose individual circumstances information is stored in the medical knowledge graph, gender classifications (male, female, etc.), age classifications (60s, 70s, etc.), etc. Figure 14 shows "Total" as an example, representing all patients.

The selection field I13 displays disease identifiers for multiple disease types, allowing selection. Alternatively, the selection field I13 may also display disease identifiers representing statistical populations related to disease types. As the statistical population related to disease types, the entire range of diseases that can be classified as disease classification information can be used as appropriate. Furthermore, the disease identifiers may be displayed hierarchically. For example, as shown in Figure 14, the upper layer "heart failure" and its lower layers "HFrEF" and "HFpEF" are displayed. Since this is the initial screen I2, the patient graph is not displayed in the display field I15.

When step SC1 is performed, the processing circuit 51 selects the patient and/or disease to be displayed by implementing the selection function 512 (step SC2). When selecting a patient, the user selects the patient identifier displayed in the selection field I12 via the input interface 53. When selecting a disease, the user selects the disease identifier displayed in the selection field I13 via the input interface 53. Both a patient and a disease may be selected, or only a patient or only a disease may be selected.

Furthermore, in step SC2, the processing circuit 51 selects the medical event category to be displayed by implementing the selection function 512. For example, the user can select any medical event category from among the multiple medical event categories displayed in the selection field I14 via the input interface 53. The selected medical event category may be one or multiple categories.

When step SC2 is performed, the processing circuit 51, by implementing the visualization graph generation function 513, generates a visualization graph corresponding to the patient and/or disease and medical event category selected in step SC2 (step SC3). For example, if a patient and disease are selected, the processing circuit 51 reads individual circumstances information from the patient individual circumstances DB corresponding to the selected target patient and assigns it to the medical knowledge graph. Specifically, the processing circuit 51 reads graph features and patient features for each node and assigns them to the node, and also reads disease weights and disease impact levels corresponding to the selected target disease and assigns them to the node. As described above, the graph features and patient features are based on the patient graph before convolution and the patient graph after convolution, and the user can arbitrarily set from among these.

After assigning individual circumstances information to each node, the processing circuit 51 extracts nodes corresponding to medical events belonging to the selected medical event category and the edges connecting those nodes. If one medical event category is selected, nodes corresponding to medical events belonging to that single medical event category are extracted. If multiple medical event categories are selected, nodes corresponding to medical events belonging to those multiple medical event categories are extracted. Next, the processing circuit 51 extracts a partial patient graph containing the extracted nodes and edges, and generates a visualization graph by visualizing the extracted partial patient graph. At this time, the processing circuit 51 sets the display format of each node in the visualization graph according to the individual circumstances information assigned to that node. For example, the display color of a node is set according to patient features or graph features, and the display size is set according to disease weight or disease impact.

Furthermore, the procedure for generating the visualization graph is not limited to the procedure described above. Alternatively, one could first extract partial patient graphs corresponding to medical events belonging to the selected medical event category, and then assign individual circumstances information to the nodes.

Once step SC3 is completed, the processing circuit 51, through the implementation of the display control function 514, displays the visualization graph generated in step SC3 (step SC4). The visualization graph is displayed on the display 55.

As shown in Figure 15, the visualization graph generated in step SC3 is displayed in display area I15 of display screen I3. The nodes of the visualization graph are displayed in a display color and size corresponding to the patient's patient characteristics and disease impact. Display area I15 may also display display area I11, which shows the disease probability for the target patient. The disease probability can be read from the target patient's individual circumstances database. Figure 15 shows, as an example, a visualization graph for target patient "00001", target disease "HFrEF", and medical event category "Symptoms".

As described above, the visualization graph is a display object that visualizes a patient graph, representing a series of medical events that occurred in a target patient using nodes connected by edges according to their relationships. Each node is highlighted with patient characteristics and disease impact values unique to the target patient. By observing such a visualization graph, users can visually understand which medical events were involved in the onset of the target disease. Furthermore, by observing the visualization graph, users can grasp the essential characteristics of the disease and improve their understanding of the disease mechanism.

Once step SC4 is completed, the medical knowledge graph display processing by processing circuit 51 is finished.

The above medical knowledge graph display process is merely an example and is not limited to it. For instance, in the above embodiment, one patient is selected, but all patients may be selected.

Figure 16 shows an example of a visualization graph displaying data for all patients. As shown in Figure 16, the display area of screen I4 displays a visualization graph showing data for all patients. An example of the procedure for generating a visualization graph for all patients is as follows:

Assume that both the entire patient population and the specific disease are selected in step SC2. In this case, the processing circuit 51 reads the individual patient data database for all patients, calculates statistical values for each node based on the disease impact of the specific disease for all patients, and assigns the calculated statistical values to the node. The processing circuit 51 then displays each node in a display format, such as display size, according to the assigned statistical values. If the entire patient population is selected, patient features are not assigned to nodes. Therefore, the nodes are displayed in a display format corresponding only to the disease impact. In this case, the auxiliary information I16 showing the relationship between patient features and display color does not need to be displayed.

Visualization graphs for the entire patient make it possible to understand the overall impact of the disease on individual medical events. By displaying visualization graphs for the entire patient and visualization graphs for specific patients alternately or in parallel, it is also possible to understand the specific characteristics of the disease impact on each medical event for a particular patient.

As another option, the entire disease may be selected. Figure 17 shows an example of a visualization graph displaying the entire disease. As shown in Figure 17, the display area of display screen I5 shows a visualization graph that visualizes the patient graph for the entire disease. An example of the procedure for generating a visualization graph for the entire disease is as follows.

In step SC2, a specific patient and the entire disease are selected. In this case, the processing circuit 51 reads the patient's individual circumstances database and assigns the patient's characteristics to each node. The processing circuit 51 then displays each node using a display format such as display color corresponding to the assigned patient characteristics. If the entire disease is selected, the disease impact is not assigned to the nodes. Therefore, the nodes are displayed using a display format corresponding only to the patient characteristics. When the entire disease is selected, as shown in Figure 17, it is preferable to include the disease probability next to each disease identifier in the selection field I13. For example, it would be displayed as "Heart Failure 93%", "Renal Failure 4%", "COPD 3%", etc. The disease probability can be read from the patient's individual circumstances database.

Visualization graphs of the entire disease make it possible to understand the patient characteristics of a specific patient across the entire disease in relation to individual clinical events. By displaying visualization graphs of the entire disease and visualization graphs of a specific disease alternately or in parallel, it is also possible to understand the specificity of patient characteristics for each clinical event within that specific disease.

As described above, the medical information display device 5 according to this embodiment has a processing circuit 51. The processing circuit 51 stores a medical knowledge graph having nodes corresponding to medical events and edges representing the relationships between nodes. Each node in the medical knowledge graph is assigned patient characteristics and/or disease impact levels for each patient. The processing circuit 51 identifies the patient and/or disease to be displayed. The processing circuit 51 displays a visualization graph that visualizes the medical knowledge graph according to the patient characteristics and/or disease impact levels related to the patient and/or disease to be displayed.

A visualization graph is a display object that visualizes a patient graph, representing a series of medical events that occurred in a target patient using nodes connected by edges according to their relationships. Each node is highlighted with patient characteristics and disease impact values unique to the target patient. By observing such a visualization graph, users can visually understand which medical events were involved in the onset of the target disease. Furthermore, by observing the visualization graph, users can grasp the essential characteristics of the disease and improve their understanding of the disease mechanism.

(Examples of application)
The following describes various application examples of this embodiment. In the following description, components having substantially the same function as those in this embodiment are denoted by the same reference numerals and described repeatedly only when necessary.

(Application Example 1)
In some of the embodiments described above, each node is assigned a graph feature indicating whether or not a medical event is applicable. However, this embodiment is not limited to this. In Application Example 1, each node is further assigned graph features including, in addition to whether or not a medical event is applicable, the order of occurrence, the number of occurrences, and/or the degree of occurrence. The order of occurrence refers to the order in which the medical event occurred. The number of occurrences is the number of times the medical event occurred. The degree of occurrence is the severity of the medical event. The order of occurrence, the number of occurrences, and/or the degree of occurrence are included in the medical event information. Alternatively, the order of occurrence, the number of occurrences, and/or the degree of occurrence may be calculated by analyzing the medical event information.

Figure 18 is a conceptual diagram of the assignment of occurrence order, frequency, and severity. As shown in Figure 18, each node in the patient graph is assigned values as graph features that represent the occurrence order, frequency, and severity of medical events. For example, the occurrence order may be assigned a numerical value that represents the relative occurrence order among multiple medical events. As another example, in order to represent the occurrence order and occurrence interval with a single value, a numerical value may be assigned relative to the dates in between, based on a numerical value indicating the start date (e.g., 0) and a numerical value indicating the end date (e.g., 1) of the period from which the medical events are extracted. For the frequency, for example, a numerical value representing the frequency of occurrence itself may be assigned. As another example, in order to consider the difference in the frequency of occurrence between medical events, the numerical value of the frequency of occurrence itself may be normalized using a standard (mean or maximum value) for each medical event. The severity of occurrence is assigned numerically. For example, if the medical event is a symptom, it represents the intensity of the symptom; if the medical event is a test, it represents the test result; if the medical event is drug treatment, it represents the dosage of the drug; and if the medical event is a treatment response, it represents the intensity of the treatment response. It is not necessary to assign all of the occurrence order, frequency, and severity; one or two of these may be assigned. Although not shown in Figure 18, each node is also assigned a value indicating whether the medical event is applicable or not.

In Application Example 1, the graph features of each node are given as multidimensional features consisting of whether a medical event is applicable or not, the order of occurrence, the number of occurrences, and/or the degree of occurrence. By training a machine learning model using a patient graph to which such multidimensional graph features are assigned, it becomes possible to generate a trained model that considers the order of occurrence, the number of occurrences, and/or the degree of occurrence of medical events. By considering the order of occurrence, the number of occurrences, and/or the degree of occurrence of medical events, it is expected that the accuracy of estimating medical judgment information such as the probability of disease applicability will improve.

(Application example 2)
In the above-mentioned embodiments, the period of medical event information mapped to the medical knowledge graph was not particularly limited. The processing circuit 41 in Application Example 2 changes the period of medical event information to be mapped according to the type of disease to be predicted. Specifically, multiple machine learning models are prepared according to the type of disease to be predicted. Examples of disease types to be predicted include acute diseases and chronic diseases.

Figure 19 schematically shows the input and output of two machine learning models 65A and 65B related to Application Example 2. As shown in Figure 19, the machine learning models 65A and 65B related to Application Example 2 include a machine learning model 65A for acute diseases and a machine learning model 65B for acute diseases. Machine learning model 65A is input with a patient map onto which short-term medical event information 11A is mapped. Short-term medical event information 11 includes medical event information that occurred in a relatively short period of time, tracing back from the disease diagnosis. In Figure 19, medical event categories are represented by the shape of nodes. For example, circles represent symptom categories, triangles represent treatment categories, and squares represent response categories. Machine learning model 65A is trained based on supervised learning using the patient map and disease information for acute diseases. More specifically, the learning parameters of machine learning model 65A are trained based on supervised learning using the patient map as an input sample and disease information for acute diseases as a teaching sample. This generates a trained model that estimates disease classification information for acute illnesses by inputting a patient map onto which short-term medical event information 11A has been mapped.

Similarly, the machine learning model 65B is input with a patient map that maps long-term clinical event information 11B. This long-term clinical event information 11B includes clinical event information that occurred over a relatively long period, tracing back from the disease diagnosis. The machine learning model 65B is trained based on supervised learning using this patient map and chronic disease information. More specifically, the learning parameters of the machine learning model 65B are trained based on supervised learning using the patient map as input samples and chronic disease information as teaching samples. This generates a trained model that estimates chronic disease classification information by inputting a patient map that maps long-term clinical event information 11B.

During disease estimation, the processing circuit 31 determines the time period of the medical event information to be mapped, depending on the type of disease being classified. For example, if the disease is an acute disease, the processing circuit 31 determines that the time period of the medical event information to be mapped is short, and if the disease is a chronic disease, it determines that the time period of the medical event information to be mapped is long. The processing circuit 31 then extracts medical event information for a time period appropriate to each trained model from the patient's medical event history. Specifically, it extracts short-term medical event information from the patient's medical event history, maps this short-term information to a medical knowledge graph to generate a patient map, and inputs this patient map into a trained model for acute diseases to estimate the disease classification information for acute diseases. Furthermore, the processing circuit 31 extracts long-term medical event information from the patient's medical event history, maps this long-term information to a medical knowledge graph to generate a patient map, and inputs this patient map into a trained model for chronic diseases to estimate the disease classification information for chronic diseases.

According to Application Example 2, the time period of clinical event information mapped onto the medical knowledge graph differs between the machine learning models for acute diseases and those for chronic diseases. This is addressed by generating separate pre-trained models for disease types with different time periods of clinical event information to consider. Using such pre-trained models makes it possible to improve the accuracy of disease classification information estimation.

(Application example 3)
In some of the above embodiments, the convolution process performed by the graph convolution layer does not distinguish between edge relationship types. The graph convolution layer in Application Example 3 switches parameters such as the GCN weights of the convolution process according to the edge relationship type of the edge to be processed connected to the node to be processed. Specifically, the edge relationship type is the direction of causality, the strength of causality, and/or the strength of correlation between medical events related to the edge to be processed. Alternatively, the edge relationship type may be a combination of the category of medical events of the node to which the edge to be processed is connected and the category of medical events of the adjacent node adjacent to the node to which the edge to be processed is connected. In Application Example 3 shown below, the edge relationship type is assumed to be a combination of medical event categories.

Figure 20 schematically illustrates the convolution process performed by the graph convolution layer 66 in Application Example 3. As shown in Figure 20, the patient graph 20B contains multiple nodes belonging to multiple medical event categories. The graph convolution layer 66 sequentially executes multiple convolution processes corresponding to each of the multiple medical event categories. This generates the patient graph 20C after convolution. Specifically, the graph convolution layer 66 has multiple filter layers corresponding to each of the multiple convolution processes. Each filter layer has parameters such as GCN weights trained for each different medical event category. For example, the first filter layer is trained only for nodes belonging to the findings category, and the second filter layer is trained only for nodes belonging to the symptom category. The calculation results from each filter layer are integrated and output.

The convolution process that considers edge relationship types is described by the following equation (6) of the R-GCN (Relational Graph Convolutional Network). As shown in equation (6), the graph feature X' after convolution of the node to be processed is calculated by the sum of the first and second terms. The first term is the product of the graph feature _Xj of the neighboring node j before convolution and the weight matrix _Wr of edge relationship type r before convolution. More specifically, the first term is the sum of the products of the graph feature _Xj , the weight matrix _Wr , and the normalization constant 1/c _i,r for all combinations of edge relationship type r and neighboring node j. Note that N _i ^r in equation (6) represents the index of the neighboring node of edge relationship type r of node i. The second term is calculated as the sum of the product of the graph feature X _i of the node to be processed before convolution and the weight matrix _W0 of the self-loop before convolution. As shown in equation (6), a weight matrix _Wr is determined for each edge relationship type r, and the product of the graph feature _Xj and the weight matrix _Wr is calculated. Note that the weight matrices _Wr and _W0 are the GCN weights of the graph convolutional layer and are an example of learning parameters.

As described above, a graph convolutional layer having multiple filter layers is designed according to the edge relationship type. Following the graph convolutional layer, a readout layer and a linear combination layer are sequentially connected, similar to some of the embodiments described above. The multiple filter layers are provided so as to share the subsequent readout layer and linear combination layer. Furthermore, the multiple filter layers may share a portion of the graph convolutional layer.

The machine learning model in Application Example 3, like the several examples described above, is trained based on supervised learning using patient graphs and disease information. This makes it possible to generate a trained model with multiple filter layers corresponding to edge relationship types. Furthermore, the graph convolutional layer in Application Example 3 may have its parameters, such as GCN weights, switched depending on the type of medical judgment information. The types of medical judgment information are, as described above, disease classification information, prognosis prediction information, and severity classification information. Also, if the medical judgment information is disease classification information, the parameters, such as GCN weights, of the graph convolutional layer may be switched depending on the type of disease.

According to Application Example 3, by dividing the filter layer according to the edge relationship type, it becomes possible to train the learning parameters according to the edge relationship type. This improves the accuracy of graph convolution by the graph convolution layer. Consequently, the accuracy of estimating medical judgment information such as disease probability is improved.

(Application example 4)
In some of the embodiments described above, disease classification information was assumed to be estimated as medical judgment information. However, medical judgment information is not limited to disease classification information; it may also be prognostic information such as patient survival time, or severity classification information such as cancer staging. Furthermore, medical judgment information may be a combination of at least two types of information: disease classification information, prognostic information, and severity classification information.

Figure 21 shows an example of the input and output of linearly coupled layers 68A and 68B in Application Example 4. As shown in Figure 21, the machine learning model in Application Example 4 has two linearly coupled layers 68A and 68B. The first linearly coupled layer 68A receives a feature vector 20D output from a preceding readout layer (not shown) and outputs disease classification information. The second linearly coupled layer 68B receives the same feature vector 20D and outputs prognosis prediction information.

In the machine learning process, a machine learning model comprising a graph convolutional layer 66, a readout layer 67, a first linear combination layer 68A, and a second linear combination layer 68B is trained based on multi-task learning, with the first linear combination layer 68A receiving disease classification information as teaching samples, the second linear combination layer 68B receiving prognosis prediction information as teaching samples, and a patient graph (not shown) as the input sample. Through multi-task learning, the first linear combination layer 68A is trained to output disease classification information by inputting a feature vector 20D, and the second linear combination layer 68B is trained to output prognosis prediction information by inputting a feature vector 20D. Therefore, a trained model is generated that takes a patient graph as input and outputs disease classification information and prognosis prediction information.

According to Application Example 4, a machine learning model is trained through multi-task learning based on a patient graph and two or more pieces of information from disease classification information, prognosis prediction information, and severity classification information. This makes it possible to estimate two or more types of medical judgment information from a single patient graph. Therefore, the usefulness of the trained model is improved.

(Application Example 5)
The individual elements of application examples 1 to 4 above can be combined in any way. The machine learning model related to application example 5 is constructed by combining the individual elements of application examples 1 to 4 above.

Figure 22 schematically illustrates the machine learning model related to Application Example 5. As shown in Figure 22, the trained model includes a machine learning model for acute diseases and a machine learning model for chronic diseases. Each machine learning model has a different time period for the medical event information mapped onto the input patient graph. The machine learning model for acute diseases receives a patient graph with short-term medical event information mapped onto it. The machine learning model for chronic diseases receives a patient graph with long-term medical event information mapped onto it. The patient graph is assigned graph features such as disease applicability (applicable or not), the order of occurrence, frequency, and severity of medical events. Each machine learning model's graph convolutional layer has different filter layers for each edge relationship type. Each model has two linearly coupled layers. The first linearly coupled layer takes a feature vector as input and outputs disease classification information, while the second linearly coupled layer takes a feature vector as input and outputs prognosis prediction information.

The machine learning model in Application Example 5 is trained based on multi-task learning, where the training samples for the first linear combination layer are disease classification information, and the training samples for the second linear combination layer are prognosis prediction information. This makes it possible to generate a trained model that outputs disease classification information and prognosis prediction information for both acute and chronic diseases, while performing graph convolution according to the edge relationship type.

(Application example 6)
Figure 23 is a diagram illustrating the overview of graph features related to Application Example 6. As shown in Figure 23, each node in the patient graph 20B is assigned clinical event information such as applicable/unapplicable features, temporal features, and/or spatial features as graph features. Applicable/unapplicable features represent the degree to which a clinical event corresponds to or does not correspond to the clinical event corresponding to that node. Temporal features are information about the time when the clinical event occurred. Specifically, temporal features include the date and time of occurrence, the order of occurrence, and the number of occurrences. Spatial features are information about the location where the clinical event occurred. Specifically, spatial information includes the location and site of occurrence of the clinical event. The location of occurrence is sensor information such as GPS (Global Positioning System) that represents the point where the clinical event occurred. The site of occurrence is the address and name of the medical institution, department, home, or hospital where the clinical event was diagnosed.

The graph features related to Application Example 6 below shall include applicable/unapplicable features, temporal features, and spatial features.

The processing circuit 31 in Application Example 6 generates a patient graph 20B for the target patient by mapping medical event information onto a medical knowledge graph through the implementation of a mapping function 312. Here, the medical event information includes applicable/inapplicable features, temporal features, and spatial features for each medical event. This generates a patient graph 20B in which applicable/inapplicable features, temporal features, and spatial features are assigned to each node.

Subsequently, the processing circuit 31, through the implementation of the estimation function 313, applies the patient graph 20B, which has nodes assigned with applicable/inapplicable features, temporal features, and spatial features, to the trained model 60 to estimate medical judgment information such as disease classification information 71. According to application example 6, it becomes possible to estimate medical judgment information by considering the applicable/inapplicable features, temporal features, and spatial features of each medical event. Note that each node does not need to be assigned all three types of features; it may be assigned only two or one of these types.

(Application Example 7)
Figure 24 is a diagram illustrating the overview of graph features related to Application Example 7. As shown in Figure 24, each node in the patient graph 20B is assigned time-series medical event information, consisting of multiple medical event information items with different occurrence times, as a graph feature. The values of the medical event information change over time. For example, whether a medical event is applicable or not changes over time. Occurrence time information is assigned as metadata to the medical event information. Occurrence time information is a timestamp of the medical event information, and for example, the date and time of diagnosis or the date and time of recording of the medical event is assigned. For example, as shown in Figure 24, each node is assigned medical event information at time t, medical event information at time t+1, and medical event information at time t+2 as graph features.

The processing circuit 31 in Application Example 7 estimates medical judgment information based on time-series graph features.

Figure 25 shows an example of the estimation process for medical judgment information related to Application Example 7. As shown in Figure 25, the trained model 60A related to Application Example 7 has a graph convolutional layer 61, a readout layer 62, and an RNN (Recurrent Neural Network) layer 64. The RNN layer 64 outputs disease classification information 71 from time-series graph features 20D.

As shown in Figure 25, the patient graphs to be processed are patient graph 20B0, patient graph 20B1, and patient graph 20B2, which are assigned graph features at time t and t+1, respectively. The processing circuit 31 inputs patient graphs 20B0, 20B1, and 20B2 to the graph convolution layer 61 and outputs the convolutional patient graphs corresponding to patient graph 20B0, patient graph 20B1, and patient graph 20B2. Each convolutional patient graph is then input to the readout layer 62 to output the feature vectors 20D0, 20D1, and 20D2, respectively, which are assigned graph features at time t and t+1, respectively. Feature vector 20D0 is the vector representation of the graph features at time t, feature vector 20D1 is the vector representation of the graph features at time t+1, and feature vector 20D2 is the vector representation of the graph features at time t+2.

As shown in Figure 25, the processing circuit 31 inputs feature vectors 20D0, 20D1, and 20D2 to the RNN layer 64 and outputs a single disease classification information 71. By utilizing the RNN layer 64, it becomes possible to obtain the probability of various diseases by using sequential clinical event information from multiple time points.

(Application example 8)
In Application Example 7, time-series graph features are assigned to each node. In Application Example 8, the patient graph itself changes over time.

The processing circuit 41 of the medical information learning device 4 in Application Example 8 continuously acquires learning samples (medical information) with different occurrence times, even after a trained model has been generated. As described above, the learning samples are combinations of medical event information and disease information. Based on the acquired medical information, the processing circuit 41 continuously learns the learning parameters of the machine learning model periodically or irregularly. Continuous learning generates trained models with time series at different occurrence times. The learning parameters include the adjacency matrix and/or GCN weights. The patient graph before convolution at each time point is applied to the trained model at each time point, generating the patient graph after convolution at each time point. This makes it possible to generate a time series patient graph in which the graph features of each node and the connection relationships between nodes change over time.

The pre-trained models for each occurrence time that make up the time-series pre-trained model are trained according to the processing procedure shown in Figure 10 above, based on the training samples acquired up to that time. In this case, in order to suppress large changes in the characteristics of the patient graph before and after updating the adjacency matrix A and/or GCN weights W, a regularization term may be added to the loss function as shown in equations (7) and (8) below. Equation (7) is the normalization term L _reg related to the update of the adjacency matrix A, and is expressed by the Kullback-Leibler divergence between the adjacency matrix A at the processing reference time and the adjacency matrix A' at other time points. Equation (8) is the normalization term L _reg related to the update of the GCN weights W ^(l) , and is expressed by the Kullback-Leibler divergence between the GCN weights W ^(l) at the processing reference time and the GCN weights W ^'(l) at other time points.

(Application example 9)
In some of the above embodiments, spatial information, such as location features, is assigned to each node of the patient graph as a graph feature. In application example 9, the concept of space is added to the patient graph itself.

Figure 26 schematically illustrates the relationship between patient graphs and spatial information in Application Example 9. As shown in Figure 26, spatial information concerning the patient in question is assigned to each patient graph. Spatial information includes both locational and biological information. Locational information may include sensor information such as GPS indicating the patient's current location, or it may include the location of the medical institution the patient visits, the patient's home address, or the hospital room where the patient is hospitalized. Biological information includes the patient's kinship, medical history, and gene sequence information. By embedding spatial information into patient graphs, it is possible to construct a network (hereinafter referred to as a patient graph network) in which multiple patient graphs are arranged according to the presence or degree of spatial information relationships.

Figure 27 illustrates the concept of the patient graph network 200. As shown in Figure 27, the patient graph network 200 is composed of multiple patient graphs linked according to the presence or degree of spatial information relationships. Figure 27 illustrates four patient graphs 201, 202, 203, and 204, but the number of patient graphs is not particularly limited as long as there are two or more.

Patient graphs connected by edges (for example, patient graph 201 and patient graph 202) have a spatial information relationship with each other. Conversely, patient graphs not connected by edges (for example, patient graph 201 and patient graph 204) do not have a spatial information relationship with each other. Furthermore, the distance between patient graphs connected by edges represents the degree of the spatial information relationship. The processing circuit 31 can evaluate the degree of the spatial information relationship based on location information, for example, the home addresses of both graphs or the distance between their hospital rooms. The processing circuit 31 can also evaluate biological information based on the degree of kinship between the two graphs, the medical relationship of their medical histories, and the gene sequence agreement rate. The processing circuit 31 can determine the presence or absence of a spatial information relationship based on a comparison between the degree evaluated by the above methods and a threshold. Edges may be formed based on a comprehensive evaluation of the above elements of spatial information, or they may be formed for each element individually.

By assigning spatial information to patient graphs, it becomes possible to easily search for patient graphs whose spatial information is close to that of the target patient. For example, the processing circuit 31 can extract patient graphs connected to the target patient's patient graph by edges as patient graphs of patients whose spatial information is close to that of the target patient.

Furthermore, spatial information is not limited to including both locational and biological information; it may include only one of these two.

(Application Example 10)
Application example 10 utilizes spatial information related to application example 9. The processing circuit 31 of the medical information processing device 3 in application example 10 convolves the graph features of a patient graph of a patient different from the target patient onto the patient graph of the target patient.

Figure 28 is a diagram illustrating the overview of graph features related to Application Example 10. As shown in Figure 28, each node in the patient graph 20B is assigned clinical event information as graph features, including applicable/unapplicable features, temporal features, spatial features, and/or spatially nearby patient features. Applicable/unapplicable features, temporal features, and spatial features are as explained in Application Example 6. Spatially nearby patient features are clinical event information of other patients whose spatial information is close to the target patient. For example, the node for the clinical event "headache" will be assigned applicable/unapplicable features, temporal features, and spatial features of the target patient, as well as applicable/unapplicable features, temporal features, and spatial features of the target patient's father and/or mother.

The processing circuit 31 in Application Example 10, through the implementation of the mapping function 312, maps the medical event information related to Application Example 10 onto a medical knowledge graph to generate a patient graph 20B of the target patient. The medical event information related to Application Example 10 includes applicable/inapplicable features, temporal features, spatial features, and spatially nearby patient features for each medical event. The applicable/inapplicable features, temporal features, and spatial features are the medical event information of the target patient. The spatially nearby patient features are the medical event information of other patients whose spatial information is close to the target patient. That is, the processing circuit 31 in Application Example 10, through the implementation of the mapping function 312, maps the medical event information of the target patient and the medical event information of other patients as node features to each node. The implementation of the mapping function 312 generates a patient graph 20B in which applicable/inapplicable features, temporal features, spatial features, and spatially nearby patient features are assigned to each node.

Subsequently, the processing circuit 31, through the implementation of the estimation function 313, applies the patient graph 20B, which has nodes assigned with applicable/inapplicable features, temporal features, spatial features, and spatially nearby patient features, to the trained model 60 to estimate medical judgment information such as disease classification information 71. According to application example 10, it becomes possible to estimate medical judgment information by considering the applicable/inapplicable features, temporal features, spatial features, and spatially nearby patient features for each medical event. This makes it possible to estimate medical judgment information considering not only the medical information (graph features) of the target patient but also the medical information of patients whose spatial information is spatially close to the target patient. Note that each node does not need to be assigned all of the applicable/inapplicable features, temporal features, spatial features, and spatially nearby patient features; it may be assigned only the spatially nearby patient features and any two or one of the applicable/inapplicable features, temporal features, and spatial features.

(Application Example 11)
In some of the embodiments described above, the processing circuit 31 displays a visualization graph based on a partial patient graph, which includes only nodes and edges belonging to some of the medical event categories, including symptoms, physical findings, laboratory findings, treatment, treatment response, and side effects. However, as shown in Figure 2, the processing circuit 31 may also display a visualization graph based on the entire patient graph, which covers all medical event categories. In this case, the processing circuit 31 may display the visualization graph in a way that allows each category to be distinguished by visual effects such as color. This makes it possible to get an overview of the entire patient graph.

(Other examples)
Several further embodiments can be added to or replaced with the above embodiments. For example, transfer learning may be performed on a machine learning model. First, the learning parameters of a machine learning model that estimates disease classification information for a first disease (e.g., an acute disease) may be trained, and then some of these learning parameters may be set in a machine learning model that estimates disease classification information for a second disease (e.g., a chronic disease), and the remaining learning parameters may be trained. This is useful, for example, when there are few training samples for the second disease. The second disease to which transfer learning is performed should preferably have some medical commonality or similarity with the first disease from which the transfer originated.

Regularization may be considered in machine learning processing. This is achieved by adding an arbitrary regularization term to the loss function. Suitable regularization terms include, for example, a penalty term that penalizes the magnitude of the learning parameter norm, or a penalty term that penalizes based on an index of the independence of network parameters between diseases.

In the machine learning process described above, the adjacency matrix, which defines the relationships between edges, was not treated as a learning parameter. However, this embodiment is not limited to this, and the adjacency matrix may also be treated as a learning parameter and trained in the machine learning process.

The above machine learning model may be equipped with additional modules such as an attention mechanism. For example, an attention mechanism may be provided in parallel between the output layer of the readout layer and the input layer of the linear combination layer. The attention mechanism outputs an attention mask indicating the degree of emphasis for each node. The linear combination layer estimates medical judgment information based on the attention mask and feature vectors. The inclusion of the attention mechanism improves the accuracy of the medical judgment information estimation because the feature portions of the patient graph that are useful for estimating medical judgment information are emphasized.

In the above embodiment, graph convolutional layers such as GCNs and R-GCNs were used as the model for processing the patient graph. However, this embodiment is not limited to this, and Boltzmann machines or their applications using Markov random fields (MRFs) or conditional random fields (CRFs) may also be used as the model for processing the patient graph.

The teaching sample was assumed to be disease information obtained from clinical information or disease information obtained from a medical ontology. However, the teaching sample may contain both disease information obtained from clinical information and disease information obtained from a medical ontology. In this case, the machine learning model may output a multi-label classification that outputs disease information obtained from clinical information and disease information obtained from a medical ontology.

According to at least one embodiment described above, the accuracy of medical judgments can be improved.

In the above description, the term "processor" refers to circuits such as CPUs, GPUs, or Application Specific Integrated Circuits (ASICs), programmable logic devices (e.g., Simple Programmable Logic Devices (SPLDs), Complex Programmable Logic Devices (CPLDs), and Field Programmable Gate Arrays (FPGAs)). The processor achieves its function by reading and executing a program stored in a memory circuit. Alternatively, instead of storing the program in a memory circuit, the processor may be configured to directly incorporate the program into its circuitry. In this case, the processor achieves its function by reading and executing the program incorporated into the circuitry. Furthermore, instead of executing a program, the processor may achieve the function corresponding to the program through a combination of logic circuits. In this embodiment, each processor is not limited to being configured as a single circuit; multiple independent circuits may be combined to form a single processor, and its function may be achieved through this combination. Furthermore, the multiple components shown in Figures 1, 3, 9, and 12 may be integrated into a single processor to realize their functions.

While several embodiments have been described, these embodiments are presented as examples only and are not intended to limit the scope of the invention. These embodiments can be implemented in various other forms, and various omissions, substitutions, modifications, and combinations of embodiments are possible without departing from the spirit of the invention. These embodiments and their variations are included within the scope and spirit of the invention, as well as within the scope of the claims and its equivalents.

1 Medical information storage device 2 Medical knowledge graph storage device 3 Medical information processing device 4 Medical information learning device 5 Medical information display device 31 Processing circuit 32 Memory 33 Input interface 34 Communication interface 35 Display 41 Processing circuit 42 Memory 43 Input interface 44 Communication interface 45 Display 51 Processing circuit 52 Memory 53 Input interface 54 Communication interface 55 Display

Claims (23)

An acquisition unit that acquires medical information regarding the medical events of the target patient,
A mapping unit generates a second graph relating to the target patient by mapping the medical information onto a first graph having nodes corresponding to medical events and edges representing the relationships between those nodes.
An estimation unit that estimates the medical judgment information for the target patient based on the second graph relating to the target patient, using a trained model which is a machine learning model trained to take the second graph as input and output medical judgment information,
A medical information processing device equipped with the following features. The medical information processing device according to claim 1, wherein the medical event includes an event belonging to at least one of the following categories: symptoms, physical findings, laboratory findings, treatment, treatment response, and side effects. The medical information processing apparatus according to claim 1, wherein the trained model comprises a graph convolution layer that performs a convolution operation on the second graph to output a third graph, a readout layer that converts the third graph into feature vectors, and a linear combination layer that converts the feature vectors into medical judgment information. The graph convolution layer calculates a post-convolution feature for each of the multiple nodes included in the second graph, based on the pre-convolution feature quantities relating to the target node and its neighboring nodes, the adjacency matrix representing the edges connecting the target node and its neighboring nodes, and the weights for the edges.
The readout layer converts the convolutional feature quantities of each of the plurality of nodes into the feature vector.
The medical information processing device according to claim 3 . The medical information processing apparatus according to claim 3 , further comprising a display control unit that displays a visualization graph in which the second graph or the third graph relating to the subject patient is visualized. The medical information processing apparatus according to claim 5, wherein the display control unit displays the nodes included in the visualization graph in a display format corresponding to the patient feature quantities assigned to the nodes included in the second graph or the third graph. The medical information processing apparatus according to claim 5 , wherein the display control unit displays the node in a display format corresponding to the degree of disease impact assigned to the node that corresponds to the node and is included in the second graph or the third graph. The medical information processing apparatus according to claim 5, wherein the display control unit extracts a subgraph from the second graph or the third graph that includes nodes to be displayed and edges connecting the nodes to be displayed, and displays a visualization graph that visualizes the subgraph . The aforementioned node is classified into at least two categories from symptoms, physical findings, laboratory findings, treatment, treatment response, and side effects.
The display control unit extracts a subgraph from the second graph or the third graph that includes nodes belonging to the category to be displayed and edges connecting those nodes, and displays a visualization graph that visualizes the subgraph.
The medical information processing device according to claim 5 . The medical information processing apparatus according to claim 5 , wherein the display control unit assigns the name or symbol of the medical event corresponding to the node to the node included in the visualization graph. The graph convolution layer switches the parameters of the convolution process according to the edge relationship type of the edge to be processed connected to the node to be processed.
The edge relationship type is the direction of causality, the strength of causality, and/or the strength of correlation between medical events relating to the edge to be processed.
The medical information processing device according to claim 3 . The graph convolution layer switches the parameters of the convolution process according to the edge relationship type of the edge to be processed connected to the node to be processed.
The edge relationship type is a combination of the category of medical events of the processing target node to which the processing target edge is connected and the category of medical events of the adjacent node adjacent to the processing target node.
The medical information processing device according to claim 3 . The medical information processing apparatus according to claim 3 , wherein the graph convolution layer switches the parameters of the convolution process according to the type of medical judgment information. The medical information processing apparatus according to claim 3 , wherein the graph convolution layer switches the parameters of the convolution process according to the type of disease among the medical judgment information. The pre-trained model comprises a pre-trained model for acute diseases that outputs disease types classified as acute diseases as medical judgment information, and a pre-trained model for chronic diseases that outputs disease types classified as chronic diseases as medical judgment information.
The medical information processing apparatus according to claim 1, wherein the mapping unit determines the period of the medical information to be mapped, depending on whether it is using a trained model for acute diseases to estimate disease types classified as acute diseases or using a trained model for chronic diseases to estimate disease types classified as chronic diseases. The medical information includes the sequence of occurrence, the number of occurrences, and/or the severity of the medical events.
The mapping unit maps the medical information to the nodes as node features.
The medical information processing device according to claim 1. The aforementioned medical information includes spatial and/or temporal information relating to the aforementioned medical event.
The mapping unit maps the spatial information and/or temporal information to the node as node features,
The aforementioned spatial information is information relating to the location where the medical event occurred.
The aforementioned temporal information is information relating to the time when the medical event occurred.
The medical information processing device according to claim 1. The estimation unit estimates the medical judgment information for the target patient from the multiple second graphs using a trained model that includes a graph convolution layer that outputs a second graph after convolution of multiple second graphs at different time points , and a recurrent neural network layer that outputs the medical judgment information from feature vectors at different time points corresponding to each of the second graphs after convolution at different time points.
The medical information processing device according to claim 17 . The medical information includes the medical information of the target patient and the medical information of other patients whose spatial information is adjacent to the target patient.
The spatial information includes the location information and/or biological information of the other patient.
The mapping unit maps the medical information of the target patient and the medical information of the other patient to the node as node features.
The medical information processing device according to claim 1. The medical information processing device according to claim 1, wherein the estimation unit estimates at least one piece of information corresponding to the second graph relating to the target patient, namely disease classification information, prognosis prediction information, and severity classification information, as the medical judgment information. The system further includes a display control unit that displays the second graph relating to the target patient,
The aforementioned medical events include events belonging to at least one of the following categories: symptoms, physical findings, laboratory findings, treatment, treatment response, and adverse reactions.
The display control unit displays the second graph in a way that allows the category to be identified.
The medical information processing device according to claim 1. The medical information processing device according to claim 1, wherein the first graph is a graph generated based on the medical information or medical ontology of multiple patients. We obtain medical information regarding the medical events of the target patient.
A second graph relating to the target patient is generated by mapping the medical information onto a first graph having nodes corresponding to medical events and edges representing the relationships between those nodes.
Using a trained model, which is a machine learning model trained to take the second graph as input and output medical judgment information, the medical judgment information for the target patient is estimated based on the second graph relating to the target patient.
A computer-implemented medical information processing method that includes the following features.