JP7587232B2 - Information processing device and computer program - Google Patents

Description

Article 30, paragraph 2 of the Patent Act applies. 1. Website address: https://kjciee.org/kjciee2019/, published on November 15, 2019. 2. Kansai Joint Conference of Electrical Engineering Societies, held on December 1, 2019.

The present disclosure relates to an information processing device and a computer program.

Traditionally, human brain functions have been studied using electroencephalograms, brain magnetic fields, magnetic resonance images, etc. For example, Non-Patent Document 1 mentions the possibility that there are individual differences in the frequency of brain activity, and that there may be a kind of compatibility in which brain activity is easier or harder to synchronize depending on the combination of individuals. Non-Patent Document 1 also mentions the existence of knowledge that shows that the closer a person is as a friend, the greater the similarity in resting brain activity patterns when examined using fMRI data.

Psychological Review 2016, Vol. 59, No. 3, pp. 283-291, Hideki Ohira, "Mechanisms that lead to motivation in brain activity"

If it were possible to predict the compatibility between two people before they meet, this could be used in a variety of situations.
The present disclosure provides techniques for accurately predicting compatibility between two people.

An information processing device according to one aspect of the present disclosure is an information processing device for predicting compatibility between humans. The information processing device includes an arithmetic circuit, a storage device that stores a prediction model, an input interface that acquires time-series data indicating changes in human brain activity, and an output interface that outputs a prediction result of compatibility between humans. The human brain includes a plurality of brain regions, and each of the plurality of brain regions is associated with each of a plurality of functions possessed by the brain. The time-series data includes data indicating temporal changes in activity of each of the plurality of brain regions acquired from each of the plurality of brain regions. The input interface acquires first time-series data and second time-series data, which are time-series data of a first human and a second human. The arithmetic circuit uses each of the first time-series data and the second time-series data to calculate a correlation coefficient indicating the degree of functional coupling between the brain regions for each combination of two brain regions selected from the plurality of brain regions, thereby generating a first correlation matrix, which is a set of correlation coefficients between the brain regions of the first human, and a second correlation matrix, which is a set of correlation coefficients between the brain regions of the second human. The arithmetic circuit also uses the first correlation matrix and the second correlation matrix to calculate a feature vector indicating a difference in functional connections between the brain regions of the first person and the second person, and inputs the feature vector into a prediction model to predict the compatibility between the first person and the second person. The prediction model is constructed by performing machine learning using at least a first type of teacher data that is a combination of a feature vector indicating a difference in functional connections between the brain regions of two people who are known in advance to have good compatibility and a first compatibility label indicating good compatibility, and a second type of teacher data that is a combination of a feature vector indicating a difference in functional connections between the brain regions of two people who are known in advance to not have good compatibility and a second compatibility label indicating not good compatibility.

A computer program according to one aspect of the present disclosure is a computer program executed by an arithmetic circuit of an information processing device to cause the information processing device to predict compatibility between humans. The information processing device includes an arithmetic circuit, a storage device that stores a prediction model, an input interface that acquires time series data indicating changes in human brain activity, and an output interface that outputs a prediction result of compatibility between humans. The human brain includes a plurality of brain regions, each of which is associated with a plurality of functions possessed by the brain. The time series data includes data acquired from each of the plurality of brain regions that indicates changes in activity of each brain region over time. The computer program causes the arithmetic circuit to acquire first time series data and second time series data, which are time series data of a first human and a second human, via the input interface. The computer program also causes the arithmetic circuit to use each of the first time-series data and the second time-series data to calculate a correlation coefficient indicating the degree of functional coupling between the brain regions for each combination of two brain regions selected from the plurality of brain regions, thereby generating a first correlation matrix which is a set of correlation coefficients between the brain regions of the first person and a second correlation matrix which is a set of correlation coefficients between the brain regions of the second person. Furthermore, the computer program causes the arithmetic circuit to use the first correlation matrix and the second correlation matrix to calculate a feature vector indicating a difference in functional coupling between the brain regions between the first person and the second person, and inputs the feature vector into a prediction model to predict the compatibility between the first person and the second person. The prediction model is constructed by performing machine learning using at least a first type of training data that is a combination of a feature vector showing the difference in functional connectivity between brain regions between two people who are known in advance to have good compatibility and a first compatibility label indicating good compatibility, and a second type of training data that is a combination of a feature vector showing the difference in functional connectivity between brain regions between two people who are known in advance to not have good compatibility and a second compatibility label indicating not good compatibility.

The trained prediction model according to one aspect of the present disclosure is a trained prediction model for predicting compatibility between a first person and a second person. The prediction model is constructed by performing machine learning using at least a first type of teacher data that is a combination of a feature vector indicating a difference in functional connections between brain regions between two people who are known in advance to have good compatibility and a first compatibility label indicating good compatibility, and a second type of teacher data that is a combination of a feature vector indicating a difference in functional connections between brain regions between two people who are known in advance to not have good compatibility and a second compatibility label indicating not good compatibility. When a feature vector indicating a difference in functional connections between brain regions between the first person and the second person is input, the prediction model outputs the compatibility between the first person and the second person as a prediction result.

This disclosure makes it possible to accurately predict the compatibility between two people.

A schematic diagram showing the speed dating venue A diagram showing an example of the results of tallying answers to questions A diagram showing examples of the relationship between each pair of men and women and their compatibility FIG. 1 shows an example of an fMRI image 100 of a participant's brain. A diagram showing four frequency bands and the waveforms belonging to each frequency band. FIG. 1 shows an example of a procedure for calculating a difference feature vector for evaluating functional connectivity Diagram showing the process flow during machine learning training A graph showing the accuracy rate of machine learning-based classification for each frequency A diagram showing the results of an analysis of the validity of compatibility prediction results for frequency bands that contain compatibility information. Schematic diagram of a compatibility prediction method using a prediction model constructed by the inventor Hardware configuration diagram of an information processing device used for machine learning 1 is a flowchart showing a procedure of a learning process executed by a CPU of an information processing device. A hardware configuration diagram of an information processing device having a replicated prediction model. A flowchart showing the procedure of a compatibility prediction process executed by a CPU of an information processing device.

Below, the embodiments will be described in detail with reference to the drawings as appropriate. However, more detailed explanation than necessary may be omitted. For example, detailed explanation of already well-known matters or duplicate explanation of substantially identical configurations may be omitted. This is to avoid the following explanation becoming unnecessarily redundant and to facilitate understanding by those skilled in the art. Note that the inventor provides the attached drawings and the following explanation so that those skilled in the art can fully understand this disclosure, and the subject matter described in the claims is not limited by them.

(Findings Obtained by the Inventor)
From the results of various research studies accumulated so far, it is known that resting brain function contains information about individual differences. In other words, it has become clear that brain function acquired at rest can be used to predict the brain activity during the individual's later actions. On the other hand, there are also research reports that show that the "compatibility" between two people can be determined by individual differences and behavioral tendencies.

The inventor hypothesized that by looking at the relationship between the brain functions of two people, it might be possible to predict psychological phenomena that will occur between those two people, in other words, to predict the compatibility between those two people. The inventor then conducted experiments and verifications to confirm that this hypothesis was correct, and concluded that information indicating the brain functions of two people contains information regarding the compatibility between those two people, and that by utilizing this information, it is possible to determine whether the "compatibility" between those two people is good or bad. A specific explanation is given below.

The inventor conducted a speed dating experiment with multiple men and women. "Speed dating" is a party where men and women who wish to marry or date gather together in the hopes of forming couples. In this experiment, there were a total of 43 participants, 22 men and 21 women, with an average age of 20.3 years old.

Figure 1 is a schematic diagram showing the speed dating venue. The procedure for the speed dating experiment is as follows. First, the women Wa to Wu are seated, and then each of the men Ma to Mu sits facing each of the women. Then, each pair of men and women is allowed to talk freely for three minutes. The conversation ends when the moderator gives a signal, after which, as indicated by the arrows, the men all move seats at once and sit down facing the woman next to them. The above process is repeated until all participants have finished talking with the opposite sex. After the conversations have ended, both male and female participants are asked whether they would like to talk to the opposite sex again.

Figure 2A shows an example of the results of tallying up responses to questions. Example (a) shows that both the man Ma and the woman Wa wanted to talk to each other. Example (b) shows that the man Mb wanted to talk to the woman Wa again, but the woman Wa did not want to talk to the man Mb again. In this experiment, case (a) was judged to be "good compatibility," and case (b) was judged to be "not very compatible."

Figure 2B shows an example of the relationship between each pair of men and women and their compatibility. From the results of the responses, information was obtained for all pairings of men and women indicating compatibility, with either "1" indicating "good compatibility" or "0" indicating "not very good compatibility."

On a separate day before the speed dating experiment, each participant underwent functional magnetic resonance imaging (fMRI) imaging at rest. fMRI is a method of visualizing hemodynamic responses associated with brain activity in humans and animals using MRI (magnetic resonance imaging). Images were taken continuously for six minutes. To maintain a resting state, each participant was instructed to do nothing while looking at the ten o'clock mark displayed on the display screen of the fMRI device.

Figure 3 shows an example of an fMRI image 100 of a participant's brain. The brain 110 includes multiple brain regions. For example, 116 brain regions are predefined, and each brain region is associated with one of the multiple brain functions, such as vision and somatosensation. Time series waveforms 102 of each brain region can be obtained from successively captured fMRI images. The time series waveforms are obtained as time series digital data (hereinafter also referred to as "time series data 102"). Note that in this specification, the time series data 102 is not data of time series waveforms obtained directly from the fMRI images, but data obtained by previously performing noise removal processing on the directly obtained time series waveform data. Noise removal processing is not essential.

The inventor performed band-pass processing on the obtained time series data 102 and divided the time series data 102 into four frequency bands. In other words, the time series data 102 obtained from each brain region was extracted by dividing it into four frequency bands.

Figure 4 shows four frequency bands and the waveforms belonging to each frequency band. The four frequency bands are (a) 0.015-0.030 Hz, (b) 0.030-0.059 Hz, (c) 0.059-0.118 Hz, and (d) 0.118-0.216 Hz. The reason for adopting the overall frequency band of 0.015-0.216 Hz is that it was thought that frequencies lower than 0.015 Hz did not contain any information that could be considered meaningful. For the upper limit of 0.216 Hz, the upper limit of the frequency of the data obtained by the fMRI device used in this experiment was adopted. The reason for dividing into four frequency bands is that recent research has shown that these four frequency bands are related to brain functions. However, the specific relationship between each frequency band and each brain function has not yet been found.

The inventors targeted each individual's signal waveforms, extracted signals from 116 predefined brain regions for each frequency band, and assessed functional connectivity across all combinations between brain regions.

Figure 5 shows an example of the procedure for calculating a differential feature vector for evaluating functional connectivity. As mentioned above, in this disclosure, there are 116 brain regions, but for generalization, there are M regions (M is an integer equal to or greater than 2).

Table 1 below shows the functional connectivity between certain human brain regions expressed in the form of an M×M matrix. The frequency bands are any of (a) to (d) above.

The (i,j) and (j,i) components of the matrix shown in Table 1 are Pearson's correlations that indicate the degree of functional connectivity between the i-th brain region and the j-th brain region. In other words, the matrix shown in Table 1 is a correlation matrix. Therefore, the upper and lower triangular components of the matrix shown in Table 1 are in a mutually transposed relationship. Note that since Pearson's correlation is well known, an example of the formula for calculating the correlation coefficient will be omitted.

The correlation matrix 120 showing the functional connectivity between brain regions of a male, Ma, shown in Figure 5, is an image in which each element of the matrix, as shown in Table 1, is colored according to the magnitude of its value for the sake of convenience in visualization. The correlation matrix 122 showing the functional connectivity between brain regions of a female, Wa, shown in Figure 5, is similar.

In the present disclosure, the compatibility between the two people is evaluated using the correlation matrix 120 of the man Ma and the correlation matrix 122 of the woman Wa. Specifically, first, the absolute value of the difference between the correlation matrix 120 of the man Ma and the correlation matrix 122 of the woman Wa is calculated. The obtained difference matrix 124 represents the magnitude of the difference between each component of the correlation matrix 120 of the man Ma and each component of the correlation matrix 122 of the woman Wa. The upper and lower triangular components of the difference matrix 124 are also mutually transposed.

In the present disclosure, a feature vector 126 is generated from the difference matrix 124. The feature vector 126 is a row vector obtained by arranging the upper triangular components or the lower triangular components of the difference matrix 124 in the row direction. The feature vector 126 indicates the degree of difference between the functional connectivity between the brain regions of a male Ma and the functional connectivity between the brain regions of a female Wa.

The inventors have hypothesized that the feature vector 126 contains information about the compatibility between the man Ma and the woman Wa. Therefore, the inventors have decided to use machine learning to predict the compatibility between men and women.

Figure 6 shows the flow of processing during machine learning learning according to the present disclosure. Machine learning is performed using a computer system 10. In the present disclosure, the machine learning is supervised learning. First, correlation matrices for all males and all females in the speed dating are calculated for each of the four frequency bands. In addition, feature vectors 126 are calculated for all combinations of male and female participants for each frequency band using the method shown in Figure 5.

Then, the computer system 10 performs machine learning using the feature vectors corresponding to pairs of male and female participants and the compatibility of the pairs obtained by the method shown in Figures 2A and 2B as training data 130. By using such training data 130, the information regarding the compatibility between the man and the woman contained in each feature vector 126 is associated with the compatibility, and the results are verified.

As a machine learning method, the inventors adopted sparse logistic regression with an elastic net regularization term. Elastic net is a method that uses lasso regression and ridge regression in combination. Lasso regression is a method that optimizes a model to minimize loss using a loss function with an L1 regularization term added, and reduces unnecessary features. Ridge regression is a method that optimizes a model to minimize loss using a loss function with an L2 regularization term added, and avoids overfitting. By using elastic net, it is possible to achieve both the reduction in the dimensionality of features caused by lasso regression and the avoidance of overfitting caused by ridge regression.

数１及び数２中の各要素は以下の意味で用いられている。
Ｎ：観測数
ｙｉ：観測値ｉに対応する相性ラベル
ｘｉ：観測値ｉに対応する特徴量ベクトル
λ：正則化パラメータ（正の値）
β_０、β：パラメータ（それぞれスカラー、ベクトル）
α：正則化の程度、すなわちＬ１正則化とＬ２正則化の混合比
Ｐ_αの項は、Ｎ個のサンプル、Ｐ個の特徴量の場合のエラスティックネットのペナルティ項である。Ｐ_αの項により、リッジ回帰のペナルティ（α＝０）と、ラッソ回帰のペナルティ（α＝１）の間にペナルティを設定することができる。また、λが増えると、βの非ゼロの要素が減る。 Equation 1 is an evaluation formula for the elastic net. Equation 2 is an equation expressing P _α (β) in Equation 1.

The elements in Equation 1 and Equation 2 are used in the following sense.
N: Number of observations yi: Compatibility label corresponding to observed value i xi: Feature vector corresponding to observed value i λ: Regularization parameter (positive value)
β _0, β: parameters (scalar and vector, respectively)
α: Degree of regularization, i.e., the mixture ratio of L1 and L2 regularization. _Pα is the penalty term for the elastic net when there are N samples and P features. _{The Pα} term allows us to set a penalty between the penalty of ridge regression (α=0) and the penalty of lasso regression (α=1). Also, as λ increases, the number of non-zero elements of β decreases.

The above-mentioned "α" and "λ" are hyperparameters. The inventors adopted three candidates for α, (0.0001, 0.001, 0.01), and three candidates for "λ", (0.15, 0.50, 0.85), and determined "α" and "λ" by changing the combinations of these candidates in the following steps (1) to (4).
(1) First, divide all the teaching data into “training data” and “test data.” (2) Build a predictive model for a certain α and λ using the training data.
(3) Measure the accuracy of the classifier on the test data for the created predictive model.
(4) Based on the obtained accuracy, the values of α and λ that result in the highest accuracy of the classifier are adopted as the best combination.

The accuracy of the classifier was determined by 10-fold cross-validation (steps (1) to (3) above). In 10-fold cross-validation, the training data set is divided into 10 parts. Nine of these data sets are used for training, and the remaining one is used for evaluation. Then, the training data set and the evaluation data set are changed while repeating the process 10 times. For example, 10 cross-validations yield 10 values indicating accuracy. The average of these values was used as the value indicating accuracy when using that training data (step (4) above).

Figure 7A shows the distribution of the actual accuracy compared to the random accuracy, that is, the accuracy distance. The horizontal axis of Figure 7A represents the accuracy distance from the chance level when the chance level is set to "0.00". The random accuracy is obtained by randomizing the training data and then constructing a classifier each time by the above-mentioned process. "Randomizing the training data" here means setting the compatibility label randomly. Originally, the feature vector obtained from a male-female pair with good compatibility is associated with the compatibility label "1" indicating good compatibility to generate training data. In addition, the feature vector obtained from a male-female pair that cannot be said to have good compatibility is associated with the compatibility label "0" indicating that the compatibility cannot be said to be good to generate training data. By "randomizing the training data", the feature vector and the compatibility label are randomly associated. The inventor attempted to find a significant difference between the actual accuracy and the random accuracy by generating training data that was randomly associated separately from the training data that was correctly associated. In the experiment, 1,000 random accuracy results were compared with the actual accuracy, and a significant classification result was determined if the probability that the accuracy distance exceeded the chance level, i.e., the probability that the actual accuracy was higher than the random accuracy, was 95% or higher.

Upon verification, two pairs of high frequency bands (0.059 Hz to 0.118 and 0.118 Hz to 0.216 Hz) showed significant discrimination results. This result means that the functional connections of these frequency bands contain information about compatibility. Hereinafter, 0.059 Hz to 0.118 and 0.118 Hz to 0.216 Hz will be referred to as "frequency bands containing information about compatibility." Note that the "frequency band containing information about compatibility" does not always have to include both frequency bands, and may be either 0.059 Hz to 0.118 or 0.118 Hz to 0.216 Hz. Furthermore, the "frequency band containing information about compatibility" may also include a portion of the frequency band from 0.059 Hz to 0.118 Hz or a portion of the frequency band from 0.118 Hz to 0.216 Hz.

Figure 7B shows the results of analyzing the validity of the compatibility prediction results for the frequency band containing compatibility information. (a) and (b) show the results of the validity analysis for the frequency band from 0.059 Hz to 0.118, and (c) and (d) show the results of the validity analysis for the frequency band from 0.118 Hz to 0.216 Hz. On the circumference, brain networks related to 116 brain areas classified by brain function are assigned. Specifically, they are the visual network, somatosensory network, switching network, limbic system, effective control network, default mode network, cerebellum, etc. Each brain network is further subdivided according to function. However, the details of these are not mentioned in this specification. The purpose is to see the relationship of functional connections of the brain networks.

Figure 7B (a)-(d) show the relationship between the functional connectivity of brain networks based on the feature vectors (or difference matrices) of two people predicted to have good compatibility. The smaller the component of the difference matrix, the more similar the strength of the functional connectivity of the brain network corresponding to that component, i.e., the correlation value of the time-series data between brain regions, is. Conversely, the larger the component of the difference matrix, the more different the strength of the functional connectivity of the brain network corresponding to that component is. (a) and (c) indicate that the more similar they are, the better the compatibility, and (b) and (d) indicate that the more different they are, the better the compatibility.

Along the circumference of (a) to (d) in Figure 7B, the network locations corresponding to some brain functions are labeled with labels indicating the network. The relationship between each label and the brain function is shown below.
Vis: Vision
SomMot: Somatosensory
Cerebellum: The cerebellum (connection with other networks)
Limbic: Limbic system (involved in emotions, learning, etc.)
Default: Default mode (involved in internal attention and sociality)

According to Figure 7B, the contributing connections of the networks corresponding to visual and sensorimotor functions (Vis, SomMot) are relatively sparse, while the contributing connections of the networks corresponding to higher brain functions that are associated with personality and behavioral tendencies (Cerebellum, Limbic, Default) are relatively dense. From this, the following fact can be inferred. That is, the way in which visual and somatosensory processing takes place in the brain does not contribute much to compatibility. On the other hand, the way in which processing takes place in the brain that is related to personality and behavioral tendencies contributes to compatibility.

Based on these results, the inventors concluded that it is possible to predict the compatibility between two people by using the functional connectivity of the two sets of frequency bands mentioned above. In addition, they concluded that it is possible to predict the compatibility between two people, a man and a woman, by constructing a predictive model using machine learning as follows.

(a) Time series waveforms of each brain region of the male and female participants were obtained for each frequency band, and correlation matrices (functional connectivity) were calculated.
(b) A feature vector indicating differences in functional connectivity between brain regions between two people who are known to have good compatibility, calculated using each correlation matrix, is combined with a first compatibility label indicating good compatibility (e.g., "1").
(c) A feature vector indicating the difference in functional connectivity between brain regions calculated using each correlation matrix between two people who are known in advance to not be compatible with each other is combined with a second compatibility label (e.g., "0") indicating that the compatibility is not good.
(d) Using the combinations obtained in (b) and (c) above as training data, machine learning is performed to construct a predictive model.

Figure 8 is a schematic diagram of a compatibility prediction method that uses a prediction model constructed by the present inventor. A prediction model trained by the above method is constructed in the information processing device 10.

First, correlation matrices are calculated from the time-series waveforms of each brain region of the male and female for which compatibility is to be predicted, and a feature vector 132 is calculated from the difference in absolute values of these correlation matrices. The calculated feature vector is input to the information processing device 10. When the information processing device 10 inputs the feature vector 132 into a prediction model, a value indicating either good or poor compatibility is output as the output from the prediction model. For example, the output from the prediction model is a compatibility label "1" indicating good compatibility, or a compatibility label "0" indicating poor compatibility.

Note that because compatibility labels "1" or "0" are used as training data, binary compatibility labels "1" or "0" are output. However, if the compatibility labels are, for example, four values as training data, it is also possible to output one of the four values from the prediction model. The "four values" here refer to four numerical values (compatibility labels) corresponding to, for example, "very good compatibility," "good compatibility," "poor compatibility," and "very bad compatibility." Furthermore, five compatibility labels may be used, including "neither good nor bad compatibility." The number of labels may be set arbitrarily.

By performing machine learning according to the type of compatibility label used in the training data, it is possible to output three or more types of compatibility from the prediction model. The type of compatibility is determined relatively according to the degree of compatibility. When the number of compatibility types used as training data is three or more, the prediction model may output a label indicating the relatively best compatibility, a label indicating the relatively worst compatibility, and one or more labels relatively between them. In the present disclosure, whether the number of compatibility labels at the time of learning and prediction is two or three or more, at least a label corresponding to the relatively best compatibility (or a first compatibility label indicating good compatibility) and a label corresponding to the relatively worst compatibility (or a second compatibility label indicating that compatibility cannot be said to be good) are included in the training data and output as a prediction result.

Furthermore, although the inventor obtained the above findings through speed dating as described above, it is believed that the above findings do not only indicate compatibility between men and women. This is because, as long as the prediction is made using time-series waveforms of brain regions, it is believed that the predictive model can predict compatibility between people other than just between men and women. For example, it is strongly speculated that it can appropriately determine compatibility between people of the same sex, or compatibility between people with relatively high and low social or organizational status, such as a boss and a subordinate in a company organization, a teacher and a student in a school organization, a medical professional and a patient, or a store clerk and a customer.

Below, a hardware configuration for constructing a prediction model based on the above findings and a compatibility prediction process using the constructed prediction model will be described.
9 is a hardware configuration diagram of an information processing device 10 used for performing machine learning. The information processing device 10 may be, for example, a PC or a server computer. The information processing device 10 includes a CPU 21, a communication interface (I/F) 22, and a storage device 23.

The CPU 21 is an example of an arithmetic circuit of the information processing device in this embodiment. The CPU 21 executes a control program 26 stored in the storage device 23 to realize predetermined functions including learning and executing a prediction model 27. The information processing device 10 realizes the functions of the information processing device of this embodiment by the CPU 21 executing the control program 26. The control program 26 is an example of a computer program in this embodiment. Note that the arithmetic circuit configured as the CPU 21 in this embodiment may be realized by various processors such as an MPU or GPU, or may be configured by one or more processors.

The communication interface 22 is, for example, an Ethernet (registered trademark) communication terminal or a USB (registered trademark) terminal. Alternatively, the communication interface 22 is a communication circuit that communicates in accordance with standards such as IEEE802.11, 4G, or 5G. The communication interface 22 can be connected to a communication network such as an intranet or the Internet. Furthermore, the information processing device 10 may communicate directly with other devices via the communication interface 22, or may communicate via an access point.

The storage device 23 is a storage medium that stores computer programs and data necessary to operate the information processing device 10. The storage device 23 may be, for example, a hard disk drive (HDD) or a solid state drive (SSD) that is a semiconductor storage device. The storage device 23 may include a temporary storage element configured, for example, by a RAM such as a DRAM or an SRAM, and may function as a working area for the CPU 21. The storage device 23 stores the control program 26 executed by the CPU 21, and stores the prediction model 27 after the prediction model 27 is constructed.

Figure 10 is a flowchart showing the steps of the learning process executed by the CPU 21 of the information processing device 10. Please also refer to Figure 6.

In step S11, the CPU 21 acquires the teacher data 130 (FIG. 6) via the communication interface 22. As shown in FIG. 6, the teacher data 130 is a set of a feature vector and a compatibility label.

In step S12, the CPU 21 performs machine learning of the prediction model. In this embodiment, machine learning is performed using sparse logistic regression with an elastic net regularization term.

In step S13, the CPU 21 adjusts the hyperparameters. In step S14, the CPU 21 generates a prediction model 27. As described in relation to Equation 1 and Equation 2, the prediction model 27 is constructed by using "α" and "λ" as hyperparameters and adopting the values of α and λ that maximize the accuracy of the classifier.

The prediction model 27 constructed as described above can be used as is to predict the compatibility between any two people using the information processing device 10. Alternatively, by duplicating the data constituting the prediction model 27 to any information processing device, compatibility predictions can be made in information processing devices other than the information processing device 10 that generated the prediction model 27.

Figure 11 is a hardware configuration diagram of an information processing device 30 having a replicated prediction model 27. The information processing device 30 may also be, for example, a PC or a server computer. The information processing device 30 includes a CPU 41, an input interface (I/F) 42, a storage device 43, and an output interface (I/F) 44.

The CPU 41, the input interface 42, and the storage device 43 have the same configurations and functions as the CPU 21, the communication interface 22, and the storage device 23 shown in FIG. 9. Therefore, the above description of the CPU 21, the communication interface 22, and the storage device 23 is used to explain the CPU 41, the input interface 42, and the storage device 43.

The storage device 23 stores a prediction model 27. The prediction model 27 is constructed and replicated by the information processing device 10 and stored in the storage device 23. The prediction model 27 may be incorporated as part of a computer program executed by the CPU 41, for example, or may be provided as data separate from the computer program.

The output interface 44 is, for example, a video output terminal connected to a display 50 provided outside the information processing device 30. Alternatively, the output interface 44 is a communication terminal or communication circuit that is connected to a communication network 52 or the like and is capable of performing data communication. When the output interface 44 is a communication terminal or communication circuit, its specific content is the same as that of the communication interface 22 (Figure 9). Therefore, the above description of the communication interface 22 is used here.

Figure 12 is a flowchart showing the procedure of the compatibility prediction process executed by the CPU 41 of the information processing device 30. It is assumed that the two people whose compatibility is to be predicted are predetermined, and that the time series waveforms (time series data) of each person's brain region are also prepared in advance.

In step S21, the CPU 41 acquires time series data of two people via the input interface 42. In step S22, the CPU 41 uses each time series data to calculate a correlation coefficient for each combination of two brain regions for each person. In step S23, the CPU 41 calculates a feature vector using the correlation matrices (first correlation matrix and second correlation matrix) calculated from the time series data of each person. The feature vector is a row vector obtained by arranging in the row direction the upper triangular components or lower triangular components of the difference matrix calculated as the absolute value of the difference between the first correlation matrix and the second correlation matrix.

In step S24, the CPU 41 inputs the obtained feature vector into the prediction model 27 and executes processing of the prediction model 27.

In step S25, the CPU 41 outputs the predicted compatibility between the two people from the prediction model 27. The CPU 41 may output the predicted result as text or an image on the display 50, or may output the predicted result over the communication network 52. In the latter case, for example, each of the two people can check the predicted result using an information processing device such as their own smartphone.

In the above embodiment, an example has been described in which the prediction model 27 is constructed by machine learning using sparse logistic regression with an elastic net regularization term. In this embodiment, the prediction model 27 may be generated by other machine learning methods. For example, a support vector machine, a random forest, gradient boosting, an artificial neural network, or the like may be used.

In this specification, an example has been described in which the time-series data is fMRI images captured continuously. However, the use of fMRI images is just one example. Time-series data acquired by other methods can also be used as long as the method can acquire the activity of multiple brain regions. For example, time-series electroencephalogram data (EEG) acquired by an electroencephalograph showing changes in potential caused by electrical activity of the brain can be adopted. Alternatively, time-series magnetic field data (MEG) acquired by a magnetoencephalograph showing changes in magnetic field caused by electrical activity of the brain can be adopted. Furthermore, time-series brain data, which is changes in cerebral blood flow dynamics measured using near-infrared spectroscopy with a NIRS measuring device, can be adopted. At present, fMRI images need to be acquired at a medical institution, but if helmet-type electroencephalographs and the like become widespread, it will be possible for individuals to acquire EEG, and it is thought that it will be easier to acquire time-series data for using the prediction model 27. Note that the data obtained by fMRI images, EEG, MEG, and NIRS are not exclusive, and multiple combinations may be used.

The above-mentioned flowcharts (FIGS. 10 and 12) can be realized as computer programs executed by a CPU. FIG. 12 may be realized as a computer program incorporating data from the prediction model 27.

This disclosure is not limited to the above-described embodiments, and various modifications are possible. In other words, embodiments obtained by combining technical means modified as appropriate by a person skilled in the art are also within the scope of this disclosure.

This disclosure is applicable to information processing devices, computer programs, and trained prediction models that predict the compatibility between two people.

10, 30 Information processing device 21 CPU
22 Communication interface (I/F)
23 Storage device 27 Prediction model 41 CPU
42 Input interface (I/F)
43 Storage device 44 Output interface (I/F)
100 fMRI image of brain 102 Time series waveform/time series data 110 Brain 120, 122 Correlation matrix 124 Difference matrix 126 Feature vector 130 Training data

Claims (19)

An information processing device for predicting compatibility between people,
An arithmetic circuit;
A storage device that stores a prediction model;
An input interface for acquiring time-series data showing changes in human brain activity;
and an output interface that outputs a prediction result of compatibility between people;
the human brain includes a plurality of brain regions, each of the plurality of brain regions being associated with a respective one of a plurality of functions of the brain;
the time-series data includes data obtained from each of the plurality of brain regions and indicating a temporal change in activity of each of the brain regions;
The input interface acquires first time series data and second time series data, which are time series data of a first person and a second person, respectively;
The arithmetic circuit includes:
using each of the first time-series data and the second time-series data to calculate a correlation coefficient indicating a degree of functional connectivity between brain regions for each combination of two brain regions selected from the plurality of brain regions, thereby generating a first correlation matrix which is a set of correlation coefficients between the first human brain regions and a second correlation matrix which is a set of correlation coefficients between the second human brain regions;
Calculating a feature vector indicating a difference in functional connectivity between brain regions between the first human and the second human using the first correlation matrix and the second correlation matrix;
inputting the feature vector into the prediction model to predict compatibility between the first person and the second person;
The prediction model comprises at least
A first type of training data is a combination of a feature vector indicating a difference in functional connectivity between brain regions between two people who are known to have good compatibility in advance and a first compatibility label indicating good compatibility; and
An information processing device constructed by performing machine learning using a second type of training data which is a combination of a feature vector indicating differences in functional connectivity between brain regions between two people who are known in advance to not be compatible with each other and a second compatibility label indicating that the people do not get along well. The time series data is
Time series waveform data of a predetermined frequency band obtained from a plurality of fMRI images taken by an fMRI device;
Time-series electroencephalogram (EEG) data obtained by an electroencephalograph, which shows changes in electrical potential caused by electrical activity in the brain;
Time-series magnetic field data obtained by a magnetoencephalograph, which shows changes in the magnetic field caused by electrical activity in the brain, and/or
The information processing device according to claim 1 , wherein the time-series brain data is a change in cerebral blood flow dynamics measured by a NIRS measuring device using near-infrared spectroscopy. 3. The information processing device according to claim 2, wherein when the time series data is time series waveform data of the predetermined frequency band, the frequency band includes all or a part of 0.118 Hz to 0.216 Hz, and/or all or a part of 0.059 Hz to 0.118 Hz. There are M brain regions,
the first correlation matrix and the second correlation matrix are M×M matrices;
The information processing device according to claim 1 , wherein the (i, j) and (j, i) components of the matrix are correlation coefficients indicating a degree of functional connectivity between the i-th brain region and the j-th brain region. The information processing device according to claim 4, wherein the feature vector is calculated as an absolute value of the difference between the first correlation matrix and the second correlation matrix. The information processing device according to any one of claims 1 to 5, wherein the prediction model is constructed using sparse logistic regression with an elastic net regularization term. the first human being and the second human being are a male and a female;
7. An information processing device according to claim 1, wherein the two people known to have good compatibility used in constructing the predictive model are a man and a woman, and the two people known to have poor compatibility are a man and a woman. the first person and the second person are of the same sex;
7. An information processing device according to claim 1, wherein the two people known to have good compatibility used in constructing the predictive model are of the same sex, and the two people known to have poor compatibility are of the same sex. The first person and the second person are people who have a relatively high or low social or organizational status,
An information processing device as described in any one of claims 1 to 6, wherein the two people who are known to have a good compatibility and who are used to construct the predictive model are a person who has a relatively high status and a person who has a relatively low status in society or within an organization, and the two people who are known to not have a good compatibility are a person who has a relatively high status and a person who has a relatively low status in society or within an organization. The storage device stores the time series data,
10. The information processing device according to claim 1, wherein the input interface is an input terminal connected to the arithmetic circuit, and acquires the time-series data read from the storage device and inputs the data to the arithmetic circuit. The time series data is collected or stored by an external device;
10. The information processing apparatus according to claim 1, wherein the input interface is a communication terminal that receives the time series data transmitted from the external device via a communication line. The information processing device according to any one of claims 1 to 11, wherein the output interface is an output terminal connected to the arithmetic circuit and outputs the prediction result output from the arithmetic circuit. The information processing device according to claim 12, wherein the output interface is a video output terminal. The information processing device according to any one of claims 1 to 11, wherein the output interface is a communication terminal that outputs the prediction result to a communication line. the prediction model is constructed by performing the machine learning using one or more other types of teacher data in addition to the first type of teacher data and the second type of teacher data,
The one or more other types of teacher data,
An information processing device as described in any one of claims 1 to 14, which is a combination of a feature vector indicating differences in functional connectivity between brain regions between two people whose degree of compatibility, determined relatively in relation to whether they have good compatibility or not, is known in advance, and one or more compatibility labels indicating the degree of compatibility. A computer program that is executed by an arithmetic circuit of an information processing device to cause the information processing device to predict compatibility between people,
The information processing device includes:
The arithmetic circuit;
A storage device that stores a prediction model;
An input interface for acquiring time-series data showing changes in human brain activity;
and an output interface that outputs a prediction result of compatibility between people;
the human brain includes a plurality of brain regions, each of the plurality of brain regions being associated with a respective one of a plurality of functions of the brain;
the time-series data includes data obtained from each of the plurality of brain regions and indicating a temporal change in activity of each of the brain regions;
The computer program causes the arithmetic circuit to
acquiring first time series data and second time series data, which are time series data of a first person and a second person, via the input interface;
using each of the first time-series data and the second time-series data, a correlation coefficient indicating a degree of functional connection between brain regions is calculated for each combination of two brain regions selected from the plurality of brain regions, thereby generating a first correlation matrix which is a set of correlation coefficients between the first human brain regions and a second correlation matrix which is a set of correlation coefficients between the second human brain regions;
calculating a feature vector indicating a difference in functional connectivity between brain regions between the first person and the second person using the first correlation matrix and the second correlation matrix;
inputting the feature vector into the prediction model to predict compatibility between the first person and the second person;
The prediction model comprises at least
A first type of training data is a combination of a feature vector indicating a difference in functional connectivity between brain regions between two people who are known to have good compatibility in advance and a first compatibility label indicating good compatibility; and
A computer program constructed by performing machine learning using a second type of training data, which is a combination of a feature vector indicating differences in functional connectivity between brain regions between two people who are known in advance to not be compatible with each other and a second compatibility label indicating that the people do not get along well. The computer program of claim 16, further comprising the predictive model. A trained predictive model for predicting compatibility between a first person and a second person, comprising:
The parameters of the trained prediction model include at least
A first type of training data is a combination of a feature vector indicating a difference in functional connectivity between brain regions between two people who are known to have good compatibility in advance and a first compatibility label indicating good compatibility; and
The second type of training data is a combination of a feature vector indicating a difference in functional connectivity between brain regions between two people who are known in advance to not be compatible with each other and a second compatibility label indicating that the two people are not compatible with each other, and the training data is adjusted by machine learning using the feature vector and a second compatibility label indicating that the two people are not compatible with each other,
Computer,
A trained prediction model that functions to accept as input a feature vector indicating differences in functional connectivity between brain regions between the first human and the second human, perform a calculation on the input feature vector based on the parameters, and output a prediction result regarding compatibility between the first human and the second human. the human brain includes a plurality of brain regions, each of the plurality of brain regions being associated with a respective one of a plurality of functions of the brain;
The feature vector between the two people is
acquiring first time series data and second time series data from each of the two people, the first time series data being time series data acquired from each brain region of one of the two people and showing temporal changes in activity of the respective brain regions, and the second time series data being time series data acquired from each brain region of the other of the two people and showing temporal changes in activity of the respective brain regions;
calculating a correlation coefficient indicating a degree of functional connectivity between brain regions for each combination of two brain regions selected from the plurality of brain regions using each of the first time-series data and the second time-series data, thereby generating a first correlation matrix which is a set of correlation coefficients between one brain region of the two people and a second correlation matrix which is a set of correlation coefficients between the other brain region of the two people;
20. The trained prediction model of claim 18, wherein the trained prediction model is calculated by: calculating an absolute value of a difference between the first correlation matrix and the second correlation matrix.

Images