JP7694485B2 - Information processing device, information processing method, and information processing program - Google Patents

Description

The present invention relates to an information processing device, an information processing method, and an information processing program for classifying multivariate time series data into groups.

Multivariate time series data is classified into groups using machine learning systems, but the problem with machine learning systems is that the process by which they arrive at the recognition results is a black box.

Patent document 1 discloses an invention for an inference device that can infer which of a given class an input image belongs to, and output the inference result as well as the feature quantity that is the basis for the inference.

JP 2019-082883 A

However, the invention described in Patent Document 1 has difficulty handling time-series data because the input data is an image and a convolutional neural network (CNN) is used as the neural network.

In consideration of the above, the present invention aims to provide an information processing device, an information processing method, and an information processing program for classifying multivariate time series data into groups.

To achieve the above object, the information processing device described in claim 1 includes a first deviation calculation unit that calculates the deviation of a second group of data, which is different from the first group of data, from the first group of data, a learning unit that uses the deviation calculated by the first deviation calculation unit to train an inference device, an RNN-LSTM (Recurrent Neural Network - Long Short Term Memory), a second deviation calculation unit that calculates the deviation of a new analysis target from the first group of data, and a classification unit that applies the deviation calculated by the second deviation calculation unit to the trained inference device to perform a two-group classification of whether the new analysis target belongs to the first group of data or the second group of data.

The information processing device described in claim 1 inputs the inference device as the degree of deviation from one of the two groups. Since an increase in the input variable to the inference device indicates a large deviation from one of the groups, data can be classified into two groups based on such factors.

In addition, according to the information processing device described in claim 1, by adopting an RNN-LSTM that outputs an attention weight that indicates the importance of time when inferring data input after group classification, group classification of multivariate time series data becomes possible.

The information processing device according to claim 2, wherein the first deviation calculation unit calculates the deviation of the second group of data from the first group of data based on a Gaussian mixture model of the first group of data, and the second deviation calculation unit calculates the deviation of the new analysis target from the first group of data based on a Gaussian mixture model of the first group of data. The information processing device according to claim 1, wherein the first deviation calculation unit calculates the deviation of the second group of data from the first group of data based on a Gaussian mixture model of the first group of data.

According to the information processing device described in claim 2, the degree of discrepancy between two groups of data is calculated using a Gaussian mixture model, which is a probability distribution in which normal distribution curves are superimposed.

In the information processing device described in claim 3, the trained inference unit outputs an estimated probability indicating the probability that the new analysis target belongs to either the first group of data or the second group of data, and the classification unit estimates whether the new analysis target belongs to either the first group of data or the second group of data based on the deviation calculated by the second deviation calculation unit and the estimated probability, and calculates the contribution of the new analysis target to the classification of the second group of data based on the weighting calculated using the trained inference unit and the deviation calculated by the second deviation calculation unit after the estimation.

According to the information processing device described in claim 3, the contribution of an input variable with a large deviation to the two-group classification is calculated at an input time that was important for the two-group classification indicated by the weighting. As a result, it is possible to visualize the time and variables that have a large contribution to the two-group classification.

To achieve the above object, the information processing method described in claim 4 includes a first deviation calculation step of calculating the deviation of a second group of data, which is different from the first group of data, from the first group of data, a learning step of training an inference device, an RNN-LSTM (Recurrent Neural Network - Long Short Term Memory), using the deviation calculated in the first deviation calculation step, a second deviation calculation step of calculating the deviation of a new analysis target from the first group of data, and a classification step of applying the deviation calculated in the second deviation calculation step to the trained inference device to perform a two-group classification of whether the new analysis target belongs to the first group of data or the second group of data.

The information processing method described in claim 4 inputs the inference device as the degree of deviation from one of the two groups. Since an increase in the input variable to the inference device indicates a large deviation from one of the groups, data can be classified into two groups based on such a factor.

In addition, according to the information processing method described in claim 4, by adopting an RNN-LSTM that outputs an attention weight that indicates the importance of time when inferring data input after group classification, it becomes possible to classify multivariate time series data into groups.

To achieve the above object, the information processing program described in claim 5 causes a computer to function as a first deviation calculation unit that calculates the deviation of a second group of data, which is different from the first group of data, from the first group of data, a learning unit that uses the deviation calculated by the first deviation calculation unit to train an inference device, an RNN-LSTM (Recurrent Neural Network - Long Short Term Memory), a second deviation calculation unit that calculates the deviation of a new analysis target from the first group of data, and a classification unit that applies the deviation calculated by the second deviation calculation unit to the trained inference device to perform two-group classification of whether the new analysis target belongs to the first group of data or the second group of data.

The information processing program described in claim 5 inputs the inference device as the degree of deviation from one of the two groups. Since an increase in the input variable to the inference device indicates a greater deviation from one of the groups, data can be classified into two groups based on such factors.

In addition, according to the information processing program described in claim 5, by adopting an RNN-LSTM that outputs an attention weight that indicates the importance of time when inferring data input after group classification, it becomes possible to classify multivariate time series data into groups.

As described above, the information processing device, information processing method, and information processing program according to the present invention enable group classification of multivariate time series data.

FIG. 1 is a block diagram illustrating an example of an information processing device according to an embodiment of the present invention. FIG. 2 is a block diagram showing an example of processing of the information processing device according to the embodiment. FIG. 1 is a schematic diagram illustrating an example of a Gaussian mixture model. FIG. 13 is a schematic diagram showing the contribution of each variable and each time in the starting behavior classified as group B.

The information processing device 10 according to this embodiment will be described below with reference to FIG. 1. As shown in FIG. 1, the information processing device 10 is configured to include a computer 30. The computer 30 includes a CPU 32, a ROM 34, a RAM 36, and an input/output port 38. As an example, it is desirable for the computer 30 to be a model capable of performing advanced arithmetic processing at high speed, such as an engineering workstation or a supercomputer.

In the computer 30, the CPU 32, ROM 34, RAM 36, and input/output port 38 are connected to one another via various buses such as an address bus, a data bus, and a control bus. The input/output port 38 is connected to various input/output devices, such as a display 40, a mouse 42, a keyboard 44, a hard disk (HDD) 46, and a disk drive 50 that reads information from various disks (e.g., CD-ROM, DVD, etc.) 48.

A network 52 is also connected to the input/output port 38, allowing information to be exchanged with various devices connected to the network 52. In this embodiment, a data server 56 to which a database (DB) 54 is connected is connected to the network 52, allowing information to be exchanged with the DB 54. Multivariate time series data of the evaluation target and the like are stored in the DB 54. The multivariate time series data of the evaluation target is stored in the DB 54 from an input device 60 via the network 52. If the input device 60 is connected to the network 52, the computer 30 may acquire the data from the input device 60.

In this embodiment, the multivariate time series data to be evaluated and the like are stored in DB54 connected to data server 56, but the information in DB54 may also be stored in an external storage device such as HDD46 built into computer 30 or an external hard disk.

A program related to machine learning is installed in the HDD 46 of the computer 30. In this embodiment, the CPU 32 executes the program to start machine learning and build a trained model based on the machine learning. The CPU 32 displays the processing results of the program on the display 40.

There are several ways to install the machine learning program of this embodiment into the computer 30. For example, the program can be stored on a CD-ROM or DVD together with a setup program, and the disk can be inserted into the disk drive 50, and the setup program can be executed by the CPU 32 to install the program into the HDD 46. Alternatively, the program can be installed into the HDD 46 by communicating with another information processing device connected to the computer 30 via a public telephone line or the network 52.

By executing the machine learning program, the CPU 32 has a deviation calculation function that calculates a deviation indicating the degree of deviation of data in two groups of multivariate time series data, a learning function that performs machine learning by providing the calculated deviation to a mathematical model that is an inference device, and a specified probability output function that outputs a specified probability indicating the probability that input data belongs to either one of the two multivariate time series data or the other multivariate time series data, using the inference device after learning. By executing the machine learning program having each of these functions, the CPU 32 functions as a deviation calculation unit, a learning unit, and a specified probability output unit.

After learning, the CPU 32 has a deviation calculation function that calculates the deviation of new input multivariate time series data from the multivariate time series data that is the reference data, a classification function that uses a trained inference device to classify the input data into two groups, and a contribution calculation function that calculates the contribution of each variable to the classification. By executing a program having each of these functions, the CPU 32 functions as a deviation calculation unit, a classification unit, and a contribution calculation unit.

Figure 2 is a block diagram showing an example of the processing of the information processing device 10 according to this embodiment. As shown in Figure 2, the processing of the information processing device 10 according to this embodiment includes a learning block 110 that performs machine learning, and an inference block 130 that uses a trained model. In addition, the processing of the information processing device 10 according to this embodiment includes a divergence calculation block 120 that calculates a divergence indicating the degree of divergence of data in two multivariate time series data, and a contribution calculation block 140 that calculates the contribution of each variable to the two-group classification.

In step S100 in the learning block 110, a group A sample set (hereinafter abbreviated as "group A") of multivariate time series data serving as reference data is input. Group A is, as an example, multivariate time series data relating to the straight-line starting behavior of a vehicle driven by a group of drivers with normal cognitive abilities. Specifically, group A is eight-dimensional multivariate time series data with vehicle speed, longitudinal acceleration, longitudinal jerk, lateral acceleration, lateral jerk, yaw rate, yaw angular acceleration, and yaw angular jerk as variables for the straight-line starting behavior of the vehicle, and is time series data for 6.4 seconds (64 times) in 0.1 second increments after the vehicle starts from a stopped state.

In step S102, as group A distribution learning, for example, a graph is drawn of the change over time for each variable. Then, in step S104, as group A distribution shape, for example, a normal distribution (Gaussian distribution) is calculated for each variable and each of the 64 time periods, with the horizontal axis being a random variable indicating a single variable such as vehicle speed, and the vertical axis being the probability density indicating the frequency of occurrence of multiple drivers belonging to group A with the change in the random variable indicated on the horizontal axis.

In step S106, groups A are compared to each other and the deviation from group A is calculated. When groups A are compared to each other, the same data is compared, so theoretically the deviation is 0. In this embodiment, the deviation is calculated using a Gaussian mixture model (GMM). Details of the GMM will be described later, but specifically, it is a probability distribution in which multiple normal distribution curves are superimposed as shown in FIG. 3.

In step S108, a group B sample set (hereinafter abbreviated as "group B") of multivariate time series data is input. Group B is, as an example, multivariate time series data related to the straight-line starting behavior of a vehicle driven by a group of drivers with reduced cognitive ability. Specifically, like group A, group B is eight-dimensional multivariate time series data with the vehicle speed, longitudinal acceleration, longitudinal jerk, lateral acceleration, lateral jerk, yaw rate, yaw angular acceleration, and yaw angular jerk of the vehicle in the straight-line starting behavior as variables, and is time series data for 6.4 seconds (64 times) in 0.1 second increments after the vehicle starts from a stopped state.

In step S110, the deviation of group B from group A is calculated using the GMM. FIG. 3 is a schematic diagram showing an example of a GMM. As shown in FIG. 3, the mean (median) and covariance (spread of the peak) of normal distribution 0 and normal distribution 1 are adjusted, and then the data are added together (linearly combined) with appropriate weighting coefficients to obtain a GMM curve as shown in FIG. 3.

The horizontal axis of Figure 3 is a random variable that indicates a single variable, such as vehicle speed, at a specific time (time 1), and the vertical axis is the probability density (likelihood) that indicates the frequency of occurrence of multiple drivers belonging to group A as the random variable shown on the horizontal axis changes.

In this embodiment, the likelihood in the GMM of the value of one variable of one piece of data at one time, as indicated by the distribution of the values of the same variable of each piece of data belonging to group A at the same time, is taken as the deviation of that data from group A at the same variable and time. The calculated deviations are then expressed in logarithms and arranged in chronological order for each of the aforementioned eight dimensions to create time series data of eight dimensions and 64 time periods. In this embodiment, such time series data of eight dimensions and 64 time periods is referred to as deviation time series data.

In step S112, deviation time series data of group A with respect to group A is prepared as described above. In step S114, deviation time series data of group B with respect to group A is prepared as described above.

The information processing device 10 according to this embodiment classifies input data into either group A or group B, and uses an RNN-LSTM (Recurrent Neural Network - Long Short Term Memory) equipped with an attention mechanism that indicates input times that were important for inference. As a preprocessing step for inference, the deviation from one group (group A) is calculated for each variable for each sample (such as group B) as described above, and this is collected as deviation time series data and input to the RNN-LSTM. The deviation time series data is used as input data for the RNN-LSTM during both learning and inference.

The larger the input variable of the deviation time series data input to the RNN-LSTM, the greater the deviation from group A. Therefore, in two-group classification, if a sample is determined not to be in group A based on the deviation, the sample belongs to group B, and the magnitude of the input variable contributes to the classification into group B. After two-group classification, the RNN-LSTM outputs a weight called an Attention Weight, which indicates the importance of the time when inferring the input data. Therefore, the value of this Attention Weight makes it possible to know the input time that was important for two-group classification. Furthermore, it can be estimated that an input variable with a large deviation at an input time that was important for two-group classification contributes to classification into group B.

In step S116, machine learning is performed using the calculated deviation time series data. As described above, in this embodiment, an RNN-LSTM is adopted as the mathematical model that is the inference device. In step S116, as an example, the label [1,0] is associated with the starting behavior related to group A, and the label [0,1] is associated with the starting behavior related to group B, and the deviation time series data is input.

In step S118, a softmax function is used as the activation function to output an estimated probability that the input in step S116 is from group A or group B, and processing in learning block 110 is terminated.

The following describes a series of processes in inference block 130. In step S120, the sample to be analyzed is input.

In step S122, the deviation of the sample to be analyzed from group A is calculated. The deviation is calculated using GMM as described above.

In step S124, as in step S114, deviation time series data for group A of the samples to be analyzed is prepared.

In step S126, binary classification is performed using the estimated probability described above to determine whether the sample being analyzed belongs to group A or group B, and in step S128, the classification result is stored in RAM 36 or HDD 46.

In step S130, the Attention Weight is output, which is a weight indicating the importance of time when inferring the two-group classification of the analysis target sample. The Attention Weight is time series data with a time series length of 64, similar to the input data, the deviation time series data.

In step S132, the attention weight is multiplied by each variable of the deviation time series data, and in step S134, the product obtained in step S132 is used as the degree to which the input data suggests that it belongs to group B, i.e., the contribution of each variable and each time to the two-group classification.

Figure 4 is a schematic diagram showing the contribution of each variable and each time in the starting behavior classified as group B. Figure 4 shows that two variables, yaw angular acceleration and yaw angular jerk, especially contributed to the classification of the analyzed sample into group B immediately after starting and some time after starting (approximately 3.5 seconds).

In step S136, the classification result stored in step S128 and the contribution obtained in step S134 are output, and the process ends.

As described above, in this embodiment, in two-group classification, the input to the inference device is the degree of deviation from one of the two groups. Since an increase in the input variable to the inference device indicates a greater deviation from one of the groups, data is classified into two groups based on this.

In addition, in this embodiment, by adopting an RNN-LSTM, a type of neural network, as the inference device, it becomes possible to classify multivariate time series data into groups.

The RNN-LSTM outputs an Attention Weight that indicates the importance of the time when inferring the input data after group classification. In this embodiment, the aforementioned deviation and attention weight are used to calculate the contribution to the two-group classification of an input variable that has a large deviation at an input time that was important for the two-group classification indicated by the Attention Weight. As a result, it becomes possible to visualize the times and variables that have a large contribution to the two-group classification.

In this embodiment, as an example, time series data such as vehicle speed and acceleration during a vehicle's straight-ahead starting behavior is handled. Information related to automobile driving is time series data, so it can be used, for example, for state estimation in drive monitors. In addition, this embodiment is not limited to vehicle-related issues, and can be used for a wide range of classification problems related to time series data.

In addition, the processing that the CPU reads and executes software (programs) in each of the above embodiments may be executed by various processors other than the CPU. Examples of processors in this case include PLDs (Programmable Logic Devices) such as FPGAs (Field-Programmable Gate Arrays) whose circuit configuration can be changed after manufacture, and dedicated electrical circuits such as ASICs (Application Specific Integrated Circuits) that are processors with circuit configurations designed specifically to execute specific processing. Furthermore, the processing may be executed by one of these various processors, or may be executed by a combination of two or more processors of the same or different types (e.g., multiple FPGAs, and a combination of a CPU and an FPGA, etc.). Furthermore, the hardware structure of these various processors is, more specifically, an electrical circuit that combines circuit elements such as semiconductor elements.

In addition, in each of the above embodiments, the program is described as being pre-stored (installed) in the disk drive 50 or the like, but this is not limiting. The program may be provided in a form stored in a non-transitory storage medium such as a CD-ROM (Compact Disk Read Only Memory), a DVD-ROM (Digital Versatile Disk Read Only Memory), or a USB (Universal Serial Bus) memory. The program may also be downloaded from an external device via a network.

The "first group data" described in the claims corresponds to the "A group sample set" described in the detailed description of the invention, the "second group data" described in the claims corresponds to the "B group sample set" described in the detailed description of the invention, the "first deviation calculation unit" described in the claims corresponds to the "(learning) deviation calculation unit" described in the detailed description of the invention, the "learning unit" described in the claims corresponds to the "learning unit" described in the detailed description of the invention, the "second deviation calculation unit" described in the claims corresponds to the "(learned) deviation calculation unit" described in the detailed description of the invention, and the "classification unit" described in the claims corresponds to the "classification unit" described in the detailed description of the invention.

(Additional note 1)
Memory,
at least one processor coupled to the memory;
Including,
The processor,
Calculating the degree of deviation of a second group of data, which is different from the first group of data, from the first group of data;
A recurrent neural network - long short term memory (RNN-LSTM) is trained as an inference device using the deviation of the second group of data from the first group of data;
Calculating the deviation of the new analysis target from the first group data;
applying the deviation of the new analysis target from the first group of data to a trained inference device to perform two-group classification of whether the new analysis target belongs to the first group of data or the second group of data;
The information processing device is configured as follows.

10 Information processing device 30 Computer 32 CPU
34 ROM
36 RAM
38 Input/Output port 40 Display 42 Mouse 44 Keyboard 50 Disk drive 52 Network 56 Data server 60 Input device 110 Learning block 120 Deviation calculation block 130 Inference block 140 Contribution calculation block

Claims (5)

a first deviation calculation unit that calculates a deviation of a second group of data, which is different from the first group of data, from the first group of data;
a learning unit that uses the deviation calculated by the first deviation calculation unit to train an inference device, which is a recurrent neural network - long short term memory (RNN-LSTM);
a second deviation calculation unit that calculates a deviation of a new analysis target from the first group of data;
a classification unit that applies the deviation calculated by the second deviation calculation unit to a trained inference device to perform two-group classification of whether the new analysis target belongs to the first group of data or the second group of data; and
An information processing device comprising: the first deviation calculation unit calculates a deviation of the second group of data from the first group of data based on a Gaussian mixture model of the first group of data;
The information processing apparatus according to claim 1 , wherein the second deviation calculation unit calculates the deviation of the new analysis target from the first group of data based on a Gaussian mixture model of the first group of data. The trained inference device outputs an estimated probability indicating the probability that the new analysis target belongs to either the first group of data or the second group of data;
3. The information processing device according to claim 1 or 2, wherein the classification unit estimates whether the new analysis target belongs to the first group of data or the second group of data based on the deviation calculated by the second deviation calculation unit and the estimated probability, and calculates a contribution of the new analysis target to the classification of the second group of data based on a weighting calculated using the trained inference device after the estimation and the deviation calculated by the second deviation calculation unit. a first deviation calculation step of calculating a deviation of a second group of data, which is different from the first group of data, from the first group of data;
a learning step of learning an inference device, which is a recurrent neural network - long short term memory (RNN-LSTM), by using the deviation calculated in the first deviation calculation step;
a second deviation calculation step of calculating a deviation of a new analysis target from the first group of data;
a classification step of applying the deviation calculated in the second deviation calculation step to a trained inference device to perform two-group classification of whether the new analysis target belongs to the first group of data or the second group of data;
An information processing method comprising: Computer,
An information processing program comprising: a first deviation calculation unit that calculates the deviation of a second group of data, which is different from a first group of data; a learning unit that trains an inference unit, an RNN-LSTM (Recurrent Neural Network - Long Short Term Memory), using the deviation calculated by the first deviation calculation unit; a second deviation calculation unit that calculates the deviation of a new analysis target from the first group of data; and an information processing program that applies the deviation calculated by the second deviation calculation unit to the trained inference unit, causing the new analysis target to function as a two-group classification unit that classifies the new analysis target into either the first group of data or the second group of data.