JP7619481B2 - Model learning device, model learning method, and model learning program - Google Patents

Description

The present invention relates to a model learning device, a model learning method, and a model learning program.

Learning a model that represents input-output relationships from multivariate data is one of the problems in the fields of machine learning and artificial intelligence. In this problem, in a typical setup, a set of pairs of input values and the output values obtained from those inputs is given as numerical data. However, in cases where responses cannot be obtained from questionnaires due to psychological resistance, or when output values are difficult to observe, ordinal relationships rather than numerical values may be given as output values.

Examples of cases where values cannot be obtained due to psychological resistance include the collection and analysis of physical information or information about personal life. For example, when conducting a questionnaire survey to investigate the relationship between diet and health, people may not provide their weight due to psychological resistance. In other cases, data may be collected to predict electricity consumption from indoor conditions (temperature, humidity, etc.). Here too, actual values may not be provided due to psychological resistance, such as not wanting people to guess how much time they spend at home or their family composition.

Examples of output values that are difficult to observe include emotions such as satisfaction and excitement. In general, the true values of human emotions cannot be observed, and they involve the subjectivity of the evaluator, making absolute evaluation difficult. For this reason, data on human emotions is often obtained using multi-level evaluations such as Likert scales (Non-Patent Document 1) and SAM (self-assessment manikins) (Non-Patent Document 2). These evaluation scales are non-linear and include the bias of the evaluator, so it is not appropriate to consider them as interval scales or proportional scales. In fact, it is not appropriate to compare an emotion evaluation value of 2 points by a certain evaluator A with an emotion evaluation value of 4 points by evaluator B and argue that there is a two-fold difference, or to interpret the same difference of 2 points as being equivalent to an evaluation result of 1 point and 3 points.

These issues can be alleviated by making the questions in the questionnaire or interpreting the answers as an ordinal scale. Making the questions an ordinal scale means making the questions "Is your weight over __kg?" instead of "Please enter your weight." By not mentioning specific numerical values, it is possible to reduce psychological resistance and make it easier to obtain answers. Even in the case of emotions and feelings, the emotional evaluation value of evaluator A is 2 points and the emotional evaluation value of evaluator B is 4 points, so it can be interpreted as "A < B." This "A < B" treats the answers as an ordinal relationship, which reduces the possibility of misinterpretation compared to when it is treated as a numerical value. With this setting, it is necessary to consider the problem of predicting a specific numerical value from an ordinal relationship when a specific numerical value cannot be obtained but an ordinal relationship is given as label data.

Rank learning is a method for learning model parameters from order relationships. Based on the commonly used pairwise rank learning, the general flow of rank learning is described below. For observable multivariate data X with n _x elements, a set D of n _k pairs indicating order relationships is defined as label data. The multivariate data X and the set D of pairs are as follows:

This does not indicate that _vk >u _k , but indicates that when the input values are _vk ,u _k, the objective variables _yvk , _yuk are _yvk > _yuk . If the model to be trained is f, the learning is realized by minimizing L=exp(f(vk)-f(u k)) so that the output values f( _vk ),f(u _k ) obtained by inputting _vk ,u _k are f( _{vk )} >f( _{u k )} . In other words, the purpose of rank learning is to adjust the parameters of the score output model f to configure the intended ranking.

A well-known rank learning method is RankNet (Non-Patent Document 3). It uses a logistic function and an entropy loss function, and defines the loss function L as follows:

This loss function produces smaller output when the order relationship of paired data is preserved, making it possible to generate a score that satisfies the order relationship of the label data.

There is also a rank learning method using Gaussian distribution (Non-Patent Document 4). In this method, the loss function L is defined as follows:

These methods are models that adjust model parameters from order relationships and generate scores that can be used to construct rankings. In other words, as long as a ranking can be constructed based on label data, the value of the score does not matter. For example, as a result of evaluating emotions during movie watching on a scale of 1 to 5, it is found that v _k is more preferable than u _k for inputs v _k and u _k . By learning from this label, it is possible to satisfy f(v _k )>f(u _k ), and a ranking based on the label data can be constructed. However, there is no guarantee that f(v _k ) and f(u _k ) fall within the closed interval [1, 5], and this score cannot be regarded as an emotion evaluation value.

R. Likert. A technique for the measurement of attitudes. Archives of Psychology, Vol. 22 140, pp. 5-55, 1932. Jon D. Morris. Observations: SAM: The Self-Assessment Manikin An Efficient Cross-Cultural Measurement Of Emotional Response. (This article originally appeared in the Journal of Advertising Research November/December 1995.) Christopher Burges, Tal Shaked, Erin Renshaw, Ari Lazier, Matt Deeds, Nicole Hamilton, and Greg Hullender. Learning to rank using gradient descent. pp. 89-96, 01 2005. Wei Chu and Zoubin Ghahramani. Preference learning with gaussian processes. pp. 137-144, 08 2005.

However, existing rank learning is a model that learns order relationships from label data and outputs a score according to the order relationships, and the output does not indicate the objective variable. In other words, when only the order relationships are given as label data, it is difficult to predict a specific objective variable from the order relationships using existing rank learning alone.

The present invention has been made in light of the above-mentioned circumstances, and its purpose is to provide a model learning device, a model learning method, and a model learning program that predict a dependent variable from an order relationship, even when only an order relationship is available as label data.

One aspect of the present invention is a model learning device. The model learning device includes an input data processing unit that acquires input data, a setting parameter processing unit that acquires setting parameters, a model parameter learning unit that uses the input data and the setting parameters as inputs and rank-learns model parameters using an objective function composed of a loss function that maintains an order relationship and a regularization term that limits the output of a model, and a model parameter processing unit that outputs the learned model parameters. The input data is observable multivariate data and label data that indicates an order relationship. The setting parameters include a hyperparameter of the regularization term. The model parameter learning unit acquires model parameters that output a score that predicts a response variable.

One aspect of the present invention is a model learning method executed by a computer . The model learning method includes acquiring input data, acquiring setting parameters, and using the input data and the setting parameters as inputs, rank learning of model parameters using an objective function composed of a loss function that maintains an order relationship and a regularization term that limits the output of a model, and outputting the learned model parameters. The input data is observable multivariate data and label data that indicates an order relationship. The setting parameters include a hyperparameter of the regularization term. The rank learning obtains model parameters that output a score that predicts a response variable.

A model learning program according to one aspect of the present invention causes a computer to execute the functions of each component of the above-mentioned model learning device.

According to the present invention, a model learning device, a model learning method, and a model learning program are provided that predict a target variable from an order relationship even when only an order relationship is obtained as label data.

FIG. 1 is a block diagram illustrating an example of a functional configuration of a model learning device according to an embodiment. FIG. 2 is a block diagram illustrating an example of a hardware configuration of the model learning device according to the embodiment. FIG. 3 is a flowchart illustrating an example of a process procedure and process contents of model parameter estimation executed by the model learning device according to the embodiment. FIG. 4 is a flowchart illustrating an example of a processing procedure and processing contents of model parameter rank learning executed by the model parameter learning unit of the model learning device according to the embodiment.

Below, an embodiment of the present invention is described with reference to the drawings.

[Configuration example]
First, the operation of the model learning device 1 will be described with reference to Fig. 1. Fig. 1 is a block diagram showing an example of the functional configuration of the model learning device 1 according to the embodiment.

1, the model learning device 1 has an input data processing unit 10, a setting parameter processing unit 20, a model parameter learning unit 30, a model parameter processing unit 40, a recording unit 50, and an input/output unit 60. The recording unit 50 has an input data recording unit 51, a setting parameter recording unit 52, and a model parameter recording unit 53.

The input/output unit 60 inputs and outputs data between the external device 2. The input/output unit 60 receives input data and setting parameters from the external device 2. The input/output unit 60 also outputs model parameters to the external device 2.

The input data processing unit 10 acquires input data from the external device 2 via the input/output unit 60. The input data processing unit 10 outputs the acquired input data to the input data recording unit 51.

The input data recording unit 51 receives input data from the input data processing unit 10 and records it.

The setting parameter processing unit 20 acquires setting parameters from the external device 2 via the input/output unit 60. The setting parameter processing unit 20 outputs the acquired setting parameters to the setting parameter recording unit 52.

The setting parameter recording unit 52 receives setting parameters from the setting parameter processing unit 20 and records them.

The model parameter learning unit 30 uses the input data recorded in the input data recording unit 51 and the setting parameters recorded in the setting parameter recording unit 52 as inputs, and performs rank learning of the model parameters using an objective function consisting of a loss function that maintains the order relationship and a regularization term that limits the model output. The model parameter learning unit 30 outputs the learned model parameters to the model parameter recording unit 53.

The model parameter recording unit 53 receives model parameters from the model parameter learning unit 30 and records them.

The model parameter processing unit 40 reads the model parameters from the model parameter recording unit 53 and outputs them to the external device 2 via the input/output unit 60.

The input data acquired by the input data processing unit 10 is observable multivariate data and label data indicating an order relationship. For example, the label data is a set of pairs indicating an order relationship.

The setting parameters acquired by the setting parameter processing unit 20 include hyperparameters of the regularization term of the objective function used by the model parameter learning unit 30 for rank learning. The hyperparameters include quantiles of the objective variable. Here, a quantile is a number that exists at a position where a set of sorted numbers is equally divided by positive integers. The hyperparameters also include parameters that determine the weight of the regularization term. The setting parameters also include a learning rate parameter used to optimize the objective function.

Next, we will explain the hardware configuration of the model learning device 1. The model learning device 1 is configured as a computer. For example, the model learning device 1 is configured as a personal computer, a server computer, etc.

Figure 2 is a block diagram showing an example of a hardware configuration of the model learning device 1 according to the embodiment. As shown in Figure 2, the model learning device 1 has an input/output interface 110, a CPU 120, and a storage device 130.

The input/output interface 110, CPU 120, and storage device 130 are electrically connected to each other via bus 140, and exchange data and commands via bus 140.

The input/output interface 110 is connected to the external device 2 via a signal cable or a network. The input/output interface 110 is used to receive data from the external device 2 and to output model parameters to the external device 2.

The storage device 130 stores the programs and data necessary for the processes executed by the CPU 120. The CPU 120 performs various processes by reading and executing the necessary programs and data from the storage device 130.

The storage device 130 has a main storage device 131 and an auxiliary storage device 132. The main storage device 131 and the auxiliary storage device 132 exchange programs and data between each other.

The main memory device 131 stores programs and data temporarily required for processing by the CPU 120. For example, the main memory device 131 is composed of a volatile memory such as a RAM (Random Access Memory).

The auxiliary storage device 132 stores programs and data supplied via an external device or a network, and provides the programs and data temporarily required for processing by the CPU 120 to the main storage device 131. For example, the auxiliary storage device 132 is configured with non-volatile memory such as a hard disk drive (HDD) or a solid state drive (SSD).

The CPU 120 is a processor and is hardware that processes data and instructions. The CPU 120 has a control device 121 and an arithmetic device 122.

The control device 121 controls the input/output interface 110, the computing device 122, and the memory device 130.

The calculation device 122 reads the program and data from the main memory device 131, executes the program to process the data, and provides the processed data to the main memory device 131.

In such a hardware configuration, the input/output interface 110 constitutes the input/output unit 60. The CPU 120 and the main memory device 131 constitute the input data processing unit 10, the setting parameter processing unit 20, the model parameter learning unit 30, and the model parameter processing unit 40. The memory device 130 (e.g., the auxiliary memory device 132) constitutes the recording unit 50.

[Example of operation]
Next, the operation of the model learning device 1 will be described. The rank learning performed by the model learning device 1 is a method of restricting the distribution of output values in rank learning by a regularization term based on quantiles. In order to bring the model output closer to the objective variable, it is necessary to approximate the model output distribution to the objective variable distribution. However, when only an order relationship is obtained as label data, a specific numerical value cannot be obtained, and therefore the objective variable distribution cannot be obtained.

Therefore, instead of approximating the entire distribution, the quantiles of the objective variable are given as hyperparameters, and only a number of points determined by the quantiles are approximated. A quantile is m-1 numbers {xi 1 , xi 2 , ..., xi m- ₁ } that exist at positions that divide the distribution into m equal parts by an arbitrary positive integer m for a set of sorted numbers X = { _x1 , x2, ..., xn|x1≦ _{x2≦...≦xn} }, and xi _l is called the lmth quantile.

When the m-quantile α of the objective variable is given as a hyperparameter, the model output is restricted by minimizing the error between the m-quantile β of the model output value. Alternatively, the model output is restricted by minimizing an objective function that includes a term calculated using the m-quantile α of the objective variable and the m-quantile β of the model output value.

(Input data)
The input data acquired by the input data processing unit 10 is observable multivariate data and label data indicating an order relationship. For example, the multivariate data is observable multivariate data X having n _x elements, and the label data is a set D of n _k pairs indicating an order relationship. Hereinafter, the set D of pairs is also referred to as paired label data D. The multivariate data X and paired label data D are expressed as follows:

(Model)
The model f used in the rank learning performed by the model parameter learning unit 30 can be any model that outputs a score that serves as a ranking construction index in rank learning. The input model f may be a generalized linear model using a linear model, a logistic function, or a sigmoid function, or may be a nonlinear model such as a Gaussian process model or a deep learning model. In addition, when X is multivariate time series data, it may be a basic time series model that estimates a target variable _{y t from} a function for obtaining z _t from a latent variable z _t-1 and x _t at a certain time t, or an RNN architecture such as LSTM. It is also assumed that a model having a CNN structure expressed by the following formula is used when the multivariate data X is a set of images.

(output)
The output of the model learning device 1 is an estimation result of the model parameter θ. The estimation result of the model parameter θ is a score close to the objective variable.

(Objective function)
Next, the objective function in rank learning will be explained. The objective function is composed of a loss function E that maintains the order relationship and a regularization term Ω that limits the model output. Based on this, the objective function L is expressed as follows.

Here, λ is a hyperparameter that determines the weight of the regularization term, and α is a hyperparameter that indicates the m-quantile of the objective variable.

(Regularization)
Since the subject of the objective function is regularization, we will first discuss the regularization term Ω. There are various possible definitions for Ω that approximates the quantiles of two distributions, and here we define Ω ₁ and Ω ₂ as follows.

First, let us consider _Ω1 . If the cumulative distribution function of the objective variable is Φ(x) and the cumulative distribution function of the model output is Ψ(x), the L1 distance between the two cumulative distribution functions can be defined as a regularization term that takes a smaller value as the two distributions become more similar.

In this equation, since the specific value of the objective variable cannot be obtained, a specific value cannot be obtained for Φ(a). Therefore, assuming that the m-quantile α={α ₁ , ..., α _m-1 } of the objective variable is given, the following equation, which is obtained by discretizing the left side of the previous equation with respect to a, is minimized.

Here, if the model output sorted in ascending order is y, and a subset of elements of y that are smaller than α _l is y _{α l} , then Ψ(α _l ) can be calculated as follows.

However, n used in formula (7) is a counting process of the number of elements, which is procedural and generally cannot be used for backpropagation of errors. Therefore, by determining the regularization term Ω2 so that the (1/m) elements belonging to the l+1th quantile of the model output y are included in the range from _αl to _{αl+1 ,} Ψ( _αl )→l/m is realized and _Ω1 is approximately minimized. _Ω2 is shown in formula (8).

Here, b is the batch size (=n(y)), and α ₀ , α _m are arbitrary values indicating the upper and lower limits of the objective variable. When the lmth quantile of y is β _l , this regularization term Ω ₂ has a value of 0 for an element y _i sandwiched between two quantiles (β _l , β _l+1 ) if y _i belongs to the interval [α _l , α _l+1 ], and outputs the difference between α _l when y _i is smaller than α _l , and the difference between α _l+1 when y _i is larger than α _l+1 . This means that the loss becomes smaller as the elements of y that belong to the interval [β _l , β _l+1 ] defined by the quantile of y fall within the interval [α _l , α _l+1 ] defined by the quantile of the objective variable. Therefore, Ω ₂ =0 when the elements of y are equally divided into each interval defined by the quantile α. In other words, when b is sufficiently large _{and Ω2} →0, n( _yαl )→(l/m)b, so by substituting into equations (5) and (7), Ω→0 in the same way. Therefore, minimizing _Ω2 realizes the minimization of _Ω1 .

Next, we will describe the minimization algorithm of equation (8). First, for elements belonging to the l+1th quantile of the model output y, we define vector c storing the (l+1)mth quantile of the objective variable and vector d storing the lmth quantile of the objective variable as follows:

Using these u and l, the regularization term for the computational process is given by the following equation.

In this regularization term, the hyperparameter α can be set to any value and is variable according to the properties of the batch input during learning. This indicates that when a certain learning data set is divided into mini-batches with different properties such as X ₁ , X ₂ , ..., X _n for learning, the hyperparameter α is set for the mini-batch X ₁ and the hyperparameter β is set for the mini-batch X ₂ , and the hyperparameter is variable at the mini-batch level. For example, in a task of predicting an individual's weight from their diet, it is possible to set the hyperparameter separately for the case where the input batch is male and the case where the input batch is female. In addition, in a task of predicting human emotions while watching sports, it is possible to set the hyperparameter separately for the time period when the excitement is high and the time period when it is not.

The loss function E can be any function that reduces the output when maintaining the order of paired data. For example, it can be equation (1) as in RankNet (Non-Patent Document 3). It can also be equation (2) as in the rank learning method using Gaussian distribution (Non-Patent Document 4).

(Optimization Method)
Any optimization method such as the gradient method, the stochastic gradient method, Adam, etc. can be applied to optimize the objective function. When using the gradient method, it is sufficient to repeatedly update the parameters according to the following formula in the k-th optimization step.

Here, γ _k is a learning rate parameter. The gradient of the objective function ∇ _θ L (θ) may be a calculated and derived function or may be calculated numerically.

(Model parameter estimation)
Next, the procedure and contents of the model parameter estimation process executed by the model learning device 1 will be described with reference to Fig. 3. Fig. 3 is a flow chart showing the procedure and contents of the model parameter estimation process executed by the model learning device 1.

In step S1, the input data processing unit 10 acquires input data. The input data is the multivariate data X and pair label data D described above. The input data processing unit 10 also stores the acquired input data in the input data recording unit 51.

In step S2, the setting parameter processing unit 20 acquires setting parameters. The setting parameters include hyperparameters of the regularization term of the objective function. The hyperparameters include the m-quantile α of the objective variable and a parameter λ that determines the weight of the regularization term. The setting parameters also include a learning rate parameter γ _k used to optimize the objective function. The setting parameter processing unit 20 also stores the acquired setting parameters in the setting parameter recording unit 52.

In step S3, the model parameter learning unit 30 uses the input data recorded in the input data recording unit 51 and the setting parameters recorded in the setting parameter recording unit 52 as inputs, and performs rank learning of the model parameters using an objective function having a loss function and a regularization term. The model parameter learning unit 30 also stores the learned model parameters in the model parameter recording unit 53.

In step S4, the model parameter processing unit 40 reads the model parameters from the model parameter recording unit 53 and outputs them to the external device 2.

(Rank learning)
Next, the process procedure and process contents of the model parameter rank learning executed by the model parameter learning unit 30 will be described with reference to Fig. 4. Fig. 4 is a flow chart showing an example of the process procedure and process contents of the model parameter rank learning executed by the model parameter learning unit 30.

In step S11, the model parameter θ is initialized.

In step S12, the maximum number of iterations of rank learning is set. Also, the number of calculation iterations of rank learning is initialized. That is, the number of calculation iterations is set to 0.

In step S13, the model parameter θ is updated according to equations (3) and (10).

In step S14, the number of calculation iterations is updated. That is, the number of calculation iterations is incremented by 1.

In step S15, it is determined whether the number of calculation iterations has exceeded the maximum number of iterations. If the number of calculation iterations has not exceeded the maximum number of iterations, the process returns to step S13. If the number of calculation iterations has exceeded the maximum number of iterations, the learning of the model parameters is terminated, and the process returns to step S4 in FIG. 3.

[effect]
According to the embodiment, even if only an order relationship is obtained as label data, it is possible to predict the objective variable from the order relationship. This makes it possible to learn an arbitrary rank learning model not only as a ranking generation model but also as a regression model that outputs a numerical value that approximates the distribution of the objective variable. This makes it possible to obtain not only an order relationship but also an estimated value as a numerical value for objective variables that are difficult to evaluate absolutely, such as emotions, when a survey response cannot be given in a specific numerical value due to psychological resistance.

In the above embodiment, an example is shown in which a gradient method is used during optimization, but any method such as a stochastic gradient method or Adam can be used. Similarly, any loss function can be used for E in the objective function (Equation 3). In the model learning device 1 shown in FIG. 1 of the above embodiment, the operation of each component can be constructed as a program, and the program can be installed and executed on a computer used as the model learning device, or distributed via a network. The present invention is not limited to the above embodiment, and various modifications and applications are possible.

For example, the above embodiment is an example of applying the present invention to a pairwise rank learning method, but the present invention is not limited to this and may be applied to other rank learning methods, such as a pointwise rank learning method or a listwise rank learning method.

Note that the present invention is not limited to the above-described embodiments, and can be modified in various ways in the implementation stage without departing from the gist of the invention. The embodiments may also be implemented in appropriate combination, in which case the combined effects can be obtained. Furthermore, the above-described embodiments include various inventions, and various inventions can be extracted by combinations selected from the multiple constituent elements disclosed. For example, if the problem can be solved and an effect can be obtained even if some constituent elements are deleted from all the constituent elements shown in the embodiments, the configuration from which these constituent elements are deleted can be extracted as an invention.

REFERENCE SIGNS LIST 1 model learning device 2 external device 10 input data processing unit 20 setting parameter processing unit 30 model parameter learning unit 40 model parameter processing unit 50 recording unit 51 data recording unit 52 setting parameter recording unit 53 model parameter recording unit 60 input/output unit 110 input/output interface 120 CPU
121: control device 122: arithmetic unit 130: storage device 131: main storage device 132: auxiliary storage device 140: bus

Claims (8)

An input data processing unit for acquiring input data;
a configuration parameter processing unit that acquires configuration parameters;
a model parameter learning unit that uses the input data and the setting parameters as inputs, and rank-learns model parameters using an objective function that is composed of a loss function that maintains an order relationship and a regularization term that limits an output of a model;
A model parameter processing unit that outputs learned model parameters;
The input data is observable multivariate data and label data indicating an order relationship;
The setting parameters include hyperparameters of the regularization terms,
The model parameter learning unit obtains model parameters that output a score for predicting a dependent variable.
Model learning device. The hyperparameter includes a quantile of the objective variable, where the quantile is a number that exists at a position where a set of sorted numbers is equally divided by positive integers;
the model parameter learning unit approximates a quantile of an output value of a model to the quantile of the response variable;
The model learning device according to claim 1 . The model parameter learning unit minimizes an objective function including a term calculated using the quantiles of the objective variable and the quantiles of the output value of the model.
The model learning device according to claim 2 . The hyperparameters further include a parameter that determines a weight of the regularization term.
The model learning device according to claim 2 or 3. The model parameter learning unit optimizes the objective function using a gradient method, a stochastic gradient method, or Adam.
5. The model learning device according to claim 1. The setting parameters further include a learning rate parameter used in optimizing the objective function.
The model learning device according to claim 5 . Obtaining input data;
Obtaining configuration parameters;
rank-learning model parameters using an objective function including a loss function for maintaining an order relationship and a regularization term for limiting an output of a model, with the input data and the setting parameters as inputs;
Outputting the learned model parameters;
The input data is observable multivariate data and label data indicating an order relationship;
The setting parameters include hyperparameters of the regularization terms,
The rank learning obtains model parameters that output scores predicting the dependent variable.
A computer implemented method for learning models. A model learning program that causes a computer to execute the functions of each component of the model learning device described in any one of claims 1 to 6.

Images