JP7761166B2 - Meta-learning method, meta-learning device, and program - Google Patents

Description

The present disclosure relates to a meta-learning method, a meta-learning device, and a program.

In machine learning methods, model parameters are typically trained using task-specific training datasets. Achieving high performance on a target task requires a large amount of task-specific training data, but preparing a sufficient amount of training data can be costly depending on the task.

To solve the above problems, meta-learning methods have been proposed that utilize training data from different tasks to achieve high performance on the target task even with a small amount of training data (see, for example, Non-Patent Document 1). However, existing meta-learning methods have the problem that they cannot be applied to prediction methods that combine multiple machine learning methods.

Related technologies that have been proposed include a technique for estimating causal effects by combining multiple regression models (see, for example, non-patent document 2) and a technique for estimating uncertainty using a regression model and subsequent calibration (see, for example, non-patent document 3).

Finn, Chelsea, Pieter Abbeel, and Sergey Levine. "Model-agnostic meta-learning for fast adaptation of deep networks." Proceedings of the 34th International Conference on Machine Learning, 2017. Curth, Alicia, and Mihaela van der Schaar. "Nonparametric estimation of heterogeneous treatment effects: From theory to learning algorithms." International Conference on Artificial Intelligence and Statistics. 2021. Marx, Charles, et al. "Modular Conformal Calibration." International Conference on Machine Learning. 2022.

This disclosure has been made in consideration of the above points and aims to provide a technology for meta-learning model parameters for a method that combines multiple machine learning methods to make predictions.

A meta-learning method according to one aspect of the present disclosure includes a computer-implemented process comprising: a first input step of inputting training datasets for multiple tasks; a selection step of selecting one task from the multiple tasks; a creation step of creating a support set consisting of at least a portion of the training data in the training dataset for the one task and a query set consisting of features included in at least a portion of the training data in the training dataset; a first estimation step of estimating a first parameter representing a parameter for each task from among parameters possessed by each of multiple machine learning models so as to conform to the support set; and a learning step of learning a second parameter representing a parameter common to all tasks from among parameters possessed by each of the multiple machine learning models so as to reduce the error between a predicted value predicted from the first feature included in the query set by combining the multiple machine learning models and the training data for the predicted value.

A technology is provided for meta-learning model parameters for prediction methods that combine multiple machine learning methods.

FIG. 1 is a diagram illustrating an example of the hardware configuration of a meta-learning device according to an embodiment of the present invention. FIG. 1 is a diagram illustrating an example of the functional configuration of a meta-learning device according to an embodiment of the present invention. 10 is a flowchart illustrating an example of a meta-learning process according to the present embodiment. 10 is a flowchart illustrating an example of a prediction process according to the present embodiment.

An embodiment of the present invention will be described below. In the following embodiment, a meta-learning device 10 will be described that, when given a learning dataset for multiple tasks, meta-learns model parameters for a method that combines multiple machine learning methods to make predictions. A case will also be described in which the meta-learning device 10 uses the model parameters learned by this meta-learning to predict a target task.

As an example, the following describes a method for estimating (predicting) causal effects by combining multiple machine learning methods. However, this is just one example, and the following embodiment can be similarly applied to other problems, such as estimating (predicting) uncertainty, as long as the method combines multiple machine learning methods to make predictions.

Here, the meta-learning device 10 has a "meta-learning time" during which it meta-learns model parameters common to multiple tasks (hereinafter also referred to as task-common parameters), and a "prediction time" during which it estimates causal effects in a target task using the task-common parameters learned during meta-learning. Note that "meta-learning time" may also be referred to simply as "learning time," for example. Furthermore, "prediction time" may also be referred to as "inference time" or "test time," for example.

During meta-learning, the meta-learning device 10 is given a set of training datasets D = {D _{t | t = 1, ..., T} for T tasks, where D t =} {(x _tn , a _tn , y _tn ) | n = 1, ..., N _t } is the training dataset for the t-th task. Here, x _tn represents the n-th feature of the t-th task, a _tn represents the n-th action of the t-th task, and y _tn represents the n-th result of the t-th task. Furthermore, N _t represents the number of training data sets included in the training dataset D _t for the t-th task. Hereinafter, the set of training datasets for T tasks will also be referred to as the "meta-learning dataset." Furthermore, the t-th task will also be referred to as "task t."

At the time of prediction, the meta-learning device 10 is provided with a dataset S' = {( _xnS ^{', anS'} _{, ynS} ^' ) | n = 1, ..., ^{Ns' }} of an unknown task (i.e., a task (target task) different from any of the first to Tth tasks). Here, _xnS ^' represents the nth feature of the target task, _anS ^' represents the nth action of the target task, and _ynS ^' represents the nth result of the target task. Furthermore, Ns ^' represents the number of data included in the dataset S'. In this case, when a feature x' of the target task is given (i.e., when a feature x' whose action and result are unknown is given), the objective is to estimate the causal effect of that feature x'. Hereinafter, the dataset S' will also be referred to as a "support set S'."

In the following, we assume that the action takes on a binary value of 0 or 1 (for example, 1 if the action was taken, and 0 if the action was not taken), but it may also take on multiple values. Furthermore, we assume that the feature is a vector representing the characteristics of the object that causes the causal relationship, but the format is not limited to a vector and may be, for example, an image or graph.

In the following embodiment, to estimate causal effects, three models (treatment probability estimation model, outcome estimation model, and causal effect estimation model) are used, each realized by a different machine learning method and having task-common parameters and task-specific model parameters (hereinafter also referred to as task-specific parameters). The task-specific parameters of these three models are estimated to fit a certain dataset (support set S, described below, during meta-learning, and support set S' during prediction). Meanwhile, task-common parameters are estimated (learned) during meta-learning to increase the accuracy of estimating causal effects. Then, during prediction, the task-specific parameters are estimated to fit support set S' using the three models with the learned task-common parameters and support set S', and the causal effect of feature x' of the target task is estimated using the causal effect estimation model.

In the following embodiment, we will explain the case where the same meta-learning device 10 is used for both meta-learning and prediction, but meta-learning and prediction may be performed by different devices. In this case, the device that performs prediction may be called, for example, a "prediction device" or an "inference device."

<Example of hardware configuration of meta-learning device 10>
An example of the hardware configuration of a meta-learning device 10 according to this embodiment is shown in Figure 1. As shown in Figure 1, the meta-learning device 10 according to this embodiment is realized by the hardware configuration of a general computer or computer system, and includes, for example, an input device 101, a display device 102, an external I/F 103, a communication I/F 104, a RAM (Random Access Memory) 105, a ROM (Read Only Memory) 106, an auxiliary storage device 107, and a processor 108. Furthermore, each of these pieces of hardware is connected to each other via a bus 109 so that they can communicate with each other.

The input device 101 is, for example, a keyboard, a mouse, a touch panel, a physical button, etc. The display device 102 is, for example, a display, a display panel, etc. Note that the meta-learning device 10 may not have at least one of the input device 101 and the display device 102, for example.

The external I/F 103 is an interface with external devices such as the recording medium 103a. The meta-learning device 10 can read and write data from and to the recording medium 103a via the external I/F 103. Examples of the recording medium 103a include a flexible disk, a CD (Compact Disc), a DVD (Digital Versatile Disk), an SD memory card (Secure Digital memory card), and a USB (Universal Serial Bus) memory card.

The communication I/F 104 is an interface for connecting the meta-learning device 10 to a communication network, etc. The RAM 105 is a volatile semiconductor memory (storage device) that temporarily stores programs and data. The ROM 106 is a non-volatile semiconductor memory (storage device) that can store programs and data even when the power is turned off. The auxiliary storage device 107 is a storage device (storage device) such as an HDD (Hard Disk Drive), SSD (Solid State Drive), or flash memory. The processor 108 is an arithmetic device such as a CPU (Central Processing Unit) or GPU (Graphics Processing Unit).

The meta-learning device 10 according to this embodiment has the hardware configuration shown in FIG. 1, and is therefore capable of performing the various processes described below. Note that the hardware configuration shown in FIG. 1 is merely an example, and the hardware configuration of the meta-learning device 10 is not limited to this. For example, the meta-learning device 10 may have multiple auxiliary storage devices 107 or multiple processors 108, may not have some of the hardware shown in the figure, or may have various hardware other than the hardware shown in the figure.

<Example of functional configuration of meta-learning device 10>
An example of the functional configuration of the meta-learning device 10 according to this embodiment is shown in Figure 2. As shown in Figure 2, the meta-learning device 10 according to this embodiment includes an input unit 201, a pseudo-causal effect estimation unit 202, a meta-learning unit 203, a treatment probability estimation unit 204, an outcome estimation unit 205, a pseudo-outcome estimation unit 206, a causal effect estimation unit 207, and an output unit 208. Each of these units is realized, for example, by a processor 108 or the like executing one or more programs installed in the meta-learning device 10. The meta-learning device 10 according to this embodiment also includes a memory unit 209. The memory unit 209 may be realized, for example, by a storage device such as a database server connected to the meta-learning device 10 via a communication network or the like.

During meta-learning, the input unit 201 inputs a given meta-learning dataset D. During prediction, the input unit 201 inputs a given support set S' and a feature x' whose treatment and outcome are unknown.

During meta-learning, the pseudo-causal effect estimation unit 202 uses the learning data for each task to estimate the causal effect using an existing causal effect estimation method. Hereinafter, the causal effect estimated by the pseudo-causal effect estimation unit 202 will be referred to as the "pseudo-causal effect." Furthermore, the nth pseudo-causal effect of the tth task will be

This pseudo-causal effect is used as training data for the feature x _tn ^q included in the query set Q _t , which will be described later. Hereinafter, in the text of the specification, the pseudo-causal effect will be represented as " ^∼ τ _tn ".

The pseudo-causal effect estimation unit 202 may estimate the pseudo-causal effect using any existing causal effect estimation method, such as a DR-learner (doubly robust learner) or RA-learner (regression adjustment learner) as described in Non-Patent Document 2.

During meta-learning, the meta-learning unit 203 randomly selects one task t from the first task to the Tth task, and then creates a support set S _t and a query set Q _t from the learning dataset D _t for the selected task t.

More specifically, first, the meta-learning unit 203 randomly selects one t∈{1, ..., T}. Next, the meta-learning unit 203 randomly selects a random number of training data from the training dataset _Dt for task t, and then renumbers the numbers of the training data to create a support set _St = {( _xtnS ^, _atnS , ^ytnS )|n = 1, ..., ^{NtS }} . Furthermore, the meta-learning _unit 203 randomly selects a random number of training data from the training dataset Dt for task _t , and then renumbers the numbers of the feature _xtn and its pseudo-causal effect ^~ _τtn included in the training data to create a query set _Qt = {( _xtnq ^{, ~} _τtnq )|n = 1, ..., _Ntq }. Here, _NtS is the number of ^data included in the support ^set _St , and _Ntq is the number of _data included in _the query ^{set Qt} .

In the following, for simplicity, we will omit the "t" representing the task, and denote the support set as S = {( _xnS ^, _anS , ^ynS ) ^| n = 1, ..., ^Ns }, and the query set as Q = {( _xnq ^{, ~} _τnq ) | n ⁼ 1 _, ..., ^Nq }.

Furthermore, during meta-learning, the meta-learning unit 203 updates task-common parameters using an optimization method such as a stochastic gradient method so that the causal effect (causal effect estimate) of the feature x _n ^q estimated by the causal effect estimation model when the support set S is fitted to three models (treatment probability estimation model, outcome estimation model, and causal effect estimation model) becomes closer to the pseudo-causal effect ^∼ τ _n ^q . Note that squared error, mean squared error, or the like can be used as a measure of the closeness (distance) between the causal effect estimate of the feature x _n ^q and the pseudo-causal effect ^∼ τ _n ^q .

The treatment probability estimation unit 204 is realized by a treatment probability estimation model. During meta-learning, the treatment probability estimation unit 204 estimates the probability of treatment (hereinafter also referred to as treatment probability) for the feature x _n ^S when the treatment probability estimation model is adapted to the support set S. Furthermore, during prediction, the treatment probability estimation unit 204 estimates the treatment probability for the feature x _n ^S' when the treatment probability estimation model is adapted to the support set S'.

The task-specific parameters of the treatment probability estimation model are estimated to increase the accuracy of treatment probability estimation in the support set (support set S during meta-learning, support set S' during prediction). For example, the model shown in the following equation (1) can be used as the treatment probability estimation model.

Here, π(x) is the probability that the treatment is 1 when the feature is x, and f _p (x) represents a neural network with a task-common parameter φ _p . θ _pa represents a task-specific parameter and is the mean vector of the features (feature x _n ^S during meta-learning, feature x _n S' during prediction) whose treatment value (treatment a n ^S during meta-learning, treatment _{a n} ^{S' during} prediction) is a∈{0, 1}.

In the case of the treatment probability estimation model shown in equation (1) above, the task-specific parameters simply require calculating the average of the feature quantities, enabling efficient adaptation to the support set. However, using the model shown in equation (1) above as a treatment probability estimation model is just one example, and other models, such as a feedforward neural network, can also be used as a treatment probability estimation model.

The result estimation unit 205 is realized by a result estimation model. During meta-learning, the result estimation unit 205 estimates the feature x _n ^S and the result for the treatment aε{0, 1} when the result estimation model is adapted to the support set S. Furthermore, during prediction, the result estimation unit 205 estimates the feature x _n ^S′ and the result for the treatment aε{0, 1} when the result estimation model is adapted to the support set S′.

The task-specific parameters of the outcome estimation model are estimated to increase the accuracy of outcome estimation in the support set (support set S during meta-learning, support set S' during prediction). For example, the model shown in the following equation (2) can be used as the outcome estimation model.

Here, μ _a (x) is the result when feature x and treatment aε{0, 1}, f _a (x) is a neural network with task-common parameter φ _a , and θ _a is a task-specific parameter.

In the case of the result estimation model shown in equation (2) above, since the task-specific parameters are a linear model, the optimal task-specific parameters that fit the support set can be analytically determined using the least squares method. This allows for efficient adaptation to the support set. However, using the model shown in equation (2) above as the result estimation model is just one example; for example, a Gaussian process could also be used instead of a linear model. It is also possible to use other models, such as a feedforward neural network, as the result estimation model.

During meta-learning and prediction, the pseudo-result estimation unit 206 uses the treatment probability estimated by the treatment probability estimation unit 204, the result estimated by the result estimation unit 205, and a support set (support set S during meta-learning, support set S' during prediction) to estimate a result (hereinafter also referred to as a pseudo-result) for the feature quantities of that support set. At this time, the pseudo-result estimation unit 206 estimates the pseudo-result so that the expected value of the pseudo-result and the expected value of the causal effect are close to each other. For example, during meta-learning, the pseudo-result estimation unit 206 can estimate the pseudo-result using the following equation (3):

Here, when making a prediction, "S" in the above formula (3) can be read as "S'^" . Hereinafter, in the text of the specification, the pseudo result for the feature _xnS will be expressed as " ^~ _ynS ^" . Similarly, the pseudo result for the feature _xnS ^' will be expressed as " ^~ _ynS ^' ".

In the above formula (3), both the treatment probability estimation model π(·) and the outcome estimation model μ _a (·) are used, but only one of them may be used. Even when only one of the treatment probability estimation model π(·) and the outcome estimation model μ _a (·) is used, it is sufficient to estimate the pseudo-outcome so that the expected value of the pseudo-outcome and the expected value of the causal effect are close to each other.

The causal effect estimation unit 207 is realized by a causal effect estimation model. During meta-learning, the causal effect estimation unit 207 uses the pseudo results estimated by the pseudo result estimation unit 206 to estimate the causal effect of the feature x _n ^q when the causal effect estimation model is adapted to the support set S. During prediction, the causal effect estimation unit 207 uses the pseudo results estimated by the pseudo result estimation unit 206 to estimate the causal effect of the feature x' when the causal effect estimation model is adapted to the support set S'.

The task-specific parameters of the causal effect estimation model are estimated so that the pseudo result for the feature ( ^{the feature xnS} _during meta-learning, and the feature _xnS ^' during prediction) and the causal effect of the feature estimated by the causal effect estimation model (the causal effect estimate) are close to each other. For example, the model shown in the following formula (4) can be used as the causal effect estimation model.

Here, τ(x) represents the causal effect when the feature quantity is x, f _y (x) represents a neural network having a task-common parameter φ _y , and θ _y represents a task-specific parameter.

In the case of the causal effect estimation model shown in equation (4) above, the task-specific parameters are a linear model, so the optimal task-specific parameters that fit the support set can be analytically determined using the least squares method. This allows for efficient adaptation to the support set. However, using the model shown in equation (4) above as the causal effect estimation model is just one example; for example, a Gaussian process could also be used instead of a linear model. It is also possible to use other models, such as a feedforward neural network, as the causal effect estimation model.

During prediction, the output unit 208 outputs the causal effect of the feature x' estimated by the causal effect estimation unit 207 to a predetermined output destination. Examples of the predetermined output destination include the memory unit 209, a display device 102 such as a display, and other devices connected via a communication network.

The memory unit 209 stores various data (e.g., meta-learning dataset D, support set S' for the target task, feature values x' for the target task, task-specific parameters, task-common parameters, etc.).

<Meta-learning processing>
An example of the meta-learning process according to this embodiment will be described below with reference to FIG.

First, the input unit 201 inputs the given meta-learning dataset D (step S101).

Next, the pseudo-causal effect estimation unit 202 uses the training data (x _tn , a _tn , y _tn ) of each task to estimate the pseudo-causal effect ^∼ τ _tn for the training data (x _tn , a _tn , y _tn ) by an existing causal effect estimation method (step S102). This results in a set of pseudo-causal effects {{ ^∼ τ _tn |n=1, ..., N _t }|t=1, ..., T}.

Next, the meta-learning unit 203 randomly selects one task t from the first task to the Tth task, and creates a support set S = _{ ( _xnS , _anS , _ynS )|n = 1, ..., ^Ns } and a query set Q = {( ^xnq , ^{... τnq )} |n = 1, ..., _Nq ^} from the learning dataset Dt _for the ^selected task ^t (step S103).

Next, the treatment probability estimation unit 204 estimates the treatment probability for the feature x _n ^S when the treatment probability estimation model is adapted to the support set S (step S104). That is, the treatment probability estimation unit 204 first estimates task-specific parameters of the treatment probability estimation model, and then estimates the treatment probability for the feature x _n ^S using the treatment probability estimation model having these task-specific parameters.

For example, when using the treatment probability estimation model shown in the above formula (1), the treatment probability estimation unit 204 first obtains θ _p0 , the average vector of feature quantity x _n ^S where a _n ^S = 0, and θ _p1 , the average vector of feature quantity x _n ^S where a _n ^S = 1. Next, the treatment probability estimation unit 204 estimates the treatment probability π(x _n ^S ) using the treatment probability estimation model shown in the above formula (1). This obtains a set of treatment probabilities {π(x _n ^S )|n=1, ..., N ^S }.

Next, the result estimation unit 205 estimates the feature x _n ^S and the result for the action aε{0, 1} when the result estimation model is adapted to the support set S (step S105). That is, the result estimation unit 205 first estimates task-specific parameters of the result estimation model, and then estimates the feature x _n ^S and the result for the action aε{0, 1} using the result estimation model having these task-specific parameters.

For example, when using the result estimation model shown in the above equation (2), the result estimation unit 205 first estimates the task-specific parameter θ _a for a←a _n ^S so that μ _a (x _n ^S ) and the corresponding result y _n ^S are close to each other. This can be analytically determined, for example, by the least squares method. Next, the result estimation unit 205 estimates the result μ _a (x _n ^S ) using the result estimation model shown in the above equation (2). This results in a set of results {{μ _a (x _n ^S )|n=1, ..., N ^S }|a=0, 1}.

Next, the pseudo result estimation unit 206 estimates a pseudo result for the feature x _n ^S using the treatment probability estimated in the above step S104, the result estimated in the above step S105, and the support set S (step S106).

For example, the pseudo-result estimation unit 206 estimates _{the pseudo-result ∼ ynS by the above formula (3) using the treatment probability π(xnS), the results μ0(xnS} ^{) and μ1 (} _{xnS )} ^, _{and ( xnS} ^, _anS ^, _ynS ) included in the support set S. ^This results in a set of pseudo-results { ^{∼ ynS} | _n = ₁ , ..., ^Ns }.

Next, the causal effect estimation unit 207 estimates the causal effect of the feature x _n ^q when the causal effect estimation model is adapted to the support set S (step S107). That is, the causal effect estimation unit 207 first estimates task-specific parameters of the causal effect estimation model, and then estimates the causal effect of the feature x _n ^q using the causal effect estimation model having these task-specific parameters.

For example, when using the causal effect estimation model shown in the above formula (4), the causal effect estimation unit 207 first estimates the task-specific parameter θ _y so that the causal effect estimate τ(x _n ^S ) and the corresponding pseudo result ^∼ _yn ^S are close to each other. This can be analytically determined, for example, by the least squares method. Next, the causal effect estimation unit 207 estimates the causal effect estimate τ(x _n ^q ) using the causal effect estimation model shown in the above formula (4). This obtains a set of causal effect estimates {τ(x _n ^q )|n=1, ..., N ^q }.

Next, the meta-learning unit 203 updates the task-common parameters so that the causal effect estimate estimated in step S107 above and the corresponding pseudo-causal effect become closer (step S108).

For example, the meta-learning unit 203 updates the task-common parameters φ _p , φ _a , and φ _y so that the causal effect estimate τ(x _n ^q ) and the corresponding pseudo-causal effect ^∼ τ _n ^q become closer to each other. This can be achieved by using an optimization method such as a gradient method.

Next, the meta learning unit 203 determines whether a predetermined termination condition is met (step S109).

If it is determined in step S109 above that the termination condition is not met, the meta-learning unit 203 returns to step S103. As a result, steps S103 to S108 are repeatedly executed until the termination condition is met. On the other hand, if it is determined in step S109 above that the termination condition is met, the meta-learning unit 203 terminates the meta-learning process. As a result, learned task-common parameters are obtained.

The above termination conditions can be, for example, "the update amount of the task common parameters is less than a predetermined threshold," "the number of repetitions of steps S103 to S108 is greater than or equal to a predetermined threshold," or "the decrease in the value of the objective function (e.g., the squared error or mean squared error between the causal effect estimate and the pseudo-causal effect) is less than a predetermined threshold."

<Prediction processing>
An example of the prediction process according to this embodiment will be described below with reference to Fig. 4. Note that in the following, it is assumed that the task-common parameters have already been learned.

First, the input unit 201 inputs a given support set S'={( _xnS ^' , _anS ^' , _ynS ^' )|n=1, . . . , N ^S' } and a feature x' whose treatment and result are unknown (step S201).

Next, the treatment probability estimation unit 204 estimates the treatment probability for the feature x _n ^S' when the treatment probability estimation model is adapted to the support set S' (step S202). That is, the treatment probability estimation unit 204 first estimates task-specific parameters of the treatment probability estimation model, and then estimates the treatment probability for the feature x _n ^S' using the treatment probability estimation model having these task-specific parameters.

For example, when using the treatment probability estimation model shown in the above formula (1), the treatment probability estimation unit 204 first obtains θ _p0 , the average vector of the feature x _n ^S' where a _n ^S' = 0, and θ _p1 , the average vector of the feature x _n ^S' where a _n ^S' = 1. Next, the treatment probability estimation unit 204 estimates the treatment probability π(x _n ^S' ) using the treatment probability estimation model shown in the above formula (1). This obtains a set of treatment probabilities {π(x _n ^S' )|n=1, ..., N ^S' }.

Next, the result estimation unit 205 estimates the feature x _n ^S′ and the result for the action aε{0, 1} when the result estimation model is adapted to the support set S′ (step S203). That is, the result estimation unit 205 first estimates task-specific parameters of the result estimation model, and then estimates the feature x _n ^S′ and the result for the action aε{0, 1} using the result estimation model having these task-specific parameters.

For example, when using the result estimation model shown in the above equation (2), the result estimation unit 205 first estimates the task-specific parameter θ a for a←a _n ^S' so that μ _a (x _n ^S' ) and the corresponding result y _n ^S' are close to each other. This can be analytically determined, for example, by the least squares method. Next, the result estimation unit 205 estimates the result μ _a (x _n ^S' ) using the result estimation model shown in the above equation ( _{2). This results in a set of results {{μ a (} x _n ^S' )|n=1, ..., N ^S' }|a=0, 1}.

Next, the pseudo result estimation unit 206 estimates a pseudo result for the feature x _n ^S′ using the treatment probability estimated in the above step S202, the result estimated in the above step S203, and the support set S′ (step S204).

For example, the pseudo-result estimation unit 206 estimates the pseudo-result ∼ ynS' using the treatment probability π( _xnS ^' ), the outcomes _μ0 ( _xnS ^' ), _μ1 ( _xnS ^' ), and ( ^{xnS '} , _{anS '} , _ynS ^' ) included in the support set ^S' using the above formula (3). This results in a set ^{of pseudo-results {∼ ynS} _' ^| n= ₁ , ..., ^Ns' }.

Next, the causal effect estimation unit 207 estimates the causal effect of the feature x' when the causal effect estimation model is adapted to the support set S' (step S205). That is, the causal effect estimation unit 207 first estimates the task-specific parameters of the causal effect estimation model, and then estimates the causal effect of the feature x' using the causal effect estimation model having these task-specific parameters.

For example, when using the causal effect estimation model shown in the above formula (4), the causal effect estimation unit 207 first estimates the task-specific parameter θ _y so that the causal effect estimate τ(x _n ^S' ) and the corresponding pseudo result ^∼ y _n ^S' are close to each other. This can be analytically determined, for example, by the least squares method. Next, the causal effect estimation unit 207 estimates the causal effect estimate τ(x') using the causal effect estimation model shown in the above formula (4). This obtains the causal effect estimate τ(x') for the feature quantity x' whose treatment and outcome are unknown.

Then, the output unit 208 outputs the causal effect estimate τ(x') estimated in step S205 above to a predetermined output destination (step S206).

<Evaluation>
To evaluate the meta-learning device 10 according to this embodiment, we used artificial data to compare it with existing methods. As existing methods, we adopted the existing meta-learning method, multi-task learning method, and task-specific learning method.

The evaluation index used was the mean squared error between the estimated causal effect and the correct answer. The evaluation results (mean and standard deviation) are shown in Table 1 below.

Here, the proposed method represents the meta-learning device 10 according to this embodiment.

As shown in Table 1 above, it can be seen that the meta-learning device 10 of this embodiment achieves a lower error compared to existing methods.

<Summary>
As described above, when given training datasets for multiple tasks, the meta-learning device 10 of this embodiment creates a support set S and a query set Q from the training datasets for each task, and then estimates task-specific parameters by fitting them to multiple models realized by multiple different machine learning methods. The meta-learning device 10 of this embodiment then estimates task-common parameters for each model so that the final prediction result is close to its pseudo-ground truth. This makes it possible to achieve high performance on a target task in a prediction method that combines multiple machine learning methods, even when only a small amount of training data for the target task is provided.

In the above embodiment, the task-specific parameters of the treatment probability estimation model were estimated by the treatment probability estimation unit 204, but the task-specific parameters of the treatment probability estimation model may also be estimated by the meta-learning unit 203. Similarly, the task-specific parameters of the result estimation model were estimated by the result estimation unit 205, but the task-specific parameters of the result estimation model may also be estimated by the meta-learning unit 203. Similarly, the task-specific parameters of the causal effect estimation model were estimated by the causal effect estimation unit 207, but the task-specific parameters of the causal effect estimation model may also be estimated by the meta-learning unit 203.

The present invention is not limited to the specifically disclosed embodiments above, and various modifications, alterations, and combinations with known technologies are possible without departing from the scope of the claims.

10 Meta-learning device 101 Input device 102 Display device 103 External I/F
103a Recording medium 104 Communication I/F
105 RAM
106 ROM
107 Auxiliary storage device 108 Processor 109 Bus 201 Input unit 202 Pseudo causal effect estimation unit 203 Meta-learning unit 204 Treatment probability estimation unit 205 Outcome estimation unit 206 Pseudo result estimation unit 207 Causal effect estimation unit 208 Output unit 209 Storage unit

Claims (8)

a first input step of inputting training datasets for multiple tasks;
a selection step of selecting one task from the plurality of tasks;
a creation step of creating a support set consisting of at least a portion of training data in a training dataset for the one task, and a query set consisting of features included in at least a portion of the training data in the training dataset;
a first estimation procedure for estimating first parameters representing parameters for each task among parameters of each of a plurality of machine learning models so as to conform to the support set;
a learning procedure for learning a second parameter representing a task-common parameter among parameters possessed by each of the plurality of machine learning models, so that an error between a predicted value predicted from a first feature included in the query set by combining the plurality of machine learning models and training data for the predicted value is small;
A meta-learning method performed by a computer. a second input step of inputting a dataset consisting of data of a target task and a second feature quantity whose predicted value is to be predicted in the target task;
a second estimation procedure for estimating the first parameter among parameters of each of the plurality of machine learning models so as to fit the dataset;
a prediction step of predicting a predicted value from the second feature amount by combining the plurality of machine learning models;
The meta-learning method of claim 1 , further comprising the steps of: the learning data includes the first feature amount, a treatment for an object having a feature represented by the first feature amount, and a result of the object, in association with each other;
3. The meta-learning method according to claim 1, wherein the plurality of machine learning models include a treatment probability estimation model that receives input features and estimates the probability that a treatment has been performed, an outcome estimation model that receives input features and a treatment and estimates an outcome, and a causal effect estimation model that receives input features and estimates a causal effect as the predicted value. the treatment probability estimation model has the first parameter θ _p and the second parameter φ _p , the outcome estimation model has the first parameter θ _a and the second parameter φ _a , and the causal effect estimation model has the first parameter θ _y and the second parameter φ _y ;
The first estimation procedure includes:
Estimating the first parameters θ _p , θ _a, and θ _y to fit the support set;
The learning procedure includes:
The meta-learning method according to claim 3 , wherein the second parameters φ _p, φ _a, and φ _y are learned so as to reduce the error. the computer further executes a pseudo-outcome estimation step of estimating a pseudo-outcome using a probability estimated by inputting the first feature into the treatment probability estimation model and a result estimated by inputting the first feature and a possible value of the treatment into the outcome estimation model;
The first estimation procedure includes:
an average of the first feature amounts associated with the same treatment is estimated as the first parameter θ _p ;
estimating the first parameter θ a so as to reduce an error between a result estimated by inputting the first feature amount and a treatment associated with the first feature amount into the result estimation model and _a result associated with the first feature amount;
5. The meta-learning method according to claim 4, wherein the first parameter _θy is estimated so as to reduce an error between the pseudo result and a causal effect estimated by inputting the first feature into the causal effect estimation model. The meta-learning method described in claim 5, wherein the computer further executes a pseudo-causal effect estimation procedure that uses a causal estimation method to estimate a pseudo-causal effect between the first feature contained in the learning data and the result as the training data. a first input unit for inputting training datasets for a plurality of tasks;
a selection unit that selects one task from the plurality of tasks;
a creation unit that creates a support set configured with at least a portion of training data of a training dataset for the one task, and a query set that is configured with features included in at least a portion of the training data of the training dataset;
a first estimation unit that estimates first parameters representing parameters for each task among parameters possessed by each of a plurality of machine learning models so as to conform to the support set;
a learning unit that learns a second parameter representing a task-common parameter among parameters possessed by each of the plurality of machine learning models so as to reduce an error between a predicted value predicted from a first feature included in the query set by combining the plurality of machine learning models and training data for the predicted value;
A meta-learning device having a first input step of inputting training datasets for multiple tasks;
a selection step of selecting one task from the plurality of tasks;
a creation step of creating a support set consisting of at least a portion of training data in a training dataset for the one task, and a query set consisting of features included in at least a portion of the training data in the training dataset;
a first estimation procedure for estimating first parameters representing parameters for each task among parameters of each of a plurality of machine learning models so as to conform to the support set;
a learning procedure for learning a second parameter representing a task-common parameter among parameters possessed by each of the plurality of machine learning models, so that an error between a predicted value predicted from a first feature included in the query set by combining the plurality of machine learning models and training data for the predicted value is small;
A program that causes a computer to execute the following.