JP7616361B2 - Learning method, inference method, learning device, inference device, and program - Google Patents

Description

Article 30, paragraph 2 of the Patent Act is applicable. Published in the Technical Research Report of the Institute of Electronics, Information and Communication Engineers, Vol. 120, No. 195, pages 1-7, Document No. IBISML2020-8, published on the website https://www.ieice.org/ken/paper/20201020VCa4/ on October 13, 2020.

The present invention relates to a learning method, an inference method, a learning device, an inference device, and a program.

The task of estimating certain information using multidimensional data with multiple attributes such as year, month, gender, age, etc. is widely performed.

Known techniques for estimating information using multidimensional data include, for example, multi-layer perceptron (MLP) and N-dimensional convolution (see non-patent document 1).

"N-dimensional convolution", Internet <URL: https://jp.mathworks.com/help/matlab/ref/convn.html>

In the above MLP, inference is performed using only a set of attribute values. For example, specific information (e.g., number of contracts) is estimated using only a set of attribute values such as (year/month, gender, age) = (April 2020, male, 30s) as input. However, it is difficult for MLP to capture wide-ranging trends (e.g., periodicity of a time series), and the estimation accuracy may not be high.

In addition, in the above N-dimensional convolution, inference is performed using an attribute value set and an adjacent attribute value set. For example, a collection of attribute value sets such as {(year/month, gender, age)|year/month=March 2020, April 2020, May 2020, gender=male, female, age=20s, 30s, 40s} is input to estimate specific information (e.g., number of contracts). However, N-dimensional convolution is difficult to apply to categorical attributes (e.g., occupation, country name, etc.) that do not have a definition of adjacency. In addition, when the number of dimensions is N, the number of attribute value sets adjacent to a certain attribute value set is 3 ^N −1, and there is a problem that the number of adjacent sets increases exponentially as the number of dimensions N increases.

One embodiment of the present invention has been made in consideration of the above points, and aims to estimate specific information from multidimensional data with high accuracy and low computational complexity.

To achieve the above-mentioned objective, a learning method according to one embodiment includes an input step of inputting multidimensional data having an estimation target attribute indicating an attribute of an estimation target and two or more non-estimation target attributes indicating attributes other than the estimation target attribute, a dimension reduction step of reducing the number of dimensions of the non-estimation target attribute of the multidimensional data, a binning step of binning values of the non-estimation target attribute of the multidimensional data after the dimension reduction, an information addition step of adding predetermined additional information to the multidimensional data after the binning, and a learning step of learning parameters of an inference model for estimating values of the estimation target attribute using the multidimensional data to which the additional information has been added.

Specific information can be estimated from multidimensional data with high accuracy and low computational effort.

FIG. 2 is a diagram illustrating an example of a hardware configuration of an inference device according to the present embodiment. FIG. 2 is a diagram illustrating an example of a functional configuration of an inference device according to the present embodiment. 10 is a flowchart illustrating an example of a learning process according to the present embodiment. FIG. 1 is a diagram illustrating an example of a multidimensional dataset with correct answers in a table format. FIG. 1 illustrates an example of a dimension-reduced multidimensional dataset with ground truth in tabular form. FIG. 1 illustrates an example of a multidimensional dataset with binned ground truths in tabular form. FIG. 2 illustrates an example of a multidimensional training data set in tabular form. FIG. 1 is a diagram illustrating an example of an inference model. A diagram showing another example of an inference model. 10 is a flowchart illustrating an example of an inference process according to the present embodiment. FIG. 1 illustrates an example of a multidimensional dataset without a correct answer in tabular form. FIG. 1 illustrates an example of a dimensionality-reduced, ground-truth multidimensional dataset in tabular form. FIG. 1 illustrates an example of a binned, ground truth multidimensional dataset in tabular form. FIG. 1 illustrates an example of a multidimensional data set for inference in tabular form.

Hereinafter, an embodiment of the present invention will be described. An inference device 10 that can estimate specific information from multidimensional data with high accuracy and low computational complexity will be described.

<Hardware Configuration>
First, the hardware configuration of an inference device 10 according to this embodiment will be described with reference to Fig. 1. Fig. 1 is a diagram showing an example of the hardware configuration of the inference device 10 according to this embodiment.

1, the inference device 10 according to this embodiment is realized by the hardware configuration of a general computer or computer system, and has an input device 101, a display device 102, an external I/F 103, a communication I/F 104, a processor 105, and a memory device 106. Each of these pieces of hardware is connected to each other via a bus 107 so as to be able to communicate with each other.

The input device 101 is, for example, a keyboard, a mouse, a touch panel, etc. The display device 102 is, for example, a display, etc. Note that the inference device 10 may not have at least one of the input device 101 and the display device 102, for example.

The external I/F 103 is an interface with an external device such as a recording medium 103a. The inference device 10 can read and write data from and to the recording medium 103a via the external I/F 103. Examples of the recording medium 103a include a CD (Compact Disc), a DVD (Digital Versatile Disk), a SD memory card (Secure Digital memory card), and a USB (Universal Serial Bus) memory card.

The communication I/F 104 is an interface for connecting the inference device 10 to a communication network. The processor 105 is, for example, various types of arithmetic devices such as a CPU (Central Processing Unit) or a GPU (Graphics Processing Unit). The memory device 106 is, for example, various types of storage devices such as a HDD (Hard Disk Drive), an SSD (Solid State Drive), a RAM (Random Access Memory), a ROM (Read Only Memory), and a flash memory.

The inference device 10 according to this embodiment has the hardware configuration shown in Fig. 1, and can thereby realize the learning process and inference process described below. Note that the hardware configuration shown in Fig. 1 is an example, and the inference device 10 may have, for example, multiple processors 105 or multiple memory devices 106.

<Functional configuration>
Next, the functional configuration of the inference device 10 according to this embodiment will be described with reference to Fig. 2. Fig. 2 is a diagram showing an example of the functional configuration of the inference device 10 according to this embodiment.

As shown in FIG. 2, the inference device 10 according to this embodiment has a learning unit 201 and an inference unit 202. Each of these units is realized, for example, by a process executed by the processor 105 of one or more programs installed in the inference device 10.

The inference device 10 according to this embodiment also includes a multidimensional data storage unit with correct answers 203, a learning dimensionality reduction model storage unit 204, a learning inference model storage unit 205, a learned dimensionality reduction model storage unit 206, a learned inference model storage unit 207, a multidimensional data storage unit without correct answers 208, and an estimation result storage unit 209. Each of these units is realized, for example, by the memory device 106. At least one of these units may be realized by a storage device or the like connected to the inference device 10 via a communication network.

The learning unit 201 learns a dimension reduction model for reducing the number of dimensions of multidimensional data, and an inference model for inferring specific information from multidimensional data that has undergone dimensional reduction using this dimension reduction model.

The inference unit 202 estimates specific information from multidimensional data using the learned dimensionality reduction model and the learned inference model.

The multidimensional data storage unit 203 with correct answers stores a set of multidimensional data with correct answers (hereinafter also referred to as a multidimensional dataset with correct answers) used when learning the dimensionality reduction model and the inference model. A multidimensional dataset with correct answers is a set of multidimensional data to which the correct answers (i.e., so-called teacher data) of specific information to be estimated by the inference model are attached. Hereinafter, among the attributes of the multidimensional data, attributes that can take the specific information to be estimated as a value are also referred to as "attributes to be estimated" and other attributes are also referred to as "attributes not to be estimated". Specific examples of multidimensional datasets with correct answers will be described later.

The learning dimension reduction model storage unit 204 stores a dimension reduction model (hereinafter also referred to as a learning dimension reduction model) to be learned by the learning unit 201. The dimension reduction model is a model for reducing the number of dimensions of multidimensional data, such as principal component analysis (PCA). In this embodiment, the dimension reduction model is PCA, but this is just one example, and other dimension reduction models may be used.

The learning inference model storage unit 205 stores an inference model (hereinafter also referred to as a learning inference model) to be learned by the learning unit 201. An inference model is a model for estimating attribute values to be estimated from multidimensional data that has been subjected to dimensionality reduction or the like, such as a neural network. Specific examples of inference models will be described later.

The learned dimensionality reduction model memory unit 206 stores the learned dimensionality reduction model learned by the learning unit 201.

The learned inference model memory unit 207 stores the learned inference model learned by the learning unit 201.

The uncorrected multidimensional data storage unit 208 stores a set of uncorrected multidimensional data to which no attribute values to be estimated are assigned, and multidimensional data with correct answers (hereinafter also referred to as an uncorrected multidimensional dataset). An uncorrected multidimensional dataset is a multidimensional dataset that includes multidimensional data to which no attribute values to be estimated are assigned. Specific examples of uncorrected multidimensional datasets will be described later.

The estimation result memory unit 209 stores the estimation result by the inference unit 202 (i.e., the estimated target attribute value estimated from the multidimensional data without a correct answer).

Here, the learning unit 201 includes a dimensionality reduction unit 211, a binning unit 212, an information addition unit 213, and a learning processing unit 214.

The dimensionality reduction unit 211 takes as input a multidimensional dataset with correct answers and a learning dimensionality reduction model, and reduces the number of dimensions of non-estimated target attributes of each multidimensional data contained in the multidimensional dataset with correct answers.

The dimension reduction unit 211 then outputs a set of multidimensional data after the dimension reduction (hereinafter also referred to as a dimension-reduced multidimensional dataset with ground truth) and the trained dimension reduction model. A specific example of the dimension-reduced multidimensional dataset with ground truth will be described later.

The binning unit 212 receives a dimension-reduced multidimensional dataset with correct answers as input and performs binning on the non-estimation target attributes of each multidimensional data included in the dimension-reduced multidimensional dataset with correct answers. The binning unit 212 then outputs a set of multidimensional data after this binning (hereinafter also referred to as a binned multidimensional dataset with correct answers). Note that binning is also called discretization, and is a technique in which the possible range of non-estimation target attributes included in multidimensional data is divided into bins of a certain interval, and the value of the non-estimation target attribute is replaced with a value that represents which bin it belongs to.

The information addition unit 213 uses the binned multidimensional dataset with correct answers as input to create a learning multidimensional dataset in which additional information is added to each piece of multidimensional data included in the binned multidimensional dataset with correct answers. Here, the additional information refers to information consisting of the values of the estimation target attributes when the values of the non-estimation target attributes other than the non-estimation target attribute are kept fixed for each non-estimation target attribute of each piece of multidimensional data included in the binned multidimensional dataset with correct answers and the value of the non-estimation target attribute is changed.

The information addition unit 213 then outputs a learning multidimensional dataset. Specific examples of learning multidimensional datasets will be described later.

The learning processing unit 214 receives the learning multidimensional dataset and the learning inference model as input, learns the learning inference model, and creates a trained inference model. The learning processing unit 214 then outputs the trained inference model.

The inference unit 202 also includes a dimensionality reduction unit 221, a binning unit 222, an information addition unit 223, and an estimation processing unit 224.

The dimension reduction unit 221 receives the uncorrected multidimensional dataset and the learned dimension reduction model as input, and reduces the dimensions of the non-estimated target attributes of each piece of multidimensional data contained in the uncorrected multidimensional dataset. The dimension reduction unit 221 then outputs a set of multidimensional data after this dimension reduction (hereinafter also referred to as the dimension-reduced uncorrected multidimensional dataset).

The binning unit 222 receives the dimension-reduced multidimensional dataset without a correct answer as input, and performs binning on the non-estimated target attributes of each multidimensional data included in the dimension-reduced multidimensional dataset without a correct answer. The binning unit 222 then outputs a set of multidimensional data after this binning (hereinafter also referred to as the binned multidimensional dataset without a correct answer).

The information addition unit 223 takes a binned, correct-answer-free multidimensional dataset as input and creates a multidimensional dataset for inference in which additional information is added to each multidimensional data contained in the binned, correct-answer-free multidimensional dataset.

The estimation processing unit 224 receives the multidimensional data set for inference and the trained inference model as input, and estimates the attribute value to be estimated using the trained inference model. The estimation processing unit 224 then outputs the attribute value to be estimated as the estimation result.

Note that each unit shown in FIG. 1 may be distributed among multiple devices. In particular, for example, the learning unit 201 and the inference unit 202 may be included in different devices. In this case, the device including the learning unit 201 may be referred to as a "learning device."

<Learning process>
Next, the learning process according to this embodiment will be described with reference to Fig. 3. Fig. 3 is a flowchart showing an example of the learning process according to this embodiment.

Step S101: First, the dimensionality reduction unit 211 of the learning unit 201 inputs a multidimensional dataset with correct answers stored in the multidimensional data storage unit 203 with correct answers.

Here, a specific example of a multidimensional dataset with correct answers will be described with reference to Figure 4. Figure 4 is a diagram showing an example of a multidimensional dataset with correct answers in a table format.

The example shown in Figure 4 is a tabular representation of a collection of multidimensional data with non-estimation target attributes such as "year/month," "gender," and "age group," and "number of contracts" as an estimation target attribute.

"Year and month" can take values indicating the year and month, such as "April 2019", "May 2019", etc. "Gender" can take values of either "Male" or "Female". "Age" can take values indicating the age, such as "Teens", "Twenties", "Thirties", etc. "Number of contracts" takes the number of contracts for the given year and month, gender, and age.

For example, in the multidimensional data in the first row of the multidimensional dataset with correct answers shown in Figure 4, (year/month, gender, age group, ..., number of contracts) = (2019/4, male, 30s, ..., 200). Similarly, in the multidimensional data in the second row, (year/month, gender, age group, ..., number of contracts) = (2019/5, female, 20s, ..., 100).

Note that the multidimensional dataset with correct answers shown in FIG. 4 is an example, and this embodiment is applicable to a set of multidimensional data having various attributes. In addition, the values that each attribute can take can be defined in various ways. For example, in the example shown in FIG. 4, "year and month" can take a value for every month of a certain year, but it may take a value for every other month of a certain year, or may take a value for every month or every other month of every other year. Similarly, "age" is not limited to taking values divided into 10-year age groups, and may take values such as "young people" or "elderly people", may take values in a range such as "20 to 25 years old", or may take a value of a specific age such as "20 years old". In addition, "number of contracts" may be the number of customers who have made a contract, or may be the total number of contracts, etc.

In the following, each multidimensional data included in the multidimensional dataset with correct answers has various non-estimation target attributes other than "year/month," "gender," and "age," and has a total of N non-estimation target attributes. In other words, each multidimensional data included in the multidimensional dataset with correct answers has N non-estimation target attributes and one estimation target attribute.

Step S102: Next, the dimension reduction unit 211 of the learning unit 201 inputs the multidimensional dataset with correct answers and the learning dimension reduction model stored in the learning dimension reduction model storage unit 204 to reduce the number of dimensions of the non-estimation target attributes of each multidimensional data included in the multidimensional dataset with correct answers. This allows the learning dimension reduction model to be trained, and a trained dimension reduction model is obtained. In the following, as an example, it is assumed that the number of dimensions (N dimensions) of the non-estimation target attributes is reduced to three dimensions, the first principal component, the second principal component, and the third principal component, by principal component analysis. Note that since principal component analysis is a known method, detailed explanations thereof will be omitted.

Then, the dimension reduction unit 211 of the learning unit 201 outputs the dimension-reduced multidimensional dataset with the correct answer to the binning unit 212, and outputs the learned dimension reduction model to the learned dimension reduction model storage unit 206. Note that the learned dimension reduction model is, for example, information representing a mapping that converts N-dimensional data having N non-estimation target attributes into three-dimensional data having the first principal component, the second principal component, and the third principal component as attributes, and its parameters, etc.

Here, a specific example of a dimension-reduced multidimensional dataset with ground truth will be described with reference to Figure 5. Figure 5 is a diagram showing an example of a dimension-reduced multidimensional dataset with ground truth in a table format.

The example shown in Figure 5 is a tabular representation of a collection of multidimensional data in which the non-estimation target attributes of each piece of multidimensional data included in the multidimensional dataset with correct answers shown in Figure 4 have been dimension-reduced to three dimensions. Each piece of multidimensional data included in the dimension-reduced multidimensional dataset with correct answers shown in Figure 5 has the "first principal component," "second principal component," and "third principal component" as non-estimation target attributes, and "number of contracts" as an estimation target attribute.

The "first principal component", "second principal component" and "third principal component" take the values of the first, second and third principal components, respectively, in principal component analysis. Note that these first to third principal components may be non-estimated target attributes of each multidimensional data included in the multidimensional dataset with correct answers shown in Figure 4, or may be newly defined attributes.

For example, the multidimensional data in the first row of the dimension-reduced multidimensional dataset with correct answers shown in Figure 5 is obtained by reducing the dimensions of the multidimensional data in the first row of the multidimensional dataset with correct answers shown in Figure 4, and is (first principal component, second principal component, third principal component, number of contracts) = (53, 28, 103, 200). Similarly, the multidimensional data in the second row is obtained by reducing the dimensions of the multidimensional data in the second row of the multidimensional dataset with correct answers shown in Figure 4, and is (first principal component, second principal component, third principal component, number of contracts) = (24, 80, 9, 100).

Note that the dimension-reduced multidimensional dataset with correct answers shown in Figure 5 is just one example, and for example, the number of dimensions after dimension reduction can be set to any value (however, an integer value of 2 or more and smaller than N) as appropriate. In general, the smaller the number of dimensions after dimension reduction, the more the amount of calculation can be reduced, but the estimation accuracy during inference decreases. For this reason, the number of dimensions after dimension reduction is appropriately determined taking into consideration the type of target task and the calculation time required for that task, etc.

Step S103: Next, the binning unit 212 of the learning unit 201 receives the dimension-reduced multidimensional dataset with correct answers as input and performs binning on the non-target attributes of each multidimensional data included in the dimension-reduced multidimensional dataset with correct answers. Then, the binning unit 212 of the learning unit 201 outputs the binned multidimensional dataset with correct answers to the information addition unit 213.

Here, a specific example of a binned multidimensional dataset with correct answers will be described with reference to Figure 6. Figure 6 is a diagram showing an example of a binned multidimensional dataset with correct answers in a table format.

The example shown in Figure 6 is a tabular representation of a collection of multidimensional data in which binning has been performed on the non-estimation target attributes of each piece of multidimensional data included in the dimensionality-reduced multidimensional dataset with correct answers shown in Figure 5. Each piece of multidimensional data included in the binned multidimensional dataset with correct answers shown in Figure 6 has the "first principal component," "second principal component," and "third principal component" as non-estimation target attributes and "number of contracts" as an estimation target attribute, and the values of each non-estimation target attribute have been binned.

For example, the multidimensional data in the first row of the binned multidimensional dataset with correct answers shown in Figure 6 is obtained by binning the values of the non-estimated target attributes of the multidimensional data in the first row of the dimension-reduced multidimensional dataset with correct answers shown in Figure 5, and is (first principal component, second principal component, third principal component, number of contracts) = (5, 3, 10, 200). Similarly, the multidimensional data in the second row is obtained by binning the values of the non-estimated target attributes of the multidimensional data in the second row of the dimension-reduced multidimensional dataset with correct answers shown in Figure 5, and is (first principal component, second principal component, third principal component, number of contracts) = (2, 8, 1, 100).

Note that the binned multidimensional dataset with correct answers shown in Figure 6 is just one example, and the bin interval width, etc., used when binning can be set to any appropriate value.

Step S104: Next, the information addition unit 213 of the learning unit 201 uses the binned multidimensional dataset with correct answers as input and creates a learning multidimensional dataset in which additional information is added to each piece of multidimensional data included in the binned multidimensional dataset with correct answers. Then, the information addition unit 213 of the learning unit 201 outputs the learning multidimensional dataset to the learning processing unit 214.

As described above, the additional information is information consisting of the values of the attributes to be estimated when the values of the non-estimation target attributes other than the non-estimation target attribute are kept fixed and the values of the non-estimation target attributes other than the non-estimation target attribute are changed (within the range of values that the non-estimation target attribute can take) for each non-estimation target attribute of each piece of multidimensional data included in the binned multidimensional dataset with correct answers. Such values of the attributes to be estimated are obtained by searching the binned multidimensional dataset with correct answers for the values of the attributes to be estimated when the values of the non-estimation target attributes other than the non-estimation target attribute are kept fixed and the values of the non-estimation target attributes are changed, and then aggregating these values of the attributes to be estimated.

Here, a specific example of a multidimensional learning dataset will be described with reference to Figure 7. Figure 7 shows an example of a multidimensional learning dataset in a table format.

The example shown in Figure 7 is a table showing a collection of multidimensional data with additional information added to each piece of multidimensional data included in the binned, answer-included multidimensional dataset shown in Figure 6. Additional information consisting of "fixed except for the first principal component", "fixed except for the second principal component", and "fixed except for the third principal component" has been added to each piece of multidimensional data included in the learning multidimensional dataset shown in Figure 7.

"Fixed except for the first principal component" is the aggregation of the values of non-estimated target attributes (number of contracts) when the value of the first principal component is changed while the values of the second and third principal components are kept fixed. In the example shown in Figure 7, the number of contracts is aggregated when the value of the first principal component is changed to "0", "1", etc.

"Fixed except for the second principal component" is the total of the values of non-estimated target attributes (number of contracts) when the value of the second principal component is changed while the values of the first and third principal components are kept fixed. In the example shown in Figure 7, the total of the number of contracts is calculated when the value of the second principal component is changed to "0", "1", etc.

"Fixed except for the third principal component" is the total of the values of non-estimated target attributes (number of contracts) when the value of the third principal component is changed while the values of the first and second principal components are kept fixed. In the example shown in Figure 7, the total of the number of contracts is calculated when the value of the third principal component is changed to "0", "1", etc.

For example, the "0" in "Fixed except for the first principal component" of the multidimensional data in the first row of the multidimensional data set for learning shown in Figure 7 is the total number of contracts when the value of the first principal component of the multidimensional data set in the first row of the multidimensional data set with binned correct answers shown in Figure 6 is changed to "0". In other words, the "0" in "Fixed except for the first principal component" of the multidimensional data set in the first row of the multidimensional data set for learning shown in Figure 7 is the total value "400" of the number of contracts where (first principal component, second principal component, third principal component) = (0, 3, 10) in the multidimensional data set with binned correct answers shown in Figure 6.

Similarly, the "1" in "Fixed except for the first principal component" of the multidimensional data in the first row of the learning multidimensional dataset shown in Figure 7 is the total number of contracts when the value of the first principal component of the multidimensional dataset in the first row of the binned correct-answer multidimensional dataset shown in Figure 6 is changed to "1". In other words, the "1" in "Fixed except for the first principal component" of the multidimensional data in the first row of the learning multidimensional dataset shown in Figure 7 is the total value "700" of the number of contracts where (first principal component, second principal component, third principal component) = (1, 3, 10) in the binned correct-answer multidimensional dataset shown in Figure 6.

Similarly, the "0" in "Fixed except for the second principal component" of the multidimensional data in the first row of the multidimensional data set for learning shown in Figure 7 is the total number of contracts when the value of the second principal component of the multidimensional data set in the first row of the multidimensional data set with binned correct answers shown in Figure 6 is changed to "0". In other words, the "0" in "Fixed except for the second principal component" of the multidimensional data in the first row of the multidimensional data set for learning shown in Figure 7 is the total value "400" of the number of contracts where (first principal component, second principal component, third principal component) = (5, 0, 10) in the multidimensional data set with binned correct answers shown in Figure 6.

Similarly, the "1" in "Fixed except for the second principal component" of the multidimensional data in the first row of the learning multidimensional dataset shown in Figure 7 is the total number of contracts when the value of the second principal component of the multidimensional dataset in the first row of the binned correct answer multidimensional dataset shown in Figure 6 is changed to "1". In other words, the "1" in "Fixed except for the second principal component" of the multidimensional data in the first row of the learning multidimensional dataset shown in Figure 7 is the total value "500" of the number of contracts where (first principal component, second principal component, third principal component) = (5, 1, 10) in the binned correct answer multidimensional dataset shown in Figure 6.

The same applies to other attributes of the additional information. Hereinafter, the set of values of the "first principal component," "second principal component," and "third principal component" of each multidimensional data included in the multidimensional training dataset is also referred to as the non-estimation target attribute value set W, and the set of additional information is represented by X.

Note that the multidimensional training dataset shown in Figure 7 is just one example, and for example, techniques such as data augmentation may be used to create multidimensional data from the multidimensional data contained in the multidimensional training dataset in which some of the additional information is unknown (for example, by setting it to "-" or a null value, etc.).

Step S105: Next, the learning processing unit 214 of the learning unit 201 learns the learning inference model using the learning multidimensional dataset and the learning inference model stored in the learning inference model storage unit 205 as inputs, and creates a learned inference model. That is, the learning processing unit 214 of the learning unit 201 learns the parameters of the learning inference model using a known backpropagation method or the like, using a vector w∈W representing a set of non-estimated attribute values and additional information x∈X corresponding to this w, so that the estimation target attribute value y is correctly estimated (i.e., so as to minimize the error between the estimation target attribute value y estimated by the learning inference model and its correct answer). Note that x corresponding to w refers to x in the same row of the set of non-estimated attribute values represented by w in the learning multidimensional dataset.

Here, the inference model is not particularly limited, but for example, a neural network as shown in Figure 8 can be used as the inference model. The neural network shown in Figure 8 is a model that receives as input a vector w representing a set of attribute values not to be estimated and its additional information x, and outputs an attribute value y to be estimated.

For example, a neural network as shown in FIG. 9 may be used as the inference model. The neural network shown in FIG. 9 is a model that receives a vector w representing a set of non-estimated attribute values and its additional information x, and outputs an estimation target attribute value y and a vector w' representing a set of non-estimated attribute values. When using this model, the parameters of the learning inference model are learned so that in addition to the estimation target attribute value y, w' correctly reproduces w. That is, the parameters of the learning inference model are learned so that in addition to the estimation target attribute value y, w' reproducing the original w is estimated in a multitask. By using a neural network as shown in FIG. 9 as the inference model, it is possible to appropriately extract features from the additional information x, so that the features can be fully reflected in the estimation target attribute value y, and it is expected that higher estimation accuracy can be obtained.

Step S106: Finally, the learning processing unit 214 of the learning unit 201 outputs the learned inference model to the learned inference model memory unit 207.

<Inference processing>
Next, the inference process according to this embodiment will be described with reference to Fig. 10. Fig. 10 is a flowchart showing an example of the inference process according to this embodiment.

Step S201: First, the dimensionality reduction unit 221 of the inference unit 202 inputs the uncorrected multidimensional data stored in the uncorrected multidimensional data storage unit 208.

Here, a specific example of a multidimensional dataset without a correct answer will be described with reference to Figure 11. Figure 11 is a diagram showing an example of a multidimensional dataset without a correct answer in a table format.

The example shown in FIG. 11 is a table showing a set of multidimensional data having non-estimation target attributes such as "year/month," "gender," and "age," and "number of contracts" as an estimation target attribute. In the multidimensional data in the first row of the multidimensional dataset without correct answers shown in FIG. 11, (year/month, gender, age, ..., number of contracts) = (2020/4, male, 30s, ..., 200). On the other hand, in the multidimensional data in the second row, (year/month, gender, age, ..., number of contracts) = (2020/5, female, 20s, ..., -). In other words, in the multidimensional data in the second row, the number of contracts, which is the estimation target attribute value, is unknown. In this way, the multidimensional dataset without correct answers includes at least multidimensional data whose estimation target attribute values are unknown.

Step S202: Next, the dimensionality reduction unit 221 of the inference unit 202 receives the superordinate multidimensional dataset and the learned dimensionality reduction model stored in the learned dimensionality reduction model storage unit 206 as input, and reduces the number of dimensions of non-estimated target attributes of each multidimensional data included in the superordinate multidimensional dataset using the learned dimensionality reduction model. Then, the dimensionality reduction unit 221 of the inference unit 202 outputs the dimension-reduced superordinate multidimensional dataset to the binning unit 212.

Here, a specific example of a dimension-reduced, ground-truthless multidimensional dataset will be described with reference to Figure 12. Figure 12 is a diagram showing an example of a dimension-reduced, ground-truthless multidimensional dataset in table form.

The example shown in Figure 12 is a tabular representation of a collection of multidimensional data in which the non-estimation target attributes of each piece of multidimensional data included in the multidimensional dataset without a correct answer shown in Figure 11 have been dimension-reduced to three dimensions. Each piece of multidimensional data included in the dimension-reduced multidimensional dataset without a correct answer shown in Figure 12 has the "first principal component," "second principal component," and "third principal component" as non-estimation target attributes, and "number of contracts" as an estimation target attribute.

For example, the multidimensional data in the first row of the dimension-reduced multidimensional dataset without correct answers shown in Figure 12 is obtained by reducing the dimensions of the multidimensional data in the first row of the multidimensional dataset without correct answers shown in Figure 11, and is (first principal component, second principal component, third principal component, number of contracts) = (58, 21, 109, 200). Similarly, the multidimensional data in the second row is obtained by reducing the dimensions of the multidimensional data in the second row of the multidimensional dataset without correct answers shown in Figure 11, and is (first principal component, second principal component, third principal component, number of contracts) = (20, 81, 6, -).

Step S203: Next, the binning unit 222 of the inference unit 202 receives the dimension-reduced multidimensional dataset without a correct answer as input and performs binning on the non-estimated target attributes of each multidimensional data included in the dimension-reduced multidimensional dataset without a correct answer. Then, the binning unit 222 of the inference unit 202 outputs the binned multidimensional dataset without a correct answer to the information addition unit 223.

Here, a specific example of a binned, no-answer multidimensional dataset will be described with reference to Figure 13. Figure 13 is a diagram showing an example of a binned, no-answer multidimensional dataset in table format.

The example shown in Figure 13 is a tabular representation of a collection of multidimensional data in which binning has been performed on the non-estimation target attributes of each piece of multidimensional data included in the dimensionality-reduced, ground-free multidimensional dataset shown in Figure 12. Each piece of multidimensional data included in the binned, ground-free multidimensional dataset shown in Figure 13 has the "first principal component," "second principal component," and "third principal component" as non-estimation target attributes and "number of contracts" as an estimation target attribute, and the values of each non-estimation target attribute have been binned.

For example, the multidimensional data in the first row of the dimension-reduced multidimensional dataset without correct answers shown in FIG. 13 is obtained by binning the values of the non-estimated target attributes of the multidimensional data in the first row of the dimension-reduced multidimensional dataset without correct answers shown in FIG. 12, and is (first principal component, second principal component, third principal component, number of contracts) = (5, 3, 10, 200). Similarly, the multidimensional data in the second row is obtained by binning the values of the non-estimated target attributes of the multidimensional data in the second row of the dimension-reduced multidimensional dataset without correct answers shown in FIG. 12, and is (first principal component, second principal component, third principal component, number of contracts) = (2, 8, 1, -).

Step S204: Next, the information addition unit 223 of the inference unit 202 uses the binned multidimensional dataset without a correct answer as input and creates a multidimensional dataset for inference in which additional information is added to each piece of multidimensional data included in the binned multidimensional dataset without a correct answer. Then, the information addition unit 223 of the inference unit 202 outputs the multidimensional dataset for inference to the estimation processing unit 224.

Here, a specific example of a multidimensional data set for inference will be described with reference to Figure 14. Figure 14 is a diagram showing an example of a multidimensional data set for inference in a table format.

The example shown in Figure 14 is a table showing a collection of multidimensional data with additional information added to each piece of multidimensional data included in the binned, answer-free multidimensional dataset shown in Figure 13. Additional information consisting of "fixed except for the first principal component", "fixed except for the second principal component", and "fixed except for the third principal component" has been added to each piece of multidimensional data included in the multidimensional dataset for inference shown in Figure 14.

For example, the "0" in "Fixed except for the first principal component" of the multidimensional data in the first row of the inference multidimensional dataset shown in Figure 14 is the total number of contracts when the value of the first principal component of the multidimensional dataset in the first row of the binned multidimensional dataset without correct answers shown in Figure 13 is changed to "0". In other words, the "0" in "Fixed except for the first principal component" of the multidimensional data in the first row of the inference multidimensional dataset shown in Figure 14 is the total value "400" of the number of contracts where (first principal component, second principal component, third principal component) = (0, 3, 10) in the binned multidimensional dataset without correct answers shown in Figure 13.

Similarly, the "0" in "Fixed except for the first principal component" of the multidimensional data in the second row of the inference multidimensional dataset shown in FIG. 14 is the total number of contracts when the value of the first principal component of the multidimensional dataset in the second row of the binned multidimensional dataset without correct answers shown in FIG. 13 is changed to "0". In other words, the "0" in "Fixed except for the first principal component" of the multidimensional data in the second row of the inference multidimensional dataset shown in FIG. 14 is the total value "500" of the number of contracts where (first principal component, second principal component, third principal component) = (0, 8, 1) in the binned multidimensional dataset without correct answers shown in FIG. 13.

Here, if the value of a non-estimation target attribute other than a certain non-estimation target attribute is changed to a certain value while keeping the value of the non-estimation target attribute fixed, and the value of the attribute to be estimated does not exist, a "-" or a null value, etc. is set in the additional information. For example, "1" in "Fixed except for the first principal component" of the multidimensional data in the first row of the inference multidimensional data set shown in FIG. 14 is set to "-". This is because the binned, correct answer-free multidimensional data set shown in FIG. 13 does not have the number of contracts when (first principal component, second principal component, third principal component) = (1, 3, 10).

The same applies to other attributes of the additional information. In the following, as in the learning process, the set of values of the "first principal component," "second principal component," and "third principal component" of each multidimensional data included in the multidimensional dataset for inference is also referred to as the non-estimation target attribute value set W, and the set of additional information is represented by X.

Step S205: Next, the estimation processing unit 224 of the inference unit 202 receives the multidimensional dataset for inference and the learned inference model stored in the learned inference model storage unit 207 as input, and estimates the attribute value to be estimated using the learned inference model. That is, the estimation processing unit 224 of the inference unit 202 receives the vector w∈W representing a set of attribute values not to be estimated and the additional information x∈X corresponding to this w as input, and estimates the attribute value y to be estimated using the learned inference model.

Step S206: Finally, the estimation processing unit 224 of the inference unit 202 outputs the estimation result of the learned inference model (i.e., the attribute value y to be estimated) to the estimation result memory unit 209.

<Summary>
As described above, the inference device 10 according to the present embodiment creates additional information for each non-estimation target attribute, which is composed of the values of the attributes to be estimated when the values of the non-estimation target attributes other than the non-estimation target attribute are changed while keeping the values of the non-estimation target attributes fixed, and then performs learning and inference using this additional information as well. This makes it possible to extract global features, thereby enabling more accurate inference.

In addition, the inference device 10 according to this embodiment reduces the number of dimensions of the non-estimation target attributes and reduces the number of attribute values of each non-estimation target attribute by binning before creating additional information. This makes it possible to reduce the amount of calculations even when the number of non-estimation target attributes and the number of their attribute values are large. For example, if the number of non-estimation target attributes before dimension reduction is N and the average of the attribute values is M, calculations must be performed for N x M attribute values if dimension reduction and binning are not performed. On the other hand, if the number of non-estimation target attributes after dimension reduction is n < N and the average of the attribute values after binning is m < M, in this embodiment, calculations need only be performed for n x m attribute values, making it possible to perform learning and inference with a smaller amount of calculations.

The present invention is not limited to the specifically disclosed embodiments above, and various modifications, variations, combinations with known technologies, etc. are possible without departing from the scope of the claims.

10 Inference device 101 Input device 102 Display device 103 External I/F
103a Recording medium 104 Communication I/F
105 Processor 106 Memory device 107 Bus 201 Learning unit 202 Inference unit 203 Multidimensional data with correct answer storage unit 204 Learning dimension reduction model storage unit 205 Learning inference model storage unit 206 Learned dimension reduction model storage unit 207 Learned inference model storage unit 208 Multidimensional data without correct answer storage unit 209 Estimation result storage unit 211 Dimension reduction unit 212 Binning unit 213 Information addition unit 214 Learning processing unit 221 Dimension reduction unit 222 Binning unit 223 Information addition unit 224 Estimation processing unit

Claims (6)

an input step of inputting multidimensional data having an estimation target attribute indicating an attribute of an estimation target and two or more non-estimation target attributes indicating attributes other than the estimation target attribute;
A dimension reduction procedure for reducing the number of dimensions of non-estimated target attributes of the multidimensional data;
a binning step of binning values of the non-estimated target attributes of the multidimensional data after the dimensionality reduction;
an information addition step of adding predetermined additional information to the multidimensional data after the binning;
a learning procedure for learning parameters of an inference model for estimating a value of the attribute to be estimated, using the multidimensional data to which the additional information is added;
The computer executes
The information addition step includes:
a total value of the value of the attribute to be estimated when a value of a non-estimation target attribute other than the non-estimation target attribute is changed is added as the additional information for each of the non-estimation target attributes of the multidimensional data after the binning . The learning procedure includes:
The learning method according to claim 1, further comprising: learning parameters of the inference model so as to minimize an error between the value of the attribute to be estimated by the inference model using the value of the attribute to be non-estimated as input and a correct answer for the value of the attribute to be estimated; and an error between the value of the attribute to be non-estimated reproduced by the inference model and the value of the attribute to be non- estimated input to the inference model. an input step of inputting multidimensional data having an estimation target attribute indicating an attribute of an estimation target and two or more non-estimation target attributes indicating attributes other than the estimation target attribute;
A dimension reduction procedure for reducing the number of dimensions of non-estimated target attributes of the multidimensional data;
a binning step of binning values of the non-estimated target attributes of the multidimensional data after the dimensionality reduction;
an information addition step of adding predetermined additional information to the multidimensional data after the binning;
an estimation step of estimating a value of the attribute to be estimated by a previously trained inference model using the multidimensional data to which the additional information is added;
The computer executes
The information addition step includes:
an inference method comprising: adding, as the additional information, an aggregate value of the value of the attribute to be estimated when a value of a non-estimation target attribute other than the non-estimation target attribute is changed for each of the non-estimation target attributes of the multidimensional data after the binning . an input unit for inputting multidimensional data having an estimation target attribute indicating an attribute of an estimation target and two or more non-estimation target attributes indicating attributes other than the estimation target attribute;
A dimension reduction unit that reduces the number of dimensions of a non-estimation target attribute of the multidimensional data;
a binning unit that bins values of the non-estimation target attributes of the multidimensional data after the dimension reduction;
an information adding unit that adds predetermined additional information to the multidimensional data after the binning;
a learning unit that learns parameters of an inference model for estimating a value of the attribute to be estimated, using the multidimensional data to which the additional information is added;
having
The information addition unit is
a learning device that adds, for each of the non-estimation target attributes of the multidimensional data after the binning, a total value of the values of the estimation target attributes when values of non-estimation target attributes other than the non-estimation target attributes are changed as the additional information . an input unit for inputting multidimensional data having an estimation target attribute indicating an attribute of an estimation target and two or more non-estimation target attributes indicating attributes other than the estimation target attribute;
A dimension reduction unit that reduces the number of dimensions of a non-estimation target attribute of the multidimensional data;
a binning unit that bins values of the non-estimation target attributes of the multidimensional data after the dimension reduction;
an information adding unit that adds predetermined additional information to the multidimensional data after the binning;
an estimation unit that estimates a value of the estimation target attribute by a pre-trained inference model using the multidimensional data to which the additional information is added;
having
The information addition unit is
an inference device that adds, for each non-estimation target attribute of the multidimensional data after the binning, an aggregate value of the value of the attribute to be estimated when the value of a non-estimation target attribute other than the non-estimation target attribute is changed as the additional information . A program for causing a computer to execute the learning method according to claim 1 or 2 , or the inference method according to claim 3 .

Images