JP7743915B2 - Model analysis device, model analysis method, and program - Google Patents

Description

This disclosure relates to the analysis of machine learning models.

In recent years, predictive models obtained through machine learning have been used in a variety of fields. Patent Document 1 describes a method for visualizing the relationship between the number of training data items used to train a discriminative model and the discrimination accuracy of the model obtained through training, for each category of training data used to train the model.

International Publication No. WO2021/085188

When making predictions using a predictive model, differences in model performance can occur depending on the categories and attributes of the input dataset. In this case, if the dataset used for prediction has a large number of categories or attributes, it becomes difficult to manually check the differences in model performance.

One objective of this disclosure is to evaluate the performance of models for each attribute of a category of a dataset used for prediction, and to visualize categories and attributes with large differences in predictive performance between models.

In one aspect of the present disclosure, a model analysis apparatus includes:
a model acquisition means for acquiring a model;
a performance calculation means for calculating the performance of the model for each attribute included in a data category;
an attribute integrating means for integrating a plurality of consecutive attributes into one based on the performance of the model;
an output means for extracting pairs of attributes whose difference in performance of the model satisfies a predetermined condition, and outputting performance information indicating the performance of the model for each attribute included in the pair;
Equipped with.

In another aspect of the present disclosure, a computer-implemented model analysis method includes:
Get the model,
Calculating the performance of the model for each attribute included in the category of data;
merging consecutive attributes into one based on the performance of the model;
Pairs of attributes whose difference in performance of the models satisfies a predetermined condition are extracted, and performance information indicating the performance of the model for each attribute included in the pair is output.

In yet another aspect of the disclosure , a program includes:
Get the model,
Calculating the performance of the model for each attribute included in the category of data;
merging consecutive attributes into one based on the performance of the model;
The computer is caused to execute a process of extracting pairs of attributes whose difference in performance of the models satisfies a predetermined condition, and outputting performance information indicating the performance of the models for each attribute included in the pair.

This disclosure makes it possible to evaluate the performance of a model for each attribute of a category of the dataset used for prediction, and to visualize categories and attributes with large differences in predictive performance between models.

1 is a block diagram showing the overall configuration of a model generation system according to a first embodiment. FIG. 2 is a block diagram showing a hardware configuration of the model generating device. FIG. 2 is a block diagram showing the functional configuration of the model generation device according to the first embodiment. 10 shows an example of displaying unfair performance information. An example of generating unfair performance information will be described. 10 is a flowchart of a model analysis process performed by the model generating device. FIG. 1 is a block diagram showing a schematic configuration of a model generation system using a server and a terminal device. FIG. 10 is a block diagram showing the functional configuration of a model analysis apparatus according to a second embodiment. 10 is a flowchart of a process performed by a model analysis device according to a second embodiment.

Hereinafter, preferred embodiments of the present disclosure will be described with reference to the drawings.
First Embodiment
[Overall configuration]
1 is a block diagram showing the overall configuration of a model generation system according to a first embodiment. The model generation system 1 includes a model generation device 100, a display device 2, and an input device 3. The model generation device 100 is an application of the model analysis device of the present disclosure, and is configured by a computer such as a personal computer (PC). The display device 2 is, for example, a liquid crystal display device, and displays evaluation information generated by the model generation device 100. The input device 3 is, for example, a mouse, a keyboard, etc., and is used by the user to give instructions and input necessary when modifying a model or viewing evaluation information.

First, we will provide an overview of the operation of the model generation system 1. The model generation device 100 generates a machine learning model (hereinafter simply referred to as a "model") using previously prepared training data. The model generation device 100 also analyzes and evaluates the generated model. Specifically, the model generation device 100 performs model predictions using evaluation data and analyzes the model's predictive performance based on the prediction results. The model generation device 100 calculates a value indicating the model's performance (hereinafter also referred to as a "performance value") for each attribute of a category in the evaluation data dataset. If the model performance for each attribute of a category differs significantly, the model generation device 100 generates a graph or the like indicating the model's performance for each attribute of the category and presents it to the user as evaluation information. The user can confirm the evaluation information and operate the input device 3 to input correction information for modifying the model.

A machine learning model is information that represents the relationship between explanatory variables and a response variable. A machine learning model is, for example, a component that estimates the results of an estimation target by calculating a response variable based on explanatory variables. A machine learning model is generated by executing a learning algorithm using training data, for which the response variable value has already been obtained, and arbitrary parameters as input. A machine learning model may be represented, for example, by a function c that maps an input x to a correct answer y. A machine learning model may estimate the numerical value of an estimation target, or may estimate the label of an estimation target. A machine learning model may output a variable that describes the probability distribution of the response variable. A machine learning model may also be referred to as a "learning model," "analysis model," "AI (Artificial Intelligence) model," "trained model," "inference model," or "prediction formula." Here, explanatory variables are variables used as inputs in a machine learning model. Explanatory variables may also be referred to as "features" or "characteristics."

The learning algorithm for generating the machine learning model is not particularly limited and may be any existing learning algorithm. For example, the learning algorithm may be a piecewise linear model using random forest, support vector machine, naive Bayes, or FAB inference (Factorized Asymptotic Bayesian Inference), or a neural network. Note that a piecewise linear model technique using FAB inference is disclosed, for example, in U.S. Patent Publication US 2014/0222741 A1.

[Hardware configuration]
2 is a block diagram showing the hardware configuration of the model generation device 100. As shown in the figure, the model generation device 100 includes an interface (I/F) 111, a processor 112, a memory 113, a recording medium 114, and a database (DB) 115.

I/F 111 inputs and outputs data to and from external devices. Specifically, training data and evaluation data used to generate the model, as well as instructions and input entered by the user using input device 3, are input to model generation device 100 via I/F 111. In addition, evaluation information for the model generated by model generation device 100 is output to display device 2 via I/F 111.

Processor 112 is a computer such as a CPU (Central Processing Unit) that controls the entire model generation device 100 by executing a pre-prepared program. Processor 112 may also be a GPU (Graphics Processing Unit), TPU (Tensor Processing Unit), quantum processor, or FPGA (Field-Programmable Gate Array). Processor 112 executes the model analysis process described below.

Memory 113 is composed of ROM (Read Only Memory), RAM (Random Access Memory), etc. Memory 113 is also used as working memory while processor 112 is executing various processes.

The recording medium 114 is a non-volatile, non-transitory recording medium such as a disk-shaped recording medium or semiconductor memory, and is configured to be detachable from the model generation device 100. The recording medium 114 records various programs executed by the processor 112. When the model generation device 100 executes various processes, the programs recorded on the recording medium 114 are loaded into the memory 113 and executed by the processor 112.

DB115 stores information about the model generated by model generation device 100 (hereinafter referred to as the "existing model") and the model after correction through retraining (hereinafter referred to as the "corrected model"). DB115 also stores, as necessary, training data, evaluation data, correction information entered by the user, etc., input via I/F111.

(Functional configuration)
3 is a block diagram showing the functional configuration of the model generation device 100 according to the first embodiment. Functionally, the model generation device 100 includes a training data DB 121, a model training unit 122, a model DB 123, an evaluation data DB 124, and an analysis unit 125.

The training data DB 121 stores training data used to generate a model. The training data D1 is input to the model training unit 122. The training data D1 is composed of multiple combinations of input data and correct answers (correct values or correct labels) for the input data.

The model training unit 122 trains a model using the training data D1 and generates a model. The model training unit 122 outputs model data M corresponding to the generated model to the model DB 123 and the analysis unit 125. The model data M includes multiple parameter information constituting the model. The parameter information includes, for example, information on explanatory variables used as input to the model, information on weights for each explanatory variable, and information on weights for each sample constituting the input data.

The model training unit 122 also retrains the existing model to generate a modified model. In this case, the model training unit 122 modifies the parameters that make up the model based on the modification information D3 input by the user using the input device 3, and retrains the model using training data for retraining as necessary. The model training unit 122 stores the model data M of the modified model obtained by retraining in the model DB 123 and outputs it to the analysis unit 125.

The evaluation data DB 124 stores evaluation data used to evaluate the generated model. The evaluation data may be, for example, a portion of the collected data that was not used as training data, newly collected data, or validation data prepared for verification. Training data may also be used as evaluation data. The evaluation data consists of multiple combinations of input data and correct answers (correct values or correct labels) for the input data.

Evaluation data is stored and used in units of datasets. Each piece of data contained in a dataset contains multiple categories. For example, in the case of a predictive model for determining creditworthiness, data on a large number of individuals to be evaluated is prepared as evaluation data, and each piece of data contains categories such as age group, gender, residential area, family structure, and income. Furthermore, each category contains multiple attributes (groups).

The analysis unit 125 uses the evaluation data to analyze the performance of the model for each attribute included in the categories of the dataset. For example, suppose a certain category, "age group," includes multiple attributes such as "20s," "30s," "40s," etc. In this case, the analysis unit 125 calculates a performance value indicating the model's performance for each attribute. Next, the analysis unit 125 extracts attributes with large performance differences (deviations). Note that large performance differences are referred to as "unfair" performance. The analysis unit 125 outputs information indicating the categories and attributes with large performance differences (hereinafter also referred to as "unfair performance information") to the display device 2 as evaluation information D2.

The display device 2 displays the evaluation information D2 output by the analysis unit 125 on the display device 2. By looking at the displayed unfair performance information, the user can learn about the differences in model performance for each category and attribute.

In addition, the user inputs correction information D3 into the input device 3 as needed to correct the current model. The correction information D3 is information related to the correction, such as information on explanatory variables used as input to the model, information on weights for each explanatory variable, and information on weights for each sample that makes up the input data. The model training unit 122 corrects the model by retraining the model using the input correction information D3.

Figure 4 shows an example of unfair performance information. Figure 4(A) shows the performance index for an entire dataset of a certain predictive model. This graph may be, for example, the average value of the performance index of the predictive model for all data included in the dataset.

Figure 4 (B) shows an example of unfair performance information. In this example, out of the multiple attributes included in the "age group" category (20s, 30s, ...), the prediction model performs well for data for the "20s" attribute, but performs poorly for data for the "60s to 90s" attribute.

Figure 4(C) shows another example of unfair performance information. In this example, it is shown that, among the multiple attributes included in the category "income," the predictive model performs well for data on the attribute "2 million to 3 million," but performs poorly for the attribute "10 million."

In this way, the analysis unit 125 extracts attributes with large differences in model performance for each attribute, and presents the categories and graphs showing performance values to the user as unfair performance information. This allows the user to modify the model so as to reduce the differences in model performance for each attribute for each category, i.e., to prevent variations in model performance for each attribute. The method for generating unfair performance information will be explained in detail later. The analysis unit 125 is an example of a model acquisition means, performance calculation means, attribute integration means, and output means.

[Example of generating unfair performance information]
Next, an example of generating unfair performance information will be described. FIG. 5 shows an example of generating unfair performance information. Now, consider a model that predicts product sales based on customer information. The data set input to the model contains information on categories 1 to 6 for multiple customers. Category 1 is "age group" and category 4 is "income."

In this case, the analysis unit 125 first divides each category into multiple attributes (groups). In the example of Figure 5, the analysis unit 125 divides category 1 (age group) into multiple attributes: "20s," "30s," etc. Similarly, the analysis unit 125 divides categories 2 to 6 into multiple attributes each.

Next, the analysis unit 125 inputs the dataset for each attribute into the model for each category, performs predictions, and calculates a performance index indicating the model's performance for each attribute. Performance indices can be various indices that can be used to evaluate model performance, such as the model's prediction accuracy and compatibility. Other examples of performance indices include the F1 score, precision, and recall. In the case of regression tasks, the coefficient of determination may also be used. Furthermore, error-based indices, where the lower the value, the better, may also be used, such as mean squared error and cross entropy. This allows for the acquisition of performance values indicating the model's performance for each attribute included in categories 1 to 6, as shown in Figure 5. In Figure 5, graphs 81 to 86 show the performance values for each attribute calculated for categories 1 to 6.

Next, the analysis unit 125 extracts combinations for each category that have a large difference in performance value for each attribute. For example, in category 1 in Figure 5, the performance value for the attribute "20s" is high, while the performance values for the attributes "60s," "70s," etc. are low. Therefore, the analysis unit 125 extracts combinations of attributes that have a large difference in performance value (hereinafter also referred to as "unfair attribute pairs"). Specifically, the analysis unit 125 extracts the attribute with the largest performance value and the attribute with the smallest performance value as an unfair attribute pair.

In addition, if the difference in performance values between consecutive or adjacent attributes is small, the analysis unit 125 may combine those attributes into a single attribute. For example, in graph 81 for category 1 in Figure 5, if the difference in performance values between the attributes "60s," "70s," "80s," and "90s" is small (the difference is within a predetermined threshold), the analysis unit 125 may combine them to create an attribute "60s to 90s." Thus, in the example of Figure 5, "20s" and "60s to 90s" are extracted as an unfair attribute pair. In this way, by combining multiple attributes with small differences in performance values, the display becomes simpler than when multiple attributes are listed, making it easier for users to recognize.

Next, the analysis unit 125 compares the difference in performance values for the unfair attribute pairs extracted for each category, and determines the category corresponding to the unfair attribute pair with the largest difference in performance values as the unfair category (hereinafter also referred to as the "unfair category"). For example, if the difference between the unfair attribute pairs in category 1 is the largest among categories 1 to 6, the analysis unit 125 determines category 1 as the unfair category. In this way, the category with the highest unfairness (least fairness) is extracted from among multiple categories.

The analysis unit 125 creates a graph showing the category name and performance values of the unfair attribute pairs for each unfair category, and outputs it to the display device 2. In the example of Figure 5, the analysis unit 125 determines that category 1 is an unfair category, and displays a graph showing the category name "Age Group" and the performance values of the unfair attribute pairs "20s" and "60s to 90s."

In the above example, the analysis unit 125 extracts one unfair category, but it may extract multiple unfair categories. For example, the analysis unit 125 may extract the category with the highest unfairness and the category with the second highest unfairness as unfair categories from among multiple categories. In the example of Figure 5, in addition to category 1 (age group) shown in graph 87, category 4 (income) shown in graph 88 is also extracted as an unfair category. For category 4, the analysis unit 125 combines the attributes "2 million," "2.5 million," and "3 million," which have small differences in performance values, to create the attribute "2 million to 3 million," and extracts the attributes "2 million to 3 million" and "10 million" as an unfair attribute pair. If there is only one category, the analysis unit 125 simply outputs unfair performance information for that category.

As described above, in this embodiment, the analysis unit 125 compares the performance values of each attribute included in each category and extracts categories with large differences in performance values for each attribute as unfair categories. The analysis unit 125 then displays information such as graphs 87 and 88 related to the unfair categories as unfair performance information on the display device 2. This allows the user to easily identify categories with large differences or variances in model performance for each attribute. Typically, when there are a large number of categories and attributes, or when the dataset is numerical data, it is difficult to manually confirm the performance differences for each category. In this regard, in this embodiment, by displaying unfair performance information such as that shown in graphs 87 and 88 in Figure 5, the user can easily identify categories with unfair performance and the problematic differences in performance values for each attribute in those categories.

[Model analysis processing]
Next, the model analysis process performed by the model generating device 100 will be described. Fig. 6 is a flowchart of the model analysis process performed by the model generating device 100. The model analysis process is a process of calculating the performance of a model for each attribute of each category and generating unfair performance information, as exemplified in Fig. 5. This process is realized by the processor 112 shown in Fig. 2 executing a program prepared in advance and operating mainly as the analysis unit 125 shown in Fig. 3.

First, the analysis unit 125 converts continuous variables contained in the dataset into categories (step S10). Specifically, the analysis unit 125 converts continuous variables such as age and income into ordered categories. For example, the analysis unit 125 divides the range between the minimum and maximum values of the continuous variable into two or more attributes (groups) and converts them into categories. In the example of age, the analysis unit 125 divides the age range from 20 to 100 into 10-year increments to create multiple attributes such as 20s, 30s, etc.

If the analysis unit 125 can prepare division points based on business knowledge, it may convert continuous variables into categories according to those division points. The analysis unit 125 may also determine division points according to percentiles so that the number of samples for each attribute is the same, and convert continuous variables into categories. If the variables included in the dataset are originally categorical, such as gender or place of residence, the above processing is not necessary.

Next, the analysis unit 125 calculates a performance value for each attribute for each category (step S11). Specifically, the analysis unit 125 divides the dataset according to the attributes of each category to create partial datasets. Then, the analysis unit 125 calculates the performance value of the model for each partial dataset.

Next, the analysis unit 125 integrates the attributes (step S12). Specifically, for an ordered category containing the original continuous variables, if the difference in performance between consecutive attributes is within a certain threshold, the analysis unit 125 integrates those attributes into a single attribute. Alternatively, the analysis unit 125 may integrate multiple attributes using a clustering technique.

Next, the analysis unit 125 determines, for each category, unfair attribute pairs with large differences in model performance values (step S13). Specifically, the analysis unit 125 extracts, for each category, the pair of attributes with the highest and lowest model performance values as unfair attribute pairs.

The method by which the analysis unit 125 extracts unfair attribute pairs is not limited to the maximum and minimum performance values. For example, the analysis unit 125 may extract pairs that satisfy conditions predetermined by the user as unfair attribute pairs. The analysis unit 125 may extract any attribute pair as an unfair attribute pair if the difference in performance values between the pair is equal to or greater than a certain value.

Next, the analysis unit 125 determines the unfair category and outputs the unfair performance information to the display device 2 (step S14). Specifically, the analysis unit 125 calculates the difference in performance values between the unfair attribute pairs for each category and determines the category with at least the largest difference in performance value as the unfair category. The analysis unit 125 then outputs unfair performance information for the unfair category, including graphs showing the performance values of the unfair attribute pairs belonging to that category, to the display device 2. As a result, unfair performance information such as graphs 87 and 88 in Figure 5 is displayed on the display device 2. The processing then ends.

[Modification]
In the above embodiment, the model generation device 100 is configured as an independent device such as a PC, but instead, the model generation device may be configured by a server and a terminal device. FIG. 7 is a block diagram showing a schematic configuration of a model generation system 1x using a server and a terminal device. In FIG. 7, the server 100x has the configuration of the model generation device 100 shown in FIG. 3. Furthermore, the display device 2x and input device 3x of the terminal device 7 used by the user are used as the display device 2 and input device 3 shown in FIG. 3.

Second Embodiment
8 is a block diagram showing the functional configuration of a model analysis device 70 according to the second embodiment. The model analysis device 70 includes a model acquisition unit 71, a performance calculation unit 72, and an output unit 73.

Figure 9 is a flowchart of processing by the model analysis device 70 of the second embodiment. The model acquisition means 71 acquires a model (step S71). The performance calculation means 72 calculates the performance of the model for each attribute included in the data category (step S72). The output means 73 extracts attribute pairs whose model performance difference satisfies a predetermined condition, and outputs performance information indicating the model performance for each attribute included in the pair (step S73).

According to the model analysis device 70 of the second embodiment, it is possible to evaluate the performance of a model for each attribute of a category of a dataset used for prediction, and to visualize categories and attributes with large differences in predictive performance between models.

Some or all of the above embodiments may also be described as, but are not limited to, the following notes:

(Appendix 1)
a model acquisition means for acquiring a model;
a performance calculation means for calculating the performance of the model for each attribute included in a data category;
an output means for extracting pairs of attributes whose difference in performance of the model satisfies a predetermined condition, and outputting performance information indicating the performance of the model for each attribute included in the pair;
A model analysis device comprising:

(Appendix 2)
the performance calculation means calculates the performance of the model for each attribute included in each of a plurality of categories;
2. The model analysis device according to claim 1, wherein the output means outputs the performance information for a category among the plurality of categories that has the largest difference in performance between the attributes included in the attribute pair.

(Appendix 3)
3. The model analysis apparatus according to claim 1, further comprising an attribute integration means for integrating a plurality of consecutive attributes into one based on the performance of the model.

(Appendix 4)
4. The model analysis device according to claim 1, wherein the output means extracts, as the attribute pair, an attribute for which the model's performance has the maximum value and an attribute for which the model's performance has the minimum value.

(Appendix 5)
5. The model analysis apparatus according to claim 1, wherein the performance information includes a graph showing the performance of the model for each attribute included in the attribute pair.

(Appendix 6)
6. The model analysis device according to claim 1, wherein the performance calculation means divides a dataset for each attribute included in the category, and calculates the performance of the model for each attribute using the dataset for each attribute.

(Appendix 7)
Get the model,
Calculating the performance of the model for each attribute included in the category of data;
A model analysis method that extracts pairs of attributes whose difference in model performance satisfies a predetermined condition, and outputs performance information indicating the performance of the model for each attribute included in the pair.

(Appendix 8)
Get the model,
Calculating the performance of the model for each attribute included in the category of data;
A recording medium having a program recorded thereon that causes a computer to execute a process of extracting pairs of attributes whose difference in performance of the model satisfies a predetermined condition, and outputting performance information indicating the performance of the model for each attribute included in the pair.

The present disclosure has been described above with reference to embodiments and examples, but the present disclosure is not limited to the above embodiments and examples. Various modifications that would be understood by a person skilled in the art can be made to the configuration and details of the present disclosure within the scope of the present disclosure.

1, 1x Model generation system 2, 2x Display device 3, 3x Input device 7 Terminal device 100 Model generation device 112 Processor 121 Training data DB
122 Model Training Department 123 Model DB
124 Evaluation data DB
125 Analysis Department

Claims (7)

a model acquisition means for acquiring a model;
a performance calculation means for calculating the performance of the model for each attribute included in a data category;
an attribute integrating means for integrating a plurality of consecutive attributes into one based on the performance of the model;
an output means for extracting pairs of attributes whose difference in performance of the model satisfies a predetermined condition, and outputting performance information indicating the performance of the model for each attribute included in the pair;
A model analysis device comprising: the performance calculation means calculates the performance of the model for each attribute included in each of a plurality of categories;
The model analysis device according to claim 1 , wherein the output means outputs the performance information for a category among the plurality of categories that has the largest difference in performance between the attributes included in the attribute pair. 3. The model analysis device according to claim 1, wherein the output means extracts, as the attribute pair, an attribute for which the model performance has a maximum value and an attribute for which the model performance has a minimum value. The model analysis device according to claim 1 , wherein the performance information includes a graph indicating the performance of the model for each attribute included in the attribute pair. 5. The model analysis device according to claim 1 , wherein the performance calculation means divides a data set for each attribute included in the category, and calculates the performance of the model for each attribute using the data set for each attribute. 1. A computer-implemented method for model analysis, comprising:
Get the model,
Calculating the performance of the model for each attribute included in the category of data;
merging consecutive attributes into one based on the performance of the model;
A model analysis method that extracts pairs of attributes whose difference in model performance satisfies a predetermined condition, and outputs performance information indicating the performance of the model for each attribute included in the pair. Get the model,
Calculating the performance of the model for each attribute included in the category of data;
merging consecutive attributes into one based on the performance of the model;
A program that causes a computer to execute a process of extracting pairs of attributes in which the difference in performance of the model satisfies a predetermined condition, and outputting performance information indicating the performance of the model for each attribute included in the pair.