JP6803423B2 - Model building system, information processing system and program - Google Patents

Description

The present invention is a model construction system that constructs a model that outputs predicted values of prediction targets related to risks or needs, an information processing system that executes processing using the model constructed by this model construction system, and a model construction system. Regarding the program related to the control of.

There is a need for an entity that makes a contract or transaction with a customer to understand the risks and needs of the customer. Customer risk is, for example, the magnitude of the possibility of financial damage such as bad debts and delinquency when issuing a credit card, and if this can be accurately grasped, whether or not to conclude a contract. And, the degree of credit can be judged appropriately. In addition, customer needs are, for example, the willingness to purchase each product when providing products (for example, insurance products, investment products, etc.), and if this can be accurately grasped, it will be related to each product. It is possible to reduce opportunity loss and increase sales. In the past, when making a contract or transaction with a certain customer, the entity tried to grasp the risk and needs of the customer based on the past contract or transaction performance of the customer. In addition, Patent Document 1 describes a technique for acquiring financial information about a company to be evaluated and calculating and providing a probability that the company to be evaluated will be in a bankruptcy state by using the acquired financial information. There is.

Japanese Unexamined Patent Publication No. 2009-408236

However, as in the past, in the case of the method of grasping the customer's risk and needs based on the customer's past contract and transaction record, only the person who can obtain the past contract and transaction record of the risk and need There is room for improvement in that it is not possible to grasp the risks and needs of persons who can grasp and cannot acquire the results of new customers.

The present invention has been made to solve such a problem, and makes it possible to provide information that can be used to judge risks and needs of an individual who cannot obtain past achievements. The purpose is.

In order to solve the above-mentioned problems, the present invention is a data set generated based on a basic database accumulating personal achievements regarding each individual's risk or needs, and the attribute information of the individual is used as a value for each record. One or more attribute information is entered and entered based on a dataset that has one or more attribute items and one or more performance-related items that have performance-related information based on personal performance as a value. I am trying to build a model that outputs the predicted value of the predicted target regarding the risk or need for an individual corresponding to multiple attribute information. In addition, when constructing the model, the pattern of one or more attribute items corresponding to one or more attribute information input to the model, the prediction target to which the model outputs the predicted value, the data set used for building the model, and the data set. A candidate model is generated by using a part of the data set as training data for each combination with the algorithm used to build the model, and each combination of the pattern of the attribute item and the prediction target is a candidate model belonging to the combination. There was a request to build a model with information about specific attribute item patterns and specific prediction targets by performing tests using a part of the data set as test data for each of the above to identify the most accurate candidate model. If, a portion of the dataset is based on the dataset and algorithm used to build the most accurate candidate model of the candidate models generated for the combination of a particular attribute item pattern and a particular predictor. I try to build a model by using it without using it as test data. Alternatively, when building the model, there was a request to build the model, along with providing a performance database that stores information on specific attribute item patterns and specific prediction targets, as well as personal performance regarding each individual's risks or needs. In the case, for the combination of the pattern of a specific attribute item and a specific prediction target, a part of the data set is used as test data for each combination of the data set used for building the model and the algorithm used for building the model. A tentative model is generated without using it, and each generated tentative model is tested using a performance database to verify the accuracy of the tentative model, and the tentative model with the highest accuracy is used as the model.

According to the present invention constructed as described above, the constructed model does not input the past performance of an individual when outputting the predicted value of the forecast target regarding the risk or needs of the individual, and is novel. The "attribute" that can be acquired even by the customer is input. Therefore, by using the constructed model, the predicted value of the target individual regarding risk or needs, that is, the target individual, can be used by using the attributes of the target individual without acquiring the past performance. Can provide information that can be used to determine the risks and needs of.

It is a block diagram which shows the functional configuration example of the model constituent apparatus of the model construction system which concerns on one Embodiment of this invention. It is a figure which shows an example of a basic database <credit card>. It is a figure which shows an example of a basic database <rent delinquency>. It is a figure which shows an example of a basic database <life insurance>. It is a figure which shows an example of a data set <addition of public information>. It is a figure which shows an example of the database for addition of complementary information. It is a figure which shows an example of a data set <public information + supplementary information addition>. It is a figure which shows an example of a data set <after merging>. It is a figure which shows the relationship between the model construction information, a prediction target, an input attribute item pattern, a data set and an algorithm. It is a figure used for the explanation of a service providing system.

Hereinafter, embodiments of the present invention will be described with reference to the drawings. FIG. 1 is a block diagram showing a functional configuration example of the model building apparatus 2 of the model building system 1 according to the present embodiment. The model building device 2 according to the present embodiment is a device operated by a predetermined company (hereinafter referred to as “management company”), and inputs predetermined information to be described later for the target individual, and the individual is described. It has a function to build a score output model SS (corresponding to the "model" of the claims) that outputs "predicted value of the prediction target regarding risk or needs". The "risks" exemplified in this embodiment include "credit loss risk when a credit card is issued (hereinafter, simply referred to as" creca risk ")" and "rent delinquency risk when renting a property (hereinafter, simply" "Rent delinquency risk") "and" total risk "are included. "Comprehensive risk" is a concept that abstracts the overall magnitude of the possibility of loss due to bad debt or delinquency when credit or lending is made. The higher the total risk, the more contracts and transactions Regardless of the content, it is suggested that there is a high possibility that losses will occur due to acts such as bad debts.

As will become clear later, in the present embodiment, the model building apparatus 2 constructs a different score output model SS for each prediction target. And, "predicted value of the forecast target for creca risk" (hereinafter, the forecast target for creca risk is simply referred to as "creca risk", and the predicted value is referred to as "creca risk score") is creca risk (credit loss occurs) The magnitude of the possibility) is expressed as a score (score). In this way, the credit card risk score is not expressed as a probability of the possibility of bad debt, but as a score. This also applies to other predicted values according to the present embodiment, and based on this, the output of the model constructed by the model building apparatus 2 may be expressed as a predicted score below.

In addition, the "predicted value of the forecast target for rent delinquency risk" (hereinafter, the forecast target for rent delinquency risk is simply referred to as "rent delinquency risk", and the predicted score is referred to as "rent delinquency risk score") is the rent delinquency risk. (The magnitude of the possibility of rent delinquency) is expressed as a score. In addition, the "predicted value of the predicted target for total risk" (hereinafter, the predicted target for total risk is simply referred to as "total risk", and the predicted score is referred to as "total risk score") means the total risk as a score. It is a thing.

In addition, the "needs" exemplified in this embodiment include "needs for purchasing life insurance" (hereinafter, simply referred to as "life insurance needs") and "comprehensive needs". "Comprehensive needs" is a concept that abstracts the overall size of the possibility of purchasing products and services such as life insurance, investment trusts, and card loans. The higher the overall needs, the more products and services. It is suggested that there is a high possibility of purchasing goods and services regardless of the type of.

The "predicted value of the forecast target for life insurance needs" (hereinafter, the forecast target for life insurance needs is simply referred to as "life insurance needs", and the predicted score is referred to as "life insurance needs score") is the life insurance needs (life). The size of the possibility of purchasing insurance) is expressed as a score. In addition, the "predicted value of the predicted target for comprehensive needs" (hereinafter, the predicted target for comprehensive needs is simply referred to as "comprehensive needs", and the predicted score is referred to as "comprehensive needs score") means the comprehensive needs as a score. It is a thing. The “risk or need” exemplified in this embodiment is merely an example, and is not limited to the one illustrated in this embodiment.

As shown in FIG. 1, the model construction device 2 includes a data set construction unit 3 and a model construction unit 4 as functional configurations. Each of the above functional blocks 3 and 4 can be configured by any of hardware, DSP (Digital Signal Processor), and software. For example, when configured by software, each of the above functional blocks 3 and 4 is actually configured to include a computer CPU, RAM, ROM, etc., and is a program stored in a recording medium such as RAM, ROM, hard disk, or semiconductor memory. Is realized by the operation of. Further, the model construction device 2 includes a basic database storage unit 5, a data set storage unit 6, a model construction information storage unit 7, a score output model storage unit 8, and a client company information storage unit 9 as storage means. The data stored in these storage means will be described later.

The data set construction unit 3 constructs a data set DS by processing one or more basic databases by a predetermined method. In particular, the data set construction unit 3 constructs a plurality of data set DSs by changing the combination of the basic database DB to be used or the basic database DB to be used and the processing method, and stores them in the data set storage unit 6. The basic database DB is a database that accumulates individual achievements regarding the risks or needs of each individual collected during the activities of the data provider. The management company is to receive the provision of the basic database DB from a plurality of data providers by contract, and the basic database storage unit 5 is cumulative with a plurality of basic database DBs provided by the plurality of data providers. Is remembered in. For example, a company that issues credit cards becomes a data provider and provides the management company with a basic database DB that it has built while conducting its own activities. Therefore, the basic database storage unit 5 stores various types of basic database DBs provided by data providers in various industries. In addition, as the number of data providers increases, the types and amounts of basic database DBs also increase. Information is provided to the basic database DB stored in the basic database storage unit 5 by a data provider as appropriate, and records are added.

FIG. 2 is a diagram showing a basic database <credit card> DB1 which is an example of the basic database DB. The basic database <Kureka> DB1 is a basic database DB provided by a data provider that issues credit cards, and has a record for each customer for the data provider. As shown in FIG. 2, the record of the basic database <credit card> DB1 has gender (item), age (item), zip code (item), annual income (item), and credit card bad debt record (item) as items. Of each item, gender (item), age (item), zip code (item) and annual income (item) are items indicating the attributes of the customer. Hereinafter, the item value of gender (item) is referred to as "gender value", the item value of age (item) is referred to as "age value", and the item value of postal code (item) is referred to as "zip code value". The item value of (item) is called "annual income value". Further, in the following, in the basic database DB and the data set DS described later, the items indicating the attributes of the customer are collectively referred to as "attribute items", and the item values of the attribute items are collectively referred to as "attribute information".

The credit card bad debt record (item) is an item having information as an item value indicating the presence or absence of credit card bad debt experience in two values, a positive class and a negative class. Hereinafter, the item value of the credit card bad debt record (item) is referred to as the "credit card bad debt record value". In the present embodiment, the actual value of credit card bad debt takes a positive class (expressed as “yes” in FIG. 2) when there is a credit card bad debt experience, and a negative class (“none” in FIG. 2) when there is no experience. (Expression). Hereinafter, in the basic database DB and the data set DS described later, items having information based on individual performance as item values, such as credit card bad debt results (items), are referred to as "performance-related items" to distinguish them from attribute items. , The item value of the performance-related item is called "performance-related information".

FIG. 3 is a diagram showing a basic database <rent delinquency> DB2, which is an example of the basic database DB. The basic database <rent delinquency> DB2 is a basic database DB provided by a data provider engaged in a real estate leasing business or a rent guarantee business, and has a record for each customer for the data provider. As shown in FIG. 3, the record of the basic database <rent delinquency> DB2 has gender (item), age (item), municipality (item), industry (item), and rent delinquency record (item) as items. .. Of each item, gender (item), age (item), city (item) and industry (item) are attribute items. Hereinafter, the item value of the city / ward / town / village (item) is referred to as "city / ward / town / village value". The city / ward / town / village value is a value representing the city / ward / town / village to which the residence belongs. In addition, the item value of the industry (item) is called "industry value". The industry value is a value representing the industry of the occupation. The rent delinquency record (item) is a record-related item that has information indicating whether or not there is experience of rent delinquency as an item value in two values, a positive class and a negative class. Hereinafter, the item value of the credit card bad debt record (item) is referred to as the "credit card bad debt record value". In the present embodiment, the actual rent delinquency value takes a positive class (expressed as "yes" in FIG. 3) when there is experience of rent delinquency, and a negative class (expressed as "none" in FIG. 3) when there is no experience. ).

FIG. 4 is a diagram showing a basic database <life insurance> DB3 which is an example of the basic database DB. Basic database <life insurance> DB3 is a basic database DB provided by an insurance company that has life insurance as one of its products and a sales company of its life insurance products, and has a record for each customer for the data provider. ing. As shown in FIG. 4, the records of the basic database <life insurance> DB3 include gender (item), age (item), municipality (item), annual income (item), marriage classification (item), and life insurance. Has a purchase record (item). Of each item, gender (item), age (item), city (item), annual income (item) and marriage classification (item) are attribute items. The marriage classification (item) is a married / unmarried classification, and hereinafter, the item value of the marriage classification (item) is referred to as "marriage classification value". The life insurance purchase record (item) is a record-related item having information as an item value indicating whether or not the person has purchased life insurance in two values, a positive class and a negative class. Hereinafter, the item value of the life insurance purchase record (item) is referred to as the "life insurance purchase record value". In the present embodiment, the actual life insurance purchase value takes a positive class (expressed as “yes” in FIG. 4) when the person has purchased life insurance, and a negative class (expressed as “yes” in FIG. 4) when he / she has no experience. ”).

The attribute items and performance-related items illustrated in this embodiment are only limited examples. For example, in addition to the attribute items illustrated in this embodiment, there are the prefecture / region code of the place of residence, occupations that are finer than the type of industry, housing categories (rental, loan, owned house, etc.), presence or absence of children, and the like. In addition to the items illustrated in this embodiment, the performance-related items may be any items as long as they can be classified into positive class and negative class based on the performance.

The data set construction unit 3 performs public information addition processing, supplementary information addition processing, and merge processing to construct a data set DS based on the basic database DB. Hereinafter, each process will be described.

<Public information addition processing>
First, the public information addition process will be described by taking as an example the case where the basic database <credit card> DB1 of FIG. 2 is targeted. As described above, the record of the basic database <Credit card> DB1 has a zip code (item) as an item. The data set construction unit 3 executes the following processing for each record. That is, the data set construction unit 3 acquires the postal code value of the postal code (item) of the record to be processed. Next, the data set construction unit 3 converts the acquired postal code value into information indicating the city / ward / town / village to which the area indicated by the postal code value belongs and information indicating the population of the city / ward / town / village. In the present embodiment, the data set construction unit 3 associates the postal code value, the information indicating the city, ward, town, and village with the information indicating the population of the city, ward, town, and village disclosed by the government or other organizations for each postal code. Access the server that has the table and convert the zip code value. The data set construction unit 3 adds cities, wards, towns and villages (items) and population (items) as new items to the records of the basic database <Kureka> DB1, and the city wards converted from these item values from the zip code values. Information indicating the town / village and information indicating the population converted from the zip code value. Hereinafter, the item value of the municipality (item) is referred to as the city / ward / town / village value, and the item value of the population (item) is referred to as the "population value".

As shown in FIG. 5, the data set DS in which the city, ward, town, village (item) and population (item) are added as attribute items to each record by performing the above processing for each record of the basic database <Kureka> DB1. Is constructed. Hereinafter, the data set DS of FIG. 5 is referred to as "data set <addition of public information> DS1". The data set construction unit 3 stores the constructed data set <additional public information> DS1 in the data set storage unit 6. The above is a simple example of the public information addition process. As described above, in the public information addition process, when the data set construction unit 3 constructs the data set DS, it is newly added to the record of the basic database DB (or the data set DS constructed based on the basic database DB). An item is added, and the item value of the newly added item is converted from the item value of the existing item based on the public information. By this public information addition process, a new type of attribute information (in this example, the city / ward / town / village value and the population value) can be added as the attribute information of each individual corresponding to each record of the dataset DS.

Here, as will become clear later, the score output model SS is composed of a function having one or a plurality of attribute information as explanatory variables. Therefore, by adding a new attribute item to the record by the public information addition process, the diversity of variable candidates can be expanded, and the score output is more accurate than when the variable candidates are limited. The possibility of building a model SS can be improved. In addition, by adding a new attribute item to the record by the public information addition process, the variety of input attribute item patterns (described later) can be expanded and various input attribute item patterns (described later) can be expanded, as will be described in detail later. Model construction information MM (described later) can be generated. In addition, the added items can be used for statistical analysis of individuals, and public information addition processing can increase the information that can be used for statistical analysis, improving the accuracy of statistical analysis. Can be improved. It should be noted that what kind of item is added by the public information addition process is determined in consideration of the usefulness of the score output model SS as a variable, the availability in statistical analysis, and the like.

In this example, the public information addition process has been described by taking as an example the case where the postal code value is converted into the city / ward / town / village value and the population value based on the public information, but the public information addition process is limited to the illustrated embodiment. It's not a thing. The information converted based on the public information may be information that can be a variable of the score output model SS or can be used for statistical analysis. For example, in the above example, the information converted based on the public information was the population value representing the population of the city, ward, town, or village to which the individual's residence belongs, but this affects which population value affects any prediction target. Possibility to give (for example, individuals living in populated areas are more likely to have credit card credit losses than individuals living in less populated areas) It is based on. In addition, the public information may be public information such as government statistics and land information. In this case, the government statistics include the employment rate, annual household income, municipal finance, number of hospitals, number of accidents, and the like. In addition, land information includes land prices, uses, city planning information, and the like. In addition, as public information, information that can convert attribute information such as information on daily life can be widely adopted. Further, the configuration is such that the information required for conversion is acquired from this server by accessing the server and the information is converted, but the conversion is performed based on the table registered in advance in the model construction device 2. It may be configured.

<Complementary information addition processing>
Next, the supplementary information addition process will be described by taking as an example the case where the supplementary information based on the basic database <life insurance> DB3 of FIG. 4 is added to the data set <public information addition> DS1 of FIG. In this embodiment, the case where the complementary information is added to the data set DS (that is, the data obtained by processing the basic database DB) is taken as an example, but the complementary information is added to the basic database DB that has not been processed. You may. For example, the person in charge of the management company specifies which basic database DB (or the data set DS based on the basic database DB) the supplementary information based on which basic database DB is to be added. In this case, the basic database DB (or data set DS) to which the complementary information is added is, for example, the data having the largest number of records.

As mentioned above, the record of the basic database <life insurance> DB3 has gender (item), age (item), municipality (item), annual income (item) and marriage classification (item) as attribute items. There is. The data set construction unit 3 selects some or all combinations of the attribute items that the records of the basic database <life insurance> DB3 have in common by the records of the data set <additional public information> DS1. Determined as a unit item group. For example, all the items common to each other are determined as a unit item group, and for example, the person in charge of the management company determines the items to be included in the unit item group. In this example, the data set construction unit 3 determines gender (item), age (item), and city / ward / town / village (item) as a unit item group.

Next, the data set construction unit 3 keys the value of each item of each item of the unit item group (corresponding to "a plurality of specific attribute information" in the claims) for each record of the basic database <life insurance> DB3. As a result, the records in which the value of each item matches are combined into one. At that time, the data set construction unit 3 deletes the annual income (item), the marriage classification (item), and the life insurance purchase record (item) in the summarized record, while the average annual income (item) and the estimated marriage probability (item). And a new life insurance purchase tendency (item) is added.

The data set construction unit 3 sets the item value of the average annual income (item) (hereinafter referred to as "average annual income value") as the average of the annual income value of the annual income (item) of each collected record. In the present embodiment, the average value is used, but it is also possible to use a value calculated by a statistical method such as a median value in addition to the average value. In addition, the data set construction unit 3 calculates the item value of the estimated marriage probability (item) (hereinafter referred to as "estimated marriage probability value") by a statistical method based on the marriage classification value of each summarized record. It is a numerical value that expresses the possibility of being married as a percentage. In addition, the data set construction unit 3 statistically sets the item values (hereinafter referred to as “life insurance purchase tendency values”) of the life insurance purchase tendency (items) based on the life insurance purchase actual values of each of the summarized records. It is a numerical value that expresses the possibility of purchasing life insurance as a ratio, which is calculated by various methods. As described above, the data set construction unit 3 uses the value of each item of each item of the unit item group as a key for the basic database <life insurance> DB3 to obtain the annual income value (attribute information), the marriage classification value (attribute information), and the value. Then, after totaling the life insurance purchase actual value (actual result related information) by a statistical method, the records in which each item value of each item of the unit item group is common are put together.

FIG. 6 shows the supplementary information addition database HK1 generated as a result of performing the above processing for the basic database <life insurance> DB3 of FIG. The first record R61 of the supplementary information addition database HK1 of FIG. 6 was generated by combining the first record R41 and the fifth record R45 of the basic database <life insurance> DB3 of FIG. It is a record, and the average annual income value, the estimated marriage probability value, and the life insurance purchase tendency value of the record R61 are calculated based on the record R41 and the record R45, respectively. In the present embodiment, complementary information such as the average annual income value and the estimated marriage probability value is generated based on one basic database DB (basic database <life insurance> DB3). This may be configured to generate complementary information corresponding to the unit item group based on a plurality of basic database DBs. The plurality of basic database DBs are, for example, all the basic database DBs that can be used to generate the target complementary information. Then, when the complementary information is generated based on a plurality of basic database DBs, it is possible to calculate the value of the complementary information by the weighted average in which the weight is changed for each basic database DB. In this way, by taking the weighted average of the average values calculated in the various basic database DBs, the complementary information can be an estimated value close to the actual situation in which the contents of the various basic database DBs are reflected. The statistical method is not limited to the weighted average value, and may be, for example, a median value or a median average value.

After constructing the database HK1 for adding supplementary information, the data set construction unit 3 puts the average annual income (item), estimated marriage probability (item), and life insurance purchase tendency as items in each record of the dataset <additional public information> DS1. Add (item) and income difference (item). Then, the data set construction unit 3 adds supplementary information to the average annual income (item), estimated marriage probability (item), and life insurance purchase tendency (item) of each record of the data set <addition of public information> DS1. Stores the average annual income value, estimated marriage probability value, and life insurance purchase tendency value of records in which each item value of each item in the unit item group of the record matches. Further, the data set construction unit 3 subtracts the average annual income value from the annual income value of each record to the income difference (item) of each record of the data set <addition of public information> DS1 (hereinafter referred to as "income difference value"). ) Is stored as an item value.

By the above processing, as shown in FIG. 7, the average annual income (item), estimated marriage probability (item), life insurance purchase tendency (item) and income difference (item) are added to each record of the dataset <public information addition> DS1. A dataset DS with the item) added is constructed. Hereinafter, the data set DS of FIG. 7 is referred to as a data set <public information + supplementary information addition> DS2. The first record R71 of the data set <public information + supplementary information addition> DS2 of FIG. 7 is for adding the supplementary information of FIG. 6 to the first record R51 of the data set <public information addition> DS1 of FIG. This is a record generated by adding information based on the first record R61 of the database HK1. The average annual income (item), estimated marriage probability (item) and income difference (item) are attribute items (therefore, the item value is attribute information), and the life insurance purchase tendency (item) is related to actual results. It is an item (hence, the item value is performance-related information).

The data set construction unit 3 stores the constructed data set <public information + supplementary information addition> DS2 in the data set storage unit 6. The above is an example of the complementary information addition process. According to this complementary information addition process, each record of one basic database DB (or a data set DS constructed based on the one basic database DB) is different from the one basic database DB. Attribute information and achievement-related information can be added based on the basic database DB of. When adding information, the consistency of the unit item group between the record of the database of the addition source and the record of the database of the addition destination is judged, so that the validity of adding the information is guaranteed.

By adding new attribute items and achievement-related items to the record by the complementary information addition process, the variety of variable candidates of the score output model SS can be expanded, and compared with the case where the variable candidates are limited. , The possibility of constructing a highly accurate score output model SS can be improved. In addition, by adding a new attribute item to the record by the complementary information addition process, the variety of input attribute item patterns (described later) can be expanded and various input attribute item patterns (described later) can be expanded, as will be described in detail later. Model construction information MM (described later) can be generated. In addition, since the added items can be used for statistical analysis of individuals, it is possible to increase the information that can be used for statistical analysis by supplementary information addition processing, and the accuracy of statistical analysis can be increased. Can be improved. In particular, in the present embodiment, the management company is provided with basic database DBs from a plurality of different companies, and is an environment capable of collecting a plurality of different basic database DBs constructed for different individuals with different purposes. It is in. The supplementary information addition process is a process that is preferably performed by utilizing the fact that the management company is in such an environment.

Regarding the average annual income value and the estimated marriage probability value, if the number of annual income values and the number of marriage classification values that are the basis of these calculations are sufficient (when the number of records to be summarized is sufficient), the average annual income The value and the estimated marriage probability value are also statistically reliable values, but if the number is not sufficient, the average annual income value and the statistical reliability of the estimated marriage probability value become a problem. For example, if there is only one basic annual income value for the average annual income value, the statistical property of "average" will be lacking. Based on this, the data set construction unit 3 may perform the following processing instead of aggregating the item values of the newly added items by the statistical method in the supplementary information addition processing.

To simplify the explanation, an example of adding an average annual income (item) to a record of the basic database X will be described. First, a plurality of basic database DBs stored in the basic database storage unit 5 will be used as an example. The attribute information of the attribute item (however, the attribute item included in the record of the basic database X) is input, and a model for predicting the average annual income of an individual having the plurality of attribute information is generated. The model is, for example, a machine learning model (or statistical model). The basic database DB is selected as an item from those having the plurality of attribute items and annual income (item). Then, the data set construction unit 3 obtains the item value of the average annual income (item) added to one record of the basic database X by inputting the attribute information of the plurality of attribute items of the one record into the model. It is the predicted value of the average annual income. A configuration that performs the above processing may be used.

Further, in this example, the supplementary information addition process has been described by taking as an example the case where the supplementary information based on the basic database <life insurance> DB3 of FIG. 4 is added to the data set <addition of public information> DS1 of FIG. The supplementary information addition process is not limited to the illustrated embodiment. For example, the items newly added to the record are not limited to the illustrated average annual income (item), estimated marriage probability (item), and life insurance purchase tendency (item), but are based on the items possessed by the record in the basic database DB. Appropriately defined.

<Merge process>
Next, the merging process will be described by taking as an example the case where the basic database <credit card> DB1 of FIG. 2 and the basic database <rent delinquency> DB2 of FIG. 3 are merged. The merge process is a process of merging two different basic database DBs. The merge process described below triggers, for example, that the person in charge of the management company has specified the basic database <Credit card> DB1 and the basic database <Rent delinquency> DB2, and then instructed to start the process. Is executed as.

When merging the basic database <Credit card> DB1 and the basic database <Rent delinquency> DB2, the data set construction unit 3 first verifies the strength of the correlation of the performance-related information of each database. In this example, the strength of the correlation of performance-related information, that is, the strength of the correlation between the actual value of Creca bad debt and the actual value of rent delinquency, is the presence or absence of experience of credit card bad debt by a common person and the rent delinquency. It means the strength of the correlation with the presence or absence of experience, and the higher the possibility that a person who has experienced a credit card bad debt has also experienced a rent delinquency at the same time, the stronger the correlation. Verification of the strength of correlation of performance-related information in each database may be performed by any method. For example, a calculation formula is used in which each record of each database is input and information indicating the strength of correlation of performance-related information of each database is output. As an example, this calculation formula calculates the high possibility that a person having a combination of a plurality of attribute information will experience a credit card bad debt and a rent delinquency at the same time for each combination of a plurality of attribute information. It is a calculation formula that determines that there is a strong correlation if there is a strong tendency for both credit card bad debt and rent delinquency to occur regardless of the combination of information. In addition, for each combination of performance-related information, a table in which the combination of performance-related information and the information indicating the strength of correlation are associated is registered in advance, and this table is used to display the performance-related information of each database. The configuration may be such that the strength of the correlation is verified. Further, the following configuration may be used. That is, first, based on the basic database <Kureka> DB1, one or more attribute information is input, and the high possibility of causing credit card bad debt (expressed as "estimated Creca bad debt probability" for convenience) is determined. A model to be output (this model is expressed as "model <credit card credit card>" for convenience) is constructed. Enter one or more attribute information corresponding to each record in the basic database <Rent delinquency> DB2 into this model <Credit card credit card loss>, and correlate the obtained estimated credit card credit card loss probability with the actual rent delinquency value of each record. It may be configured to analyze the sex and vice versa, and determine the strength of the correlation based on the analysis result.

If there is no strong correlation between the performance-related information of each database, the data set construction unit 3 does not merge the databases. On the other hand, when the performance-related information of each database has a strong correlation of a certain level or more (in this example, it is assumed that there is a certain level of correlation or more), the data set construction unit 3 merges each database in the following manner. To build the dataset DS.

FIG. 8 shows the basic database <Credit card> DB1 of FIG. 2 (provided that there are only four records whose contents are clearly indicated in FIG. 2) and the basic database <Rent delinquency> DB2 of FIG. FIG. 3 is a diagram showing a data set DS constructed by merging (assuming that there are only four records in FIG. 3). Hereinafter, the data set DS of FIG. 8 is referred to as a data set <after merging> DS3. In the data set <after merging> DS3 of FIG. 8, the records R81 to R84 correspond to the records R21 to R24 of the basic database <creca> DB1 of FIG. 2, respectively. Further, in the data set <after merging> DS3 of FIG. 8, the records R85 to R88 correspond to the records R31 to R34 of the basic database <rent delinquency> DB2 of FIG. 3, respectively.

As shown in FIG. 8, the merge is basically performed by adding each record of the other database to each record of one database. At that time, as described later, attribute items are added and performance-related items are replaced. Further, as shown in FIG. 8, each record of the data set <after merging> DS3 is set to include all kinds of attribute items possessed by the records of each database (for attribute items for which attribute information cannot be acquired). Is a null value as appropriate.).

Further, as shown in FIG. 8, the record of the data set <after merging> DS3 does not include each performance-related item of each database as an item, and instead, the total risk (item) in which each performance-related item is synthesized. ) Is included. As shown in FIG. 8, the data set construction unit 3 sets the item value (hereinafter, referred to as “total risk value”) of the total risk (item) of each record to the following value. That is, the data set construction unit 3 sets the total risk value of the records R81 to R84 based on the basic database <credit card> DB1 to be the same as the actual credit card bad debt value. Further, the data set construction unit 3 sets the total risk value of the records R85 to R88 based on the basic database <rent delinquency> to be the same as the actual rent delinquency value. As a result, the total risk value will be the value that inherits the actual credit card bad debt value and the actual rent delinquency value. The total risk (item) is a performance-related item, and the total risk value is performance-related information.

Since the merge of the basic database DB is performed when the correlation of the performance-related information of each database is high, the total risk value obtained by synthesizing the performance-related information related to different types of risks is the total risk (credit and loan). It can be used as a value indicating the magnitude of (the possibility of loss due to bad debt or delinquency, etc.). In this example, the total risk value is a combination of item values related to two types of risk, but when three or more basic database DBs are merged, three or more types of item values are combined. Will be done. Basically, the more types of item values to be combined, the stronger the validity of the total risk value. In addition, in this example, the comprehensive risk (item) was explained, but if the performance-related items of each basic database DB to be merged are both items related to needs (possibility of purchasing products and services), these are Comprehensive needs (items) are newly added to the record by synthesizing the items.

Further, as shown in FIG. 8, each record of the data set <after merging> DS3 includes a weight (item) as an item, and the weight (item) stores a weight value as an item value. The data set construction unit 3 adds a weight (item) to the record when the performance-related information of the performance-related item of each merged basic database DB takes two values of a positive class and a negative class. The weight value is used as training data when each record of the data set <after merging> DS3 (including the data set DS generated based on the data set DS) is trained in the score output model SS using the loss function. This is the "weight" given to the loss function so that the difference between the regular ratio of the basic database <Kureka> DB1 and the regular ratio of the basic database <rent delinquency> DB2 is absorbed. Since the weight value is reflected in the loss function, it is possible to use the data set DS constructed by merging a plurality of basic database DBs having different positive example ratios as the training data of the score output model SS. Hereinafter, an example of the weight value calculation method will be described.

Now, in the basic database <Credit card> DB1, it is assumed that the credit card bad debt record value is in the positive class at a rate of X% (in this case, the positive case ratio (ratio of the positive class) is X%). On the other hand, it is assumed that the positive ratio of the basic database <rent delinquency> DB2 is Y%. When X% ≠ Y%, the regular ratio of the basic database <Credit card> DB1 and the regular ratio of the basic database <Rent delinquency> DB2 are different. In this case, the data set construction unit 3 first randomly divides the basic database <credit card> DB1 into several subsets. The number of divisions should be appropriately determined by the number of records in the basic database <Credit card> DB1, but for the sake of simplification and clarification of the explanation, in this example, it is assumed that the divisions are divided into subsets SA1 to SA4. At this time, the data set construction unit 3 adjusts so that the positive example ratio between the subsets does not vary greatly. The dataset construction unit 3 then constructs the model MA1 using the subsets SA1 to SA3. The data set construction unit 3 applies this model MA1 to the subset SA4 and obtains a predicted value. The data set construction unit 3 obtains predicted values for all the records of the basic database <Credit card> DB1 by performing this work four times while changing the subset. The data set construction unit 3 calculates the average WA of the predicted values for four times. This average WA can be said to be the probability that the predicted value will be in the positive class.

Next, the data set construction unit 3 divides the subset, constructs a model for each combination of the subsets, and acquires the predicted value for the basic database <rent delinquency> DB2 in the same manner, and averages the predicted values WB. Is calculated. The data set construction unit 3 calculates the ratio of the average WA of the predicted values and the average WB of the predicted values. In the calculation of the ratio, the one with the larger value is the numerator and the one with the smaller value is the denominator. When the average WA of the predicted values is "0.025" and the average WB of the predicted values is "0.02", the ratio is "0.025 / 0.02 = 1.25". The data set construction unit 3 bases the weight value of the record based on the basic database <Kureka> DB1 and the record based on the basic database <rent delinquency> DB2 based on the ratio of the average WA of the predicted values and the average WB of the predicted values. Calculate the weight value of. For example, when the ratio is "1.25", the weight value is "1" for the record based on the basic database <Credit card> DB1, and the weight value is "1." For the record based on the basic database <Rent delinquency> DB2. 25 ”is given. The above is an example of the weight value calculation method.

The merge process has been described above by taking as an example the case where the basic database <credit card> DB1 of FIG. 2 and the basic database <rent delinquency> DB2 of FIG. 3 are merged. Public information addition processing, complementary information processing, and merging processing for merging a different basic database DB (or data set DS based on the basic database DB) are performed on the data set DS generated by the merging process. As a result, the items of each record will increase and be enriched, and the number of data set DS will increase.

The public information addition process, the complementary information addition process, and the merge process that can be executed when the data set construction unit 3 constructs the data set DS have been described above. The data set construction unit 3 uses the plurality of basic database DBs stored in the basic database storage unit 5 to perform the above processing in various modes to construct a plurality of data set DSs, and the data set storage unit 6 Remember in. For example, the person in charge of the management company specifies which basic database DB is used in what mode, and which process is executed in what mode, and the data set construction unit 3 performs data according to the designation. A set DS is constructed and stored in the data set storage unit 6. As a result, various types of dataset DSs having suitability as learning data used for constructing (learning) the score output model SS to be predicted are constructed and stored in the dataset storage unit 6. The data set DS stored in the data set storage unit 6 is appropriately added with records as the number of records in the original basic database DB increases.

The model construction unit 4 constructs a score output model SS suitable for the client company through the steps described later, and stores the score output model SS in the score output model storage unit 8. The client company means a company that wants to receive a service using the score output model SS. Further, the score output model SS according to the present embodiment is a model that inputs one or a plurality of attribute information and outputs a predicted score of a specific prediction target. That is, the score output model SS inputs one or more attribute information, and for an individual who has all the attributes indicated by the input one or more attribute information, the magnitude of the risk or need for a specific prediction target and the risk thereof. It outputs a score that represents the certainty that the needs will be realized.

For example, as described above, one of the prediction targets in the present embodiment is "credit card risk", but the score output model SS related to credit card risk inputs one or more attribute information and outputs the credit card risk score. To do. This Creca Risk Score is given when a credit card is issued to an individual who has all of the entered one or more attribute information (for example, an individual who is 30 years old, has a male gender, has a zip code of 12 ...). , The risk of bad debt is quantitatively expressed by a scored value.

Here, the score output model SS internally outputs the probability corresponding to the prediction target as an intermediate output. For example, the score output model SS related to credit card risk outputs the'probability'of bad debt as an intermediate output. It is possible to use this intermediate output as the final output, but in this case, in a highly accurate model, the output is biased toward "0" or "1", so for example, " The difference of "0.01" and the difference of "0.01" in the vicinity of "0.98" have completely different meanings even if they are the same "0.01". In consideration of this, the score output model SS internally scores (scores) the intermediate output, which is a probability, by a conversion formula using a logarithm.

As a specific example, the conversion formula is "W × LOG (p / (1-p))". W is a score coefficient and p is an intermediate output which is a probability. Further, "2" is used for the bottom of the LOG (however, it is not limited to "2"). The score coefficient is a coefficient that defines the width of the score. For example, when "100" is used as the score coefficient, the score is distributed in the range of "0 to 1000". It is possible to set the upper limit of the score by adjusting the score coefficient in consideration of the score distribution. The lower limit is generally "0". In the case of a dataset DS with a low positive example ratio, the upper limit of the score can be set to "1500" or "2000". This is because the score cannot be used correctly in a distribution that concentrates on the upper limit. Basically, the number of records to be the upper limit is set so as not to be larger than the number of records to be "upper limit -10" to "upper limit -1". However, the conversion formula for scoring the intermediate output, the width of the score, the upper and lower limits of the score, etc. are not limited to those exemplified.

In the present embodiment, the score output model SS is a function including one or a plurality of attribute information as explanatory variables and parameters for adjusting each variable, and is composed of a function having a predicted value to be predicted as an objective variable. .. Then, constructing the score output model SS means adjusting and determining the value of each parameter of the score output model SS by learning using the data set DS. In the following, the pattern of the attribute item corresponding to one or more attribute information input to the score output model SS is referred to as an “input attribute item pattern”.

The processing of the model construction unit 4 can be roughly divided into a model construction information generation process and a model selection / construction process. Hereinafter, each step will be described in sequence.

<Model construction information generation process>
First, the model construction information generation process will be described. In the model construction information generation process, the model construction unit 4 combines the prediction target, the input attribute item pattern, the dataset DS used for model construction (learning), and the algorithm used for model construction (learning). Model construction information MM is generated for each. FIG. 9 is a diagram showing the relationship between the model construction information and the prediction target, the input attribute item pattern, the data set DS, and the algorithm. The model construction information MM is information that specifies a prediction target, an input attribute item pattern, a data set DS, and an algorithm. For example, the model construction information MMa in FIG. 9 is information that specifies the prediction target Ya, the input attribute item pattern Na, the data set DSa, and the algorithm Aa.

The number of combinations of prediction targets and input attribute item patterns may be enormous, but what combination of score output model SS will be needed in the future (whether it is desired by the client company). In consideration of such factors, it is operated so that the combination of the prediction target and the input attribute item pattern corresponding to the score output model SS that may be required by the client company is covered without becoming unnecessarily large. Selected by the person in charge of the company. In addition, when selecting the dataset DS, the record of the dataset DS has performance-related items corresponding to the prediction target (for example, when the prediction target is creca risk, the credit card bad debt record corresponding to creca risk (for example) It is performed appropriately in consideration of the fact that the record has an item) and that the record has an item corresponding to the input attribute item pattern. If there are a plurality of dataset DSs having sufficient eligibility, a plurality of dataset DSs may be selected. Further, as an algorithm, a plurality of algorithms applicable to model construction (learning) are prepared. Algorithms are, for example, logistic regression, decision trees, random forests, gradient boosting, SVMs, naive Bayes, deep learning, applications of these, and combinations thereof (of course, not limited to the examples). ).

Based on a certain model construction information MM, the model construction unit 4 uses the attribute information corresponding to the input attribute item pattern specified by the information as the explanatory variable, and the predicted value of the prediction target specified by the information as the objective variable. The data set DS specified by the information is used to train the model by the algorithm specified by the information, whereby the model can be constructed (final parameters are determined). In the following, a model constructed by using a part of the data set DS as training data and a part as test data is expressed as a "candidate model", and the entire data set DS is used as training data. The model constructed by the above is expressed as a "temporary model" to distinguish between them (details will be clarified later).

As shown in FIG. 9, basically, the model construction information MM is generated for each combination of the prediction target, the input attribute item pattern, the data set DS, and the algorithm. However, if the accuracy of the candidate model generated based on the model construction information MM is lower than a certain level, the model construction unit 4 discards the model construction information MM (it is not stored in the model construction information storage unit 7). .. Further, the model construction unit 4 basically includes a plurality of model construction information MMs belonging to the combination of the prediction target and the input attribute item pattern (as shown in FIG. 9, basically the data set DS and the algorithm). Of the (existing for each combination), the one with the highest accuracy of the candidate model to be constructed is specified as preset model information. Hereinafter, a process of discarding the predetermined model construction information MM and a process of specifying the preset model information will be described.

When a certain model construction information MM is targeted for processing, the model construction unit 4 constructs a candidate model based on the model construction information. To elaborate on the process of generating the candidate model, the model building unit 4 describes the data set DS specified by the model building information MM according to the algorithm specified by the model building information MM (hereinafter referred to as “target algorithm”). Each of the records (hereinafter referred to as "target data set") is converted into unit processing data and stored in the buffer. Unit processing data is data that has been converted into a format that can be used for learning a model using the target algorithm after deleting unnecessary information from the records of the target data set, and has a correct label. Is.

For example, the unit processing data is data in which a correct answer label is added to a feature vector expressing the features of each item value of each item of the input attribute item pattern as a numerical string based on actual performance-related information. In this case, each element of the feature vector corresponds to each item value of each item of the input attribute item pattern, and each element serves as an explanatory variable of the candidate model. Further, the model building unit 4 is constructed when the records of the target data set have weights (items) (that is, when a plurality of basic database DBs having different target data sets are merged and constructed). Stores the weight value in association with the unit processing data. Hereinafter, a set of unit processing data generated based on the target data set is referred to as a "processing data group".

Next, the model building unit 4 randomly divides the processing data group. In this example, the division ratio is 25:75. When dividing, adjust so that the ratio of positive examples of each part after division does not differ significantly. Next, the model construction unit 4 constructs a candidate model using the target algorithm using 75% of the divided data as training data. Further, when the model building unit 4 adjusts the parameters by a method of reducing the loss value calculated by using the loss function (as an example, the stochastic gradient descent method), each unit is based on the weight value. Adjust the weights given to the positive and negative classes of the processing data appropriately.

Next, the model building unit 4 verifies the accuracy of the generated candidate model using the remaining 25% as test data. The model building unit 4 verifies the accuracy of the candidate model using, for example, evaluation indexes such as AUC (Area Under the ROC Curve), KS (Kolmogorov-Smirnov), F1 value, and Log-Loss (naturally, the candidate model). The method of evaluating is not limited to the illustrated method). Further, the model construction unit 4 verifies the accuracy of the candidate model by using a cross-validation method in which the training data and the test data are randomly exchanged and evaluated as needed. In the present embodiment, for convenience of explanation, the accuracy of the candidate model is calculated as a “candidate model accuracy value” which is a quantitative value using a certain evaluation index.

When the candidate model accuracy value does not meet the standard, the model construction unit 4 discards the model construction information MM and does not store the model construction information MM in the model construction information storage unit 7. On the other hand, when the candidate model accuracy value satisfies the reference (when the accuracy is higher than the reference), the model construction unit 4 adds the candidate model accuracy value to the model construction information MM.

As described above, after discarding the model construction information MM whose candidate model accuracy value does not meet the standard and adding the candidate model accuracy value to the satisfying model construction information MM, the model construction unit 4 sets the prediction target and the input attribute item. For each combination with the pattern, the model construction information MM having the highest candidate model accuracy value is specified among the model construction information MMs belonging to the combination. Hereinafter, the specified model construction information MM is referred to as "preset model information". For example, in the example of FIG. 9, the model construction unit 4 selects the model construction information MM having the highest candidate model accuracy value from the group G1 of the model construction information MM belonging to the combination of the prediction target Ya and the input attribute item pattern Na. Determined as preset model information. Further, the model construction unit 4 determines the model construction information MM having the highest candidate model accuracy value from the group G2 of the model construction information MMs belonging to the combination of the prediction target Ya and the input attribute item pattern Nb as preset model information.

<Model selection / construction process>
In the model selection / construction process, the model construction unit 4 selects a model construction information MM suitable for the client company based on the information provided by the client company, and outputs a score based on the selected model construction information MM. Build a model SS. The following is an example of a case where the client company is a credit card issuing company, a model construction information MM suitable for the client company is selected, and a score output model SS that outputs a Creca risk score is constructed based on the model construction information MM. Then, the processing of the model construction unit 4 will be described. In addition, this client company has at least gender (item), age (item) and city / ward / town / village (item) as attribute items, and records that have credit card credit loss record (item) as performance-related items are accumulated. It is assumed that the actual record database is held.

First, the client company is provided with an interface having a combination of attribute items and an input field for selecting and inputting a prediction target for which a prediction score is desired to be obtained. A service using the score output model SS is basically requested to provide a predicted score (in this example, a credit card risk score) of an individual having one or more attribute information, and provides a predicted score accordingly. It is to do. Based on this, the client company selects a combination of attribute items (however, selection may be made) of the records of the performance database managed by the client company, and inputs them in the input field of the interface. In this example, it is assumed that the combination of gender (item), age (item) and city / ward / town / village (item) is entered. In addition, the client selects and inputs the credit card bad debt risk as the prediction target for which the prediction score is desired to be obtained.

Depending on the item, the item value may be expressed by different methods between the management company and the client company. For example, the age (item) may be expressed as a specific age in increments of 1 year, in increments of 10 years such as in the thirties, and young people. , May be expressed by the age group such as middle-aged. Based on this, for items that can express item values in different ways, the interface is provided with a field for selecting and inputting the expression method. In addition, the client company uploads the performance database it owns to the model construction device 2 by a predetermined means. When the combination of attribute items entered by the client company (hereinafter referred to as "selected attribute item pattern"), the input prediction target (hereinafter referred to as "selection prediction target"), and the expression method for a specific item are specified. The information indicating the expression method in the above and the performance database (hereinafter, these are collectively referred to as "client company-related information") are stored in the client company information storage unit 9.

There are four methods for the model building unit 4 to build a score output model SS suitable for the client company, the first to fourth methods. Hereinafter, each method will be described in sequence. In the first method, the model construction unit 4 refers to the client company-related information stored in the client company information storage unit 9 and acquires a combination of the selection prediction target and the selection attribute item pattern. Next, the model construction unit 4 selects preset model information as the model construction information MM among the model construction information MMs belonging to the combination of the selection prediction target and the selection attribute item pattern stored in the model construction information storage unit 7. In this example, the model building unit 4 specifies a combination of gender (item), age (item), and city / ward / town / village (item) as an input attribute item, and among the model building information MMs that specify the credit card risk as a prediction target. , Select the model construction information MM determined as the preset model information.

The model building unit 4 builds the score output model SS based on the model building information MM. Building the score output model SS based on the model construction information MM means that the data set DS specified in the model construction information MM is used and the specified algorithm is used to construct the specified input attribute item pattern (= selection attribute). It means to build a model that outputs the prediction score of the specified prediction target (= selection prediction target) by inputting the item pattern). The model construction unit 4 stores the constructed score output model SS in the score output model storage unit 8 in association with the identification information of the client company.

When constructing the score output model SS, the model construction unit 4 uses the data set DS for constructing the score output model SS when the client company specifies the expression method of the item value for a certain item. Change the item value of that item to the representation method specified by the client company. Further, the model building unit 4 does not divide the data set DS stored in the data set storage unit 6 at that time into training data and test data, and uses all the records as training data to generate the score output model SS. To construct. Note that using all records as training data means not using a part of them as test data as in the case of generating candidate models, and records that can cause noise are selected from the training data. Exclusions are of course done. This is because the accuracy verification has already been completed, so it is not necessary to prepare test data, and the larger the number of training data, the better the accuracy of the constructed score output model SS. .. Further, by using the data set DS stored in the data set storage unit 6 at that time as the data set DS, the score output model SS including the records newly added by the activity of the data provider is learned. It can be performed. It should be noted that the point of changing the expression method of the item value in the predetermined case and the point of using all the records of the data set DS when constructing the score output model SS are provisionally described in the second to fourth methods described later. The same applies when generating a model.

According to the above first method, the score output model SS can be quickly constructed with a small load as compared with the second to fourth methods described later. In particular, the score output model SS (strictly speaking, the candidate model) corresponding to the preset model information is a model whose high accuracy has been proved by a test using the data set DS, so that it is universal enough for the predicted output score. It is assumed that it has sex, and it can be said that it is qualified as a model used for services provided to client companies.

However, in the first method, the accuracy of the score output model SS has not been verified by a test using the performance database of the client company, and the score output model SS is compared with the second to fourth methods described later. The explanatory power of the output of is inferior. Based on this, for example, it is possible to make a difference in the service level depending on the consideration and adopt the first method in the case of a low service level. Since the first method does not use the performance database of the client company, it is not necessary to receive the performance database.

The first method may have the following configuration. That is, in the model construction information generation process, the model construction unit 4 learns a part of the candidate model (a part of the data set DS specified by the model construction information MM) together with the model construction information MM or instead of the model construction information MM. (Model constructed by using as training data) or temporary model (model constructed by using all of the dataset DS specified by the model construction information as training data) is stored. Then, in the model selection / construction process, the model construction unit 4 receives a request to construct a score output model SS together with information on a pattern of a specific attribute item and a specific prediction target, and a candidate model corresponding to the preset model information. Alternatively, score output a tentative model (the most accurate candidate model or the tentative model corresponding to the most accurate candidate model among the candidate models generated for the combination of the pattern of a specific attribute item and a specific prediction target). Model SS. In the case of the above configuration, since it is not necessary to construct a temporary model in the model selection / construction process, the processing load in this process is small in that respect. However, if the records of the dataset DS increase after the candidate model or the provisional model is generated, the candidate model or the provisional model stored in the model construction information storage unit 7 will not reflect the increased records. In that respect, the accuracy is inferior to that in the case of generating a temporary model each time in the model selection / construction process.

The second method is as follows. That is, the model construction unit 4 refers to the client company-related information stored in the client company information storage unit 9, and acquires a combination of the selection prediction target and the selection attribute item pattern. Next, the model construction unit 4 creates a temporary model based on each of the model construction information MMs stored in the model construction information storage unit 7 and which belong to the combination of the selection prediction target and the selection attribute item pattern. To construct. The tentative model is a model constructed by using all the records of the data set DS specified by the model construction information MM as training data.

Next, the model construction unit 4 verifies the accuracy of each of the provisional models by using the performance database of the client company as test data. The model building unit 4 verifies the accuracy of the tentative model by evaluating the relationship between the predicted score output by the tentative model and the actual database based on judgment, correlation, stability, and the like. For the evaluation of the discriminant power, the AUC, KS, F1 values and the like described in the verification of the accuracy of the candidate model can be used. For the evaluation of the correlation, Pearson / Spearman correlation, the chi-square value of the score distribution, the Kullback-Leibler information amount standard of the score distribution, and the like can be used. For the evaluation of stability, the rate of change (variance) using time series variables, the rate of change of score distribution (distribution distance), and the like can be used. In this example, the model building unit 4 verifies the accuracy of the tentative model using any of the evaluation indexes, and calculates the tentative model accuracy value, which is a quantitative value representing the accuracy of the tentative model, for each tentative model. It shall be.

It should be noted that processing for using each record of the performance database as test data, such as selection of information, feature vectorization, and assignment of correct answer labels, is appropriately performed. Since it is guaranteed that the data set DS used for constructing the temporary model is different from the actual database of the client company, the model construction unit 4 does not perform cross-validation. The model construction unit 4 identifies (selects) the model construction information MM having the highest tentative model accuracy value, and uses the tentative model generated by the model construction information MM as a score output model SS suitable for the client company. Is stored in the score output model storage unit 8 in association with the identification information of the client company.

According to this second method, among the tentative models, the model judged to have the highest accuracy using the performance database constructed based on the actual activities of the client company is selected as the score output model SS. In that respect, the model suitable for the client company can be selected. In particular, in the second method, the accuracy of each (all) of the provisional models based on each (all) of the model construction information MM belonging to the combination of the selection prediction target and the selection attribute item pattern using the performance database of the client company. Since the optimum model is selected after the verification of, it is possible to surely select the tentative model that is considered to be the most suitable for the client company from the tentative models that can be constructed, and in that respect, all models. It has an advantage over the third and fourth methods (described later), which do not verify all the provisional models based on the construction information MM. On the other hand, in the second method, it is necessary to construct a tentative model for each model construction information MM belonging to the combination of the selection prediction target and the selection attribute item pattern and to verify the accuracy using the actual database as test data. , The load is heavy and the processing time is long. The following third method has improved this point.

The second method may have the following configuration. That is, in the model construction information generation process, the model construction unit 4 learns a part of the candidate model (a part of the data set DS specified by the model construction information MM) together with the model construction information MM or instead of the model construction information MM. (Model constructed by using as training data) or temporary model (model constructed by using all of the dataset DS specified by the model construction information as training data) is stored. Then, in the model selection / construction process, the model construction unit 4 provides information on the pattern of a specific attribute item and a specific prediction target, and a performance database, and when there is a request for model construction, the model construction unit 4 has a specific attribute. Test the candidate model or tentative model belonging to the item pattern and the combination of specific prediction targets using the performance database to verify the accuracy of the candidate model or tentative model, and select the most accurate candidate model or tentative model. Use the score output model. In the case of the above configuration, since it is not necessary to construct a temporary model in the model selection / construction process, the processing load in this process is small in that respect. However, if the records of the dataset DS increase after the candidate model or the provisional model is generated, the candidate model or the provisional model stored in the model construction information storage unit 7 will not reflect the increased records. In that respect, the accuracy is inferior to that in the case of generating a temporary model each time in the model selection / construction process.

As a premise of the third method, the model building unit 4 separately executes the following processing when performing the processing by the second method. That is, in the process of the processing related to the second method, the model construction unit 4 sets the identification information of the model construction information MM and the temporary model constructed by the information for each model construction information MM to be processed by the client company. The combination with the verification result (provisional model accuracy value in this example) of the above is recorded and stored in the verification result database (not shown) of the model construction information storage unit 7. Here, in what cases the tentative model accuracy value increases for the model construction information MM is optimized by combining the model construction information MM as a search space based on the characteristics of various elements of the model construction information MM. It can be considered as a problem, and it is possible to efficiently search for the optimum combination (combination that maximizes the tentative model accuracy value) by a heuristic method such as the Bayesian optimization method. However, of course, it is also possible to use other neighborhood search methods and other combination optimization methods.

As the elements of this search space, for example, the following can be adopted. Type of prediction target, number of data set DS records specified by model construction information MM, data set DS items specified by model construction information MM, correct example ratio of dataset DS specified by model construction information MM, model construction information It is generated based on the basic statistical value for each item of the data set DS specified by the MM, the AUC for each item in the data set DS specified by the MM, the algorithm specified by the model construction information MM, and the model construction information MM. Hyperparameters of the tentative model, the degree of influence of the variables on the tentative model.

In the third method, first, the model building unit 4 builds a selection model based on the verification result database in which records are accumulated to some extent. This selection model is constructed based on the model construction information MM when the characteristics of various elements of a certain model construction information MM are in a predetermined state (when the characteristics of the elements are related in a predetermined manner). It is a model that outputs the certainty that the candidate model to be performed is highly accurate. In this example, the selection model takes the feature quantities of the features of various elements of the model construction information MM as inputs (variables), and outputs an evaluation value that quantitatively represents the certainty that a high-precision candidate model is constructed. It shall be a model. The "characteristics of various elements of the model construction information MM", which is the input of the selection model, corresponds to the "state of each element affecting the construction of the tentative model" in the claims.

When selecting the model construction information MM for a certain client company, the model construction unit 4 uses the selection model to calculate the evaluation value of the model construction information MM belonging to the combination of the selection prediction target and the selection attribute item pattern. To do. Then, the model construction unit 4 selects the model construction information MM only for the model construction information MM having an evaluation value equal to or higher than a certain value. The optimum model construction information MM is selected by the same method as the second method (a method of using the performance database of the client company as test data). According to this third method, the model construction information MM that performs verification using the performance database is compared with the case where all the model construction information MMs belonging to the combination of the selection prediction target and the selection attribute item pattern are targeted. Since the number is narrowed down, the score output model SS can be efficiently constructed.

Further, in the fourth method, the model building unit 4 executes the following processing. Here, in selecting the model construction information MM capable of constructing a highly accurate model by the combination optimization method, in addition to the characteristics of various elements of the model construction information MM, the characteristics of various elements of the client company's performance database are selected. By adding this, it becomes possible to select the model construction information MM with higher accuracy.

Based on this, in the fourth method, first, the model construction unit 4 not only features the various elements of the model construction information MM, but also the actual database of the client company, based on the verification result database in which records are accumulated to some extent. Build a selection model that takes into account the characteristics of various elements. In this selection model, when the characteristics of various elements of a certain model construction information MM are in a predetermined state and the characteristics of various elements of the performance database of the client company are in a predetermined state, the model construction information MM The candidate model constructed based on is a model that outputs the certainty of high accuracy. In this example, the selection model is a high-precision candidate by inputting (variables) the feature quantities of the features of various elements of the model construction information MM and the feature quantities of the features of various elements of the performance database of the client company. It is assumed that the model outputs an evaluation value that quantitatively expresses the certainty that the model is constructed.

The various elements of the actual database are, for example, the number of data items in the actual database, the correct example ratio of the actual database, the basic statistical values for each data item in the actual database, the AUC for each data item in the actual database, and the prediction target of the actual database. .. When selecting the score output model SS for a certain client company, the model construction unit 4 calculates the features of various elements of the performance database provided by the client company, and uses the selection model to select the features. The evaluation value of each model construction information MM belonging to the combination of the prediction target and the selection attribute item pattern is calculated. Then, the model construction unit 4 selects a model only for the model construction information MM having an evaluation value of a certain value or higher. As a result, the accuracy is higher than that of the third method, and more efficiently than the second method, which targets all model construction information MMs belonging to the combination of the selection prediction target and the selection attribute item pattern. The score output model SS can be constructed. The "various elements of the performance database" that are the inputs of the selection model correspond to the "state of each element related to the performance database" in the claims.

Next, a service using the score output model SS and how the predicted score is used in the client company will be described. FIG. 10 is a diagram used for explaining a service providing system 10 (corresponding to the “information processing system” in the claims) that provides a service using the score output model SS for a client company. As shown in FIG. 10, the service providing system 10 is connected to the business system 11 of the client company via the network N including the Internet. Further, a Web API for receiving the service is disclosed by a company that operates the service providing system 10 (it does not necessarily have to be an asset management company, but it is assumed that it is an asset management company in this example). As shown in FIG. 10, the business system 11 has a terminal 12 connected to a LAN. Although details are omitted, authentication of the client company and encryption of communication performed by each device are appropriately performed. When the client company is authenticated, the service providing system 10 acquires the identification information of the client company.

Hereinafter, the service provided by the service providing system 10 will be described on the assumption that the client company is a credit card issuing company and a new customer requests the issuance of a credit card. When requesting the issuance of a credit card, the "value of each item included in the selection attribute item pattern specified by the client company" (hereinafter referred to as "new customer attribute information group") for the new customer is For example, it can be properly obtained by filling in the required fields on the application form.

The client company uses the browser of the terminal 12 of the business system 11 to make a request including information specifying a predetermined WebAPI and using the new customer attribute information group as a query parameter, and requests a response of the new customer's Creca risk score. .. The service providing system 10 may be a score output model storage unit 8 (naturally, it may be another storage means, and when it is another storage means, the model construction system 1 or the service provision) in response to a request from the client company. The score output model SS associated with the identification information of the client company is acquired from the storage means of the device belonging to the system other than the system 10). The score output model SS acquired here is a model suitable for the client when the score output model SS is constructed by the above-mentioned second to fourth methods. The service providing system 10 inputs a new customer attribute information group into the score output model SS, and obtains a credit card risk score output by the score output model SS. The service providing system 10 responds to the terminal 12 with the acquired credit card risk score.

From the above, the client company can obtain a credit card risk score for new customers. Here, the new customer is a customer who cannot acquire past achievements. However, the constructed score output model SS does not input the past performance of the individual when outputting the predicted score (predicted value of the predicted target regarding risk or needs) of a certain individual, and the above example. As shown in, the "attribute" that can be acquired even by a new customer is input. Therefore, by using the constructed score output model SS, the predicted score, that is, the risk and needs of the target individual, can be obtained by using the attributes of the target individual without acquiring the past achievements. Can provide information that can be used for judgment.

The client company uses the acquired credit card risk score in, for example, the following aspects. That is, for example, the client company refers to the acquired credit risk score as one of the useful materials for determining the credit risk. Further, for example, when the client company is constructing a model that outputs a predicted value regarding the customer's credit card risk, the client company uses the credit card risk score by, for example, the following two methods. First, the client company uses the Creca Risk Score as one of the variables in its model (normalization, weighting to adjust the impact, etc. are done appropriately). Second, the client company considers the final credit card risk by combining the predicted value output by its own model with the credit card risk score. To give a simplified example, even if the predicted value (the larger the value, the higher the risk), if the creca risk is smaller than a certain level, the predicted value alone will eventually be used for the credit card. Rather than deciding not to issue it, we will consider it as a customer who may not have a high credit card risk.

Although an example of the service providing system 10 has been described above by taking the case of providing the credit card risk score as an example, the aspect of the service providing system 10 is not limited to the example. Of course, it may be configured to provide a score related to needs such as life insurance needs instead of a score related to risk such as the Creca risk score. In this case, the client can obtain a predicted score for the needs of various products and services based on the customer's attributes alone, and based on the predicted score, the potential needs for various products and services for the customer can be obtained. Can be recognized. Therefore, for example, it is possible to recommend a product or service that the customer potentially desires to a customer who has newly started a transaction, which can lead to a reduction in opportunity loss. In addition, when a client company provides a product or service that does not exist, it obtains a total risk score or a total needs score as the prediction score of the corresponding prediction target, and utilizes this as useful information. Be able to judge the risks and needs of customers.

Further, in the above example, the case where the client company obtains the predicted score for a new customer is taken as an example, but of course, the client company can also obtain the predicted score for the existing customer. This allows the client company to determine the potential risks and needs of existing customers. Further, in the above example, the case where the service providing system 10 is provided in the network N is taken as an example, but a system corresponding to the service providing system 10 may be provided on the LAN of the client company.

Although the embodiment of the present invention has been described above, the above-described embodiment is merely an example of the embodiment of the present invention, and the technical scope of the present invention is interpreted in a limited manner. It should not be done. That is, the present invention can be implemented in various forms without departing from its gist or its main features.

For example, in the above embodiment, the score output model SS outputs the magnitude of risk or need as a score (score) instead of a “probability” as an output. Regarding this, the score output model SS may be used instead of the score, or a model that outputs the probability together with the score may be used.

Further, a device other than the model building device 2 may store a part or all of the data stored in the model building device 2 in the above embodiment. Further, in the above embodiment, an external device other than the model building device 2 may execute at least a part of the processing executed by the model building device 2. For example, an external device (for example, a cloud server connected via a network) that is communicably connected to the model building device 2 can perform a part or all of the processing executed by the data set building unit 3 or the model building unit 4. It may be a configuration to be executed. In this case, the model building device 2 and the external device jointly function as a "model building system" within the scope of the claims.

1 Model construction system 3 Data set construction department 4 Model construction department

Claims (16)

It is a data set generated based on a basic database that accumulates personal achievements related to each individual's risk or needs, and each record has one or more attribute items that have individual attribute information as a value, and the individual achievements. For an individual who inputs one or more of the attribute information based on the data set having one or more performance-related items having the performance-related information as a value based on the above, and corresponds to the input one or more of the attribute information. Equipped with a model building unit that builds a model that outputs the predicted value of the forecast target regarding the risk or needs of
The model building unit
The pattern of one or more attribute items corresponding to one or more attribute information input to the model, the prediction target from which the model outputs the prediction value, and the data set used for constructing the model. And, for each combination with the algorithm used to build the model, a candidate model is generated by using a part of the data set as training data.
For each combination of the pattern of the attribute item and the prediction target, each of the candidate models belonging to the combination is tested using a part of the data set as test data to identify the candidate model with the highest accuracy. And
When there is a request to build the model together with the pattern of the specific attribute item and the information about the specific prediction target, among the candidate models generated for the combination of the pattern of the specific attribute item and the specific prediction target. the most accurate based on said data set and algorithms used for construction of high the candidate model, to the feature to build the model be used without using a portion of the data set as the test data makes the chromophore at the distal end Dell construction system. The model building unit
For the candidate model with the highest accuracy, a tentative model generated by using the candidate model or the data set related to the candidate model without using a part of the candidate model as test data is stored.
When there is a request to build the model together with the pattern of the specific attribute item and the information about the specific prediction target, among the candidate models generated for the combination of the pattern of the specific attribute item and the specific prediction target. The model construction system according to claim 1 , wherein the candidate model having the highest accuracy or the tentative model corresponding to the candidate model having the highest accuracy is used as the model. A data set generated based on a basic database that accumulates personal achievements related to each individual's risk or needs, and each record has one or more attribute items that have individual attribute information as a value, and the individual achievements. For an individual who inputs one or more of the attribute information based on the data set having one or more performance-related items having the performance-related information as a value based on the above, and corresponds to the input one or more of the attribute information. Equipped with a model building unit that builds a model that outputs the predicted value of the prediction target regarding the risk or needs of
The model building unit
When there is a request to build the model, along with providing a performance database that stores information about specific attribute item patterns and specific forecast targets, as well as personal performance regarding each individual's risks or needs.
Regarding the combination of the pattern of the specific attribute item and the specific prediction target, a part of the data set is provided for each combination of the data set used for building the model and the algorithm used for building the model. A tentative model is generated by using it without using it as test data, and each of the generated tentative models is tested using the performance database to verify the accuracy of the tentative model, and the tentative model with the highest accuracy is used as described above. It features and be makes the chromophore at the distal end del build system to be a model. The model building unit
The pattern of one or more attribute items corresponding to one or more attribute information input to the model, the prediction target from which the model outputs the prediction value, and the data set used for constructing the model. And, for each combination with the algorithm used to build the model, a candidate model is generated by using a part of the data set as training data.
For each combination of the pattern of the attribute item and the prediction target, each of the candidate models belonging to the combination is tested using a part of the data set as test data to obtain the candidate model having higher accuracy than the standard. For each of the specified candidate models, model construction information that specifies a combination of the prediction target, the pattern of the attribute item, the data set, and the algorithm is generated and stored.
When there is a request to build the model, along with providing a performance database that stores information about specific attribute item patterns and specific forecast targets, as well as personal performance regarding each individual's risks or needs.
A tentative model is generated by using a part of the data set without using a part of the data set as test data based on each of the model construction information corresponding to the combination of the pattern of the specific attribute item and the specific prediction target. The third aspect of claim 3 , wherein the generated tentative model is tested using the actual performance database to verify the accuracy of the tentative model, and the tentative model having the highest accuracy is used as the model. Model building system. The model building unit
The pattern of one or more attribute items corresponding to one or more attribute information input to the model, the prediction target from which the model outputs the prediction value, and the data set used for constructing the model. And, for each combination with the algorithm used to build the model, a candidate model is generated by using a part of the data set as training data.
For each combination of the pattern of the attribute item and the prediction target, a test using a part of the data set as test data is performed for each of the candidate models belonging to the combination, and the candidate model having higher accuracy than the standard is obtained. For each of the identified and identified candidate models, a tentative model generated by using the candidate model or the data set related to the candidate model without using a part of the candidate model as test data is stored.
When there is a request to build the model, along with providing a performance database that stores information about specific attribute item patterns and specific forecast targets, as well as personal performance regarding each individual's risks or needs.
Each of the candidate model or the tentative model corresponding to the combination of the pattern of the specific attribute item and the specific prediction target is tested using the performance database to verify the accuracy of the candidate model or the tentative model. The model construction system according to claim 3 , wherein the candidate model or the provisional model having the highest accuracy is used as the model. The model building unit
The verification results when the accuracy of the tentative model is verified by performing the test using the performance database are accumulated, and the state of each element that affects the construction of the tentative model is input based on the accumulated verification results. , Build a selection model that outputs the certainty that the high-precision provisional model will be built,
When there is a request to build the model after the construction of the selection model, the provisional model with a certain degree of accuracy or higher is obtained by the output of the selection model in which the state of each element affecting the construction of the temporary model is input. The model construction system according to any one of claims 3 to 5 , wherein the construction of the provisional model and the test using the performance database are performed only when it can be determined that the model is constructed. The model building unit
The verification results when the accuracy of the provisional model is verified by performing a test using the achievement database are accumulated, and based on the accumulated verification results, the state of each element affecting the construction of the provisional model and the achievements. A selection model is constructed by inputting the state of each element related to the database and outputting the certainty that the provisional model with high accuracy is constructed.
When there is a request to build the model, the accuracy of the selection model is higher than a certain level by inputting the state of each element affecting the construction of the provisional model and the state of each element related to the actual database. The model construction system according to any one of claims 3 to 5 , wherein the construction of the provisional model and the test using the performance database are performed only when it can be determined that the provisional model is to be constructed. .. Further provided with a data set construction unit for constructing the data set by processing one or more of the basic databases by a predetermined method.
The data set construction unit constructs a plurality of the data sets by changing the basic database to be used or the combination of the basic databases to be used, and the processing method.
The model construction according to any one of claims 1 to 7, wherein the model construction unit constructs a plurality of the models using the plurality of data sets for one prediction target. system. When constructing the data set, the data set construction unit
The eighth aspect of claim 8 , wherein a new item is added to the record of the basic database, and the item value of the newly added item is a value obtained by converting the item value of the existing item based on the public information. Model building system. When constructing the data set, the data set construction unit
For one or more basic databases, using a plurality of specific attribute information as a key, other attribute information or the performance-related information is aggregated by a statistical method, and then a record in which the specific plurality of attribute information is common is created. Summary,
One or more items are newly added to the records of the other basic database different from the one or more basic databases, or the data set based on the other basic databases, and the specific plurality of the attribute information. The model according to claim 8 , wherein the item value of each newly added item is used as the other attribute information or the performance-related information aggregated based on the one or more basic databases. Construction system. The data set construction unit has a second basic database in which the records of the first basic database have a first performance-related item whose item value is the first performance-related information, and is different from the first basic database. If the record has a second performance-related item whose item value is the second performance-related information that has a strong correlation with the first performance-related information, the first basic database and the second performance-related information are described. Build the dataset by merging with the underlying database of
When merging the first basic database and the second basic database, a new item is added to the record, and the item value of the newly added item is the item value of the first performance-related information and the second. The model construction system according to claim 8 , wherein the value is a combination of the performance-related information of the above. The data set construction unit
When the first performance-related information and the second performance-related information take two values of a positive class and a negative class, respectively, the data set after being merged into the construction of the model using the loss function by the model construction unit. Is used, the weight given to the loss function is calculated so that the difference between the normal ratio of the first basic database and the normal ratio of the second basic database is absorbed.
When a model is constructed using a loss function based on the merged data set, the model construction unit performs processing by reflecting the weight calculated by the data set construction unit. The model building system according to claim 11 . The model construction system according to any one of claims 1 to 12 , wherein the predicted value is a value that quantitatively expresses the magnitude of risk or the magnitude of needs by a score. An individual corresponding to the input one or more of the attribute information by using the model constructed by the model construction system according to any one of claims 1 to 13 and inputting the one or more of the attribute information. An information processing system that outputs the predicted value of the prediction target regarding the risk or needs of the information and executes the processing using the output predicted value. A program that is executed by a computer that controls a model building system that builds a model.
The computer is a data set generated based on a basic database accumulating personal achievements regarding each individual's risk or needs, and each record has one or more attribute items having individual attribute information as a value. , One or more of the attribute information is input based on the data set having one or more performance-related items having the performance-related information as a value based on the personal performance, and the input one or a plurality of the attribute information is input. It functions as a model building unit that builds a model that outputs the predicted value of the prediction target regarding the risk or need of the corresponding individual .
The model building unit
The pattern of one or a plurality of the attribute items corresponding to the one or a plurality of the attribute information input to the model, the prediction target to which the model outputs the predicted value, and the data set used for constructing the model. And, for each combination with the algorithm used to build the model, a candidate model is generated by using a part of the data set as training data.
For each combination of the pattern of the attribute item and the prediction target, each of the candidate models belonging to the combination is tested using a part of the data set as test data to identify the candidate model with the highest accuracy. And
When there is a request to build the model together with the pattern of the specific attribute item and the information about the specific prediction target, among the candidate models generated for the combination of the pattern of the specific attribute item and the specific prediction target. A program characterized in that a part of the data set is used without using a part of the data set as test data to construct the model based on the data set and the algorithm used for constructing the candidate model with the highest accuracy . A program that is executed by a computer that controls a model building system that builds a model.
The computer is a data set generated based on a basic database accumulating personal achievements regarding each individual's risk or needs, and each record has one or more attribute items having individual attribute information as a value. , One or more of the attribute information is input based on the data set having one or more performance-related items having the performance-related information based on the personal performance as a value, and the input one or a plurality of the attribute information is input. It functions as a model building unit that builds a model that outputs the predicted value of the prediction target regarding the risk or need of the corresponding individual.
The model building unit
When there is a request to build the model, along with providing a performance database that stores information about specific attribute item patterns and specific forecast targets, as well as personal performance regarding each individual's risks or needs.
For the combination of the pattern of the specific attribute item and the specific prediction target, a part of the data set is provided for each combination of the data set used for building the model and the algorithm used for building the model. A tentative model is generated by using it without using it as test data, and each of the generated tentative models is tested using the performance database to verify the accuracy of the tentative model, and the tentative model with the highest accuracy is used as described above. Use as a model
A program characterized by that.