JP7675431B2 - Database generation device, database generation method, and database generation program - Google Patents

Description

The present invention belongs to the technical fields of a database generation device, a database generation method, and a program for generating a database. More specifically, the present invention belongs to the technical fields of a database generation device and a database generation method that integrate multiple different databases to generate an integrated database, and a program for the database generation device.

Generally, various companies manage one or more customer databases and consumer databases, each of which contains information about their customers and general consumers. These customer and consumer databases vary widely in the number of samples stored and the items (indicators) used as databases, depending on their purposes, etc.

In addition, there are cases where a company needs to integrate an external database with different attributes or items into its in-house managed customer database or consumer database to generate a new customer database with a larger number of samples, etc. An example of a prior art document disclosing conventional technology related to such database integration is the following Patent Document 1.

The prior art disclosed in Patent Document 1 aims to "provide a database integration device or the like that can execute join processing at higher speeds," and is configured as follows: "The reception unit of the database integration device receives a request from a client to merge multiple pieces of data to be joined, and the determination unit of the database integration device determines the database system that will execute the join processing based on join feasibility information that indicates whether each combination of database systems including databases that respectively store the join target data specified by the request can read the join target data from the combined database system and execute the join processing, and whether the combined database system can read the join target data, the generation unit of the database integration device generates an execution plan for executing the join processing, and the execution unit of the database integration device transmits the request to the database system based on the execution plan."

Patent No. 6181250

However, in general, when integrating multiple customer databases or consumer databases, there are cases where the customer ID or consumer ID whose data is included in one customer database or one consumer database does not match the customer ID or consumer ID whose data is included in another customer database or another consumer database. When integrating such customer databases or consumer databases, the only way to do so in the past was to integrate data on customer IDs or consumer IDs common to multiple customer databases and consumer databases. For this reason, the integrated database obtained as a result of the integration (an integrated database containing only data on the common customer ID or consumer ID) has a small number of samples and the items and indicators as a database are limited, resulting in a problem that only databases that cannot be used as an integrated database are generated. This problem leads to the problem that the more customer databases and consumer databases are integrated, the fewer customers and consumers are common to each customer database and consumer database, making the integrated database useless.

The present invention has been made in consideration of the above problems, and one example of the objective of the present invention is to provide a database generation device and a database generation method, as well as a program for the database generation device, that can automatically generate an integrated database with a large number of samples and a wide range of database items (indicators), even when integrating multiple databases.

In order to solve the above problem, the invention described in claim 1 is a database generation device that further integrates another database, which is an integrated database obtained by integrating and expanding an integrated database related to product purchases using an integrating database, into the integrated database related to product purchases, and which is a database in which at least one of the number of samples or items as a database is different from that of the integrated database, and which is a database in which accuracy can be expected to be improved from that of the original database by integrating, an integration means for integrating a general-purpose connection database including various items or indicators and a predetermined number of samples into the integrated database to generate a second integrated database, and an accuracy based on the accuracy of the generated second integrated database. The system comprises an extraction means for extracting data of valid items actually used in integration with the integrated database from the second integrated database to generate an extracted second integrated database when the second accuracy is equal to or higher than a first accuracy, which is an accuracy based on a rate of accuracy of the integrated database; a data generation means for generating data so as to increase the number of samples in the extracted second integrated database to match the number of samples in the integrated database when a third accuracy , which is an accuracy based on a rate of accuracy of the generated extracted second integrated database, is equal to or higher than the second accuracy; and a generation means for approximating data of the extracted second integrated database including the generated data to data of a market statistics database including statistical information of real market, to generate an extracted second integrated database with an increased number of samples.
In order to solve the above-mentioned problems, the invention described in claim 7 is a database generating device that further integrates another database, which is an integrated database obtained by integrating and expanding an integrated database related to product purchases using an integrating database, and which differs from the integrated database related to product purchases in at least one of the number of samples or items as a database, into the integrated database related to product purchases, and the database generating method is executed in the database generating device having an integrating means, an extracting means, a data generating means, and a generating means, and includes an integrating step of integrating a general-purpose connection database, which is a database whose accuracy may be expected to be improved from that of the original database by integrating it, into the integrated database by the integrating means to generate a second integrated database, and an extracting step of extracting the second integrated database by extracting the second integrated database from the general-purpose connection database, which is a database whose accuracy may be improved from that of the original database by integrating it, and which includes a general-purpose connection database which includes various items or indicators and a predetermined number of samples, the data generating means for generating data so as to increase the number of samples in the extracted second integrated database to match the number of samples in the integrated database when a second accuracy, which is an accuracy based on a rate of accuracy of the integrated database, is equal to or higher than a first accuracy, which is an accuracy based on a rate of accuracy of the integrated database; the data generating means for generating data so as to increase the number of samples in the extracted second integrated database to match the number of samples in the integrated database when a third accuracy, which is an accuracy based on a rate of accuracy of the generated extracted second integrated database, is equal to or higher than the second accuracy; and the data generating means for approximating data of the extracted second integrated database including the generated data to data of a market statistics database including statistical information of a real market, to generate an extracted second integrated database with an increased number of samples.
In order to solve the above problem, the invention described in claim 8 provides a database generating device that further integrates another database, which is an integrated database obtained by integrating and expanding an integrated database related to product purchases using an integrating database, into the integrated database related to product purchases, and which is different from the integrated database in at least one of the number of samples or items as a database, and which is a database where improvement in accuracy from the accuracy of the original database may be expected by integrating, and which integrates a general-purpose connection database including various items or indicators and a predetermined number of samples into the integrated database to generate a second integrated database, and a processing unit that generates a second integrated database based on the accuracy of the generated second integrated database, the processing unit including ... When a second accuracy is equal to or higher than a first accuracy, which is an accuracy based on the accuracy rate of the integrated database, the device functions as an extraction means for extracting data of valid items actually used in integration with the integrated database from the second integrated database to generate an extracted second integrated database; when a third accuracy, which is an accuracy based on the accuracy rate of the generated extracted second integrated database, is equal to or higher than the second accuracy, the device functions as a data generation means for generating data to increase the number of samples in the extracted second integrated database to match the number of samples in the integrated database; and a generation means for approximating data of the extracted second integrated database including the generated data to data of a market statistics database including statistical information of real market, to generate an extracted second integrated database with an increased number of samples.

According to the invention of any one of claims 1, 7 and 8 , when the second precision of the second integrated database is equal to or greater than the first precision of the integrated database, an extracted second integrated database is generated, and when the third precision of the extracted second integrated database is equal to or greater than the second precision, the number of samples in the extracted second integrated database is increased to match the number of samples in the integrated database, and then the number of samples is approximated to the data in the market statistics database to generate an extracted second integrated database with an increased number of samples. Thus, an integrated database having the number of samples and items corresponding to the integrated database and also corresponding to the real market can be automatically generated as the extracted second integrated database with an increased number of samples.

In order to solve the above problem, the invention described in claim 2 provides a database generation device described in claim 1 , further comprising a second extraction means for extracting data of the valid items from the connection database and providing the data for the integration when the second precision is less than the first precision or when the third precision is less than the second precision.

According to the invention described in claim 2 , in addition to the effect of the invention described in claim 1 , when the second precision is less than the first precision, or when the third precision is less than the second precision, data of valid items actually used in the integration with the integrated database is extracted from the connection database and used for the integration, so that a more accurate second integrated database with an increased number of samples can be automatically generated.

In order to solve the above problem, the invention described in claim 3 is a database generation device described in claim 1 or claim 2 , wherein when a fourth accuracy, which is an accuracy based on the accuracy rate of the generated sample number-increasing extracted second integrated database, is less than the third accuracy, the data generation means is configured to regenerate the data to increase the number of samples in the extracted second integrated database.

According to the invention described in claim 3 , in addition to the effects of the invention described in claim 1 or claim 2 , when the fourth precision of the extracted second integrated database with an increased number of samples is less than the third precision, data for increasing the number of samples in the extracted second integrated database is regenerated, so that an extracted second integrated database with an increased number of samples with even higher precision can be automatically generated.

In order to solve the above problem, the invention described in claim 4 is a database generation device described in any one of claims 1 to 3 , wherein the extraction of data of the effective items is performed using at least one of a principal component analysis method, a variable importance method, or a method using a SHAP (SHapley Additive exPlanations) library.

According to the invention described in claim 4 , in addition to the action of the invention described in any one of claims 1 to 3 , data on effective items is extracted using at least one of the principal component analysis method, the variable importance method, or a method using the SHAP library, so that data on effective items with higher practicality can be extracted.

In order to solve the above problem, the invention described in claim 5 , in the database generation device described in any one of claims 1 to 3 , further comprises a match evaluation means for evaluating the match between the data contained in the generated sample number increased extraction second integrated database and the data contained in the connection database, and a notification means for notifying the match information indicating the evaluated match.

According to the invention described in claim 5 , in addition to the effect of the invention described in any one of claims 1 to 3 , the degree of match between the data contained in the generated sample number increased extraction second integrated database and the data contained in the connection database is evaluated, and matching degree information indicating the evaluated degree of match is notified, so that the degree of match between the finally generated sample number increased extraction second integrated database and the original connection database can be easily recognized.

In order to solve the above problem, the invention described in claim 6 provides a database generation device described in claim 5 , wherein the match evaluation means is configured to evaluate the match using at least one of the mean/variance method, the histogram method, the statistical distribution utilization method of aggregated data, the S (Signal)/N (Noise) method, or an evaluation method using Cronbach's alpha coefficient.

According to the invention described in claim 6 , in addition to the function of the invention described in claim 5 , the degree of match is evaluated using at least one of the mean/variance method, the histogram method, the statistical distribution utilization method of aggregated data, the S/N method, or an evaluation method using Cronbach's alpha coefficient, so that the degree of match can be recognized more accurately.

As described above, according to the present invention, when the second accuracy of the second integrated database is equal to or greater than the first accuracy of the integrated database, an extracted second integrated database is generated, and when the third accuracy of the extracted second integrated database is equal to or greater than the second accuracy, the number of samples in the extracted second integrated database is increased to match the number of samples in the integrated database, and then the number of samples is approximated to the data in the market statistics database to generate an extracted second integrated database with an increased number of samples .

Therefore, an integrated database having the number of samples and items corresponding to the integrated database and also corresponding to the real market can be automatically generated as a second integrated database with an increased number of samples .

1 is a block diagram showing a schematic configuration of a database generating device according to a first embodiment; 2 is a block diagram showing a schematic configuration of an extraction unit constituting the database generating device of the first embodiment; FIG. 5 is a flowchart showing a database generation process according to the first embodiment. FIG. 4 is a diagram illustrating an example of the contents of a database before a database generation process according to the first embodiment is executed. FIG. 4 is a diagram illustrating an example of the contents of a database after a database generation process according to the first embodiment is executed. 13 is a flowchart showing a database generation process according to the second embodiment.

Next, the embodiments for implementing the present invention will be described with reference to the drawings. Note that each embodiment described below is an embodiment in which the present invention is applied to a database generation device that integrates data from multiple different databases to generate a new integrated database.

(I) First embodiment
First, a first embodiment of the present invention will be described with reference to Figures 1 to 5. Figure 1 is a block diagram showing the general configuration of a database generating device of the first embodiment, Figure 2 is a block diagram showing the general configuration of an extraction unit constituting the database generating device, and Figure 3 is a flowchart showing a database generating process of the first embodiment. Figure 4 is a diagram showing an example of the contents of the database before the database generating process is executed, and Figure 5 is a diagram showing an example of the contents of the database after the database generating process is executed. In Figures 1 and 3, "database" is appropriately represented as "DB."

As shown in FIG. 1, the database generating device S of the first embodiment is specifically realized by, for example, a personal computer, and is composed of a processing unit 1 consisting of a CPU or the like, a recording unit 2 consisting of a HDD (Hard Disk Drive) or SSD (Solid State Drive) or the like, an operation unit 3 consisting of a keyboard, a mouse, or the like, and a display 4 consisting of a liquid crystal display or the like.

The processing unit 1 is composed of an evaluation unit 10, an extraction unit 11, a generation unit 12, and an integration unit 13. As shown in FIG. 2, the extraction unit 11 is composed of a principal component analysis extraction unit 110, a variable importance extraction unit 111, and a SHAP extraction unit 112.

At this time, the evaluation unit 10, the extraction unit 11, the generation unit 12, and the integration unit 13 may be realized by a hardware logic circuit including a CPU constituting the processing unit 1, or may be realized in software by the CPU reading and executing a program corresponding to the database generation process of the first embodiment described later. Similarly, the principal component analysis extraction unit 110, the variable importance extraction unit 111, and the SHAP extraction unit 112 may be realized by a hardware logic circuit including a CPU constituting the extraction unit 11, or may be realized in software by the CPU reading and executing a program corresponding to the database generation process. Note that each of the above programs may be pre-recorded in the recording unit 2 and read by the CPU, or may be configured so that the CPU obtains and uses the program recorded in an external server device (not shown) via a network such as the Internet.

At this time, the evaluation unit 10 corresponds to an example of the "first evaluation means" of the present invention, an example of the "second evaluation means", an example of the "third evaluation means", an example of the "fourth evaluation means", an example of the "fifth evaluation means", an example of the "sixth evaluation means", an example of the "seventh evaluation means" and an example of the "eighth evaluation means", and the extraction unit 11 corresponds to an example of the "extraction means" and an example of the "second extraction means" of the present invention, respectively. Furthermore, the generation unit 12 corresponds to an example of the "generation means" of the present invention, an example of the "data generation means" and an example of the "second generation means", respectively, and the integration unit 13 corresponds to an example of the "integration means" and an example of the "second integration means" of the present invention, respectively. Furthermore, the processing unit 1 corresponds to an example of the "matching degree evaluation means" of the present invention, and the display 4 corresponds to an example of the "notification means" of the present invention.

In the above configuration, the database generating device S is a database generating device that generates an integrated database 102 by integrating data of the main database 100 and data of the donor database 101 shown in FIG. 1. At this time, the data of the main database 100 and the donor database 101 to be integrated may be pre-recorded in the recording unit 2, or may be obtained via a network such as the Internet from an external server device (not shown) each time the database generating process of the first embodiment is executed.

Here, the main database 100 in the first embodiment is, for example, a database of customers or general consumers that is used daily by a company in sales and development work related to the product brand to which the product belongs, and is a database that belongs to the company or the relevant department. Such a main database 100 is a database that basically has a large number of samples (for example, more than tens of thousands of samples) and contains many items (indicators) related to the product brand, and also contains data on actual customers of the product. In contrast, the main database 100 does not contain much data (number of samples) for items (indicators) that are not directly related to the product brand.

In contrast to the main database 100 described above, the donor database 101 of the first embodiment is a database that does not belong to the company, for example, created by an external research company or a department other than the above-mentioned responsible department within the company. Such a donor database 101 has few items (indicators) related to specific products or product brands like the main database 100, and often does not have a large number of samples (for example, about 0 to 1,000 samples). However, the donor database 101 is a database that contains many items (indicators) that are not directly related to the product brand, such as items (indicators) related to the lifestyle of general purchasers (general purchasers including purchasers of products other than the above-mentioned products) and items (indicators) related to general values.

Then, the database generation device S integrates the data of the main database 100 having the attributes described above with the data of the donor database 101, and diversifies the items (indicators) to generate an integrated database 102 that is useful for the company.

More specifically, the recording unit 2 of the database generation device S first temporarily records the data of the carefully selected donor database 103 and the sample generation carefully selected donor database 104 (described later) generated in the database generation process of the first embodiment, and also records other data necessary for the database generation process, and outputs it to the processing unit 1 as necessary.

Meanwhile, the evaluation unit 10 of the processing unit 1 evaluates the accuracy of each database such as the main database 100 from the viewpoint of its accuracy rate, for example, by an evaluation method using a conventional cross validation method using a so-called confusion matrix. Here, regarding the accuracy rate, for example, if a purchaser whose data on a predicted purchase item is stored in the database as a sample actually purchases the predicted purchase item, the accuracy rate of the database including the sample will improve.

Next, the extraction unit 11 extracts effective indicators from the items (indicators) of the donor database 101 that will be effective in generating the integrated database 102.

Here, the method for extracting the above-mentioned effective indicators in the extraction unit 11 of the first embodiment will be explained, particularly with reference to FIG. 2.

The extraction of effective indicators by the extraction unit 11 is performed by at least one of the principal component analysis extraction unit 110, the variable importance extraction unit 111, and the SHAP extraction unit 112 shown in FIG. 2. At this time, the principal component analysis extraction unit 110 extracts effective indicators by a principal component analysis method similar to that used in the past. More specifically, the principal component analysis extraction unit 110 extracts as effective indicators items (indicators) of principal components whose cumulative contribution rate is equal to or greater than a cumulative contribution rate threshold value (e.g., 70%) that is previously set in a changeable manner, and whose absolute value of the principal component loading amount is equal to or greater than a principal component loading amount threshold value (e.g., 0.01) that is previously set in a changeable manner.

On the other hand, the variable importance extraction unit 111 extracts effective indices using the same variable importance method as in the past. More specifically, the variable importance extraction unit 111 extracts items (indices) whose variable importance is equal to or greater than a variable importance threshold (e.g., 0.002) that is previously set so as to be changeable, as effective indices. The SHAP extraction unit 112 also extracts effective indices using the same SHAP method as in the past. More specifically, the SHAP extraction unit 112 extracts items (indices) that fall within a SHAP threshold (e.g., the top 20) previously set for the objective variable, as effective indices. At this time, if there are multiple items (e.g., product brands, etc.) that are objective variables, the SHAP extraction unit 112 adds them to the effective indices by sum integration (OR integration). In addition, whether the extraction result of the principal component analysis extraction unit 110, the variable importance extraction unit 111, or the SHAP extraction unit 112 is used as the extraction result of the extraction unit 11 is preferably set in advance according to, for example, the attributes of the main database 100 and the attributes of the integrated database 102 to be generated.

Then, the effective indexes output from at least one of the principal component analysis extraction unit 110, the variable importance extraction unit 111, or the SHAP extraction unit 112 are output as the extraction result by the extraction unit 11 through sum integration (OR integration). Then, the data of the effective indexes as the extraction result is temporarily recorded in the recording unit 2 as the carefully selected donor database 103.

Returning to FIG. 1, the generation unit 12 of the processing unit 1 generates new data (samples) for the carefully selected donor database 103 using a new sample generation method using AI technology such as the weight-back method or GAN (generative adversarial networks) technology, in addition to the technology for which a patent application is pending (Patent Application No. 2020-085546) by the inventors of the present invention, in order to match the number of samples in the carefully selected donor database 103 with the number of samples in the main database 100 (for example, to make the number of samples in the carefully selected donor database 103 the same as the number of samples in the main database 100), and adds this to the carefully selected donor database 103 to generate the sample generation carefully selected donor database 104, which is temporarily recorded in the recording unit 2.

As a result, the integration unit 13 integrates the data of the recorded sample generation carefully selected donor database 104 and the data of the original main database 100 in the same manner as in the past, generating the integrated database 102 of the first embodiment. In such an integrated database 102, the characteristics (advantages) of the donor database 101 are applied to the main database 100, thereby compensating for the shortcomings of the main database 100. As a result, an integrated database 102 that is extremely useful for the business activities of the company to which the main database 100 belongs (i.e., an integrated database with a large number of samples and a wide range of database items (indicators)) is automatically obtained.

The operations required to execute each of the functions described above are executed by the operation unit 3, and an operation signal corresponding to the operation is output to the processing unit 1. The processing unit 1 then executes the series of functions described above based on the operation signal. Information required to execute the function is displayed, for example, on the display 4 and presented to the operator of the database generation device S.

Next, the database generation process executed by the database generation device S of the first embodiment will be specifically described with reference to Figures 2 to 5.

The database generation process of the first embodiment, which is executed by the database generation device S having the above-mentioned functions, starts, for example, when a power switch (not shown) of the database generation device S is turned on.

When the database generation process is started, first, data from the main database 100 and data from the donor database 101 are acquired in the database generation process S. Next, the evaluation unit 10 of the processing unit 1 evaluates the accuracy of the main database 100 using the evaluation method described above based on the acquired data from the main database 100, and temporarily records the evaluation result in the recording unit 2 as "Evaluation A" (step S1).

Next, in parallel with step S1, the evaluation unit 10 evaluates the accuracy of the donor database 101 using the above-mentioned evaluation method based on the acquired data of the donor database 101, and temporarily records the evaluation result in the recording unit 2 as "evaluation B" (step S2). Next, the extraction unit 11 of the processing unit 1 extracts data corresponding to the effective index from the data of the donor database 101 using the above-mentioned extraction method, generates the above-mentioned carefully selected donor database 103 using the extracted data, and temporarily records it in the recording unit 2 (step S3). After that, the evaluation unit 10 evaluates the accuracy of the carefully selected donor database 103 using the above-mentioned evaluation method based on the generated data of the carefully selected donor database 103, and temporarily records the evaluation result in the recording unit 2 as "evaluation C" (step S4).

Next, the processing unit 1 judges whether the evaluation C recorded in the recording unit 2 is equal to or greater than the evaluation B (step S5). If the judgment in step S5 is that the evaluation C is less than the evaluation B (step S5: NO), the extraction of the effective index in step S3 is deemed insufficient, and the process returns to step S3 again, and the extraction unit 11 re-extracts the effective index. On the other hand, if the judgment in step S5 is that the evaluation C is equal to or greater than the evaluation B (step S5: YES), the generation unit 12 of the processing unit 1 then performs sample generation (data generation) for the carefully selected donor database 103 using the above-mentioned generation method, generates the sample generated carefully selected donor database 104, and temporarily records it in the recording unit 2 (step S6).

Then, the integration unit 13 of the processing unit 1 integrates the recorded data of the sample generation carefully selected donor database 104 and the data of the original main body database 100 in a conventional manner to generate an integrated database 102, which is temporarily recorded in the recording unit 2 (step S7). At this time, the integrated database 102 may be stored in an external server device (not shown). Next, the evaluation unit 10 evaluates the accuracy of the integrated database 102 using the evaluation method described above based on the recorded data of the integrated database 102, and temporarily records the evaluation result as "Evaluation D" in the recording unit 2 (step S8).

Next, the processing unit 1 judges whether the evaluation D recorded in the recording unit 2 is equal to or greater than the evaluation A (see step S1 above) (step S9). If the judgment in step S9 is that the evaluation D is less than evaluation A (step S9: NO), the extraction of the effective indicators in step S3 included in the generation process of the current integrated database 102 was insufficient, and the process returns to step S3 again, where the extraction unit 11 further extracts effective indicators. On the other hand, if the judgment in step S9 is that the evaluation D is equal to or greater than evaluation A (step S9: YES), the processing unit 1 then evaluates the degree of match between the data in the integrated database 102 and the data in the donor database 101 at that time (step S10).

Here, the evaluation of the degree of match performed in step S10 is to evaluate the degree to which the data of the integrated database 102, which is the result of integrating the generalized donor database 101 with the company's own main database 100, matches the data of the donor database 101, that is, whether the database is more versatile. Specifically, the evaluation method in step S10 is the same as in the past, for example, at least one of the following evaluation methods is used: the mean/variance method, the histogram method, the statistical distribution utilization method of aggregated data, the S/N method, or the Cronbach's alpha coefficient. In this case, for example, when the degree of match is determined using the mean/variance method, the degree of match is higher as the average value and the variance range match. The evaluation result of the match is then displayed (output) using, for example, the display 4, or is provided to the person in charge of the company to which the main database 100 belongs, together with the data of the integrated database 102 recorded in the recording unit 2 (step S11). If it is desired to increase versatility by, for example, increasing the degree of matching based on the evaluation result of the degree of matching in step S10 to bring the attributes of the integrated database 102 closer to the attributes of the donor database 101, it is preferable to perform the database generation process with stricter standards for determining whether the data is true or false in relation to the data in the donor database 101. On the other hand, if it is desired to increase the accuracy of the evaluation value in the evaluation unit 10 rather than the degree of matching, it is preferable to, for example, make the standards for determining the objective variables in each database stricter.

Then, the processing unit 1 judges whether or not to end the database generation process of the first embodiment, for example, by an end operation by the operation unit 3 (step S12). If the judgment in step S12 is that the database generation process is to be ended (step S12: YES), the processing unit 1 ends the database generation process as is. On the other hand, if the judgment in step S12 is that the database generation process is to be continued, for example, for another main database 100 or another donor database 101 (step S12: NO), the processing unit 1 returns to the above steps S1 and S2 and continues the above-described processing for the other main database 100 or the other donor database 101.

Next, a comparison between the main database 100 and the integrated database 102 as a result of executing the database generation process of the first embodiment will be specifically described with reference to Figures 4 and 5. Note that Figures 4 and 5 show the results of executing the database generation process of the first embodiment for the main databases 100 of not only one company but multiple companies.

First, as shown in the example of Figure 4, the main database 100 belonging to a certain company A records (accumulates) data such as attributes and whether or not the customer participated in a campaign implemented by company A in association with an ID indicating each customer. In this case, the participation in a campaign is data unique to company A, but data such as the customer's general movement history is not included (see the hatched area in Figure 4).

On the other hand, in the database generation process of the first embodiment described above, the donor database 101 of the first embodiment is applied to the main database 100 of Company A. The donor database 101 used in this case contains data indicating general lifestyles or values, such as the movement history and service usage history, as samples. When the database generation process of the first embodiment using such a donor database 101 is executed on the main database 100, the resulting integrated database 102 may contain, as exemplified in FIG. 5, samples of data such as the movement history, for which data (samples) were not obtained because they were considered to be of low relevance to the business activities of Company A. As a result, an integrated database 102 that is extremely useful for the business activities of Company A is automatically obtained.

As described above, according to the database generation process by the database generation device S of the first embodiment, the accuracy of the donor database 101 is rated B, the accuracy of the carefully selected donor database 103 is rated C, and when rating C≧rating B, the sample generation carefully selected donor database 104 is generated and integrated with the main database 100 to generate the integrated database 102 (see steps S1 to S7 in FIG. 3). Therefore, the sample generation carefully selected donor database 104 generated based on the evaluation results of the accuracy of the donor database 101 and the accuracy of the carefully selected donor database 103 is integrated with the main database 100 to generate the integrated database 102, so that the integrated database 102 with a large number of samples and a wide range of items (indicators) as a database can be automatically generated.

According to a simulation by the inventors of the present invention, the integrated database 102 (containing the same number of samples as the samples in the main database, and the number of variables is the sum of the number of variables in the main database 100 and the number of variables in the donor database 101) obtained by integrating the main database 100 (with a correct answer rate for evaluation B of 80% or more) related to general values and containing thousands of samples, with a main database 100 related to product brands (with a correct answer rate for evaluation A of less than 50%) using the database generation process of the first embodiment, has a higher correct answer rate for evaluation D than the original main database 100 and is close to the correct answer rate of the donor database 101. From these, it can be seen that the database generation process of the first embodiment makes it possible to automatically generate an integrated database 102 that not only contains a large number of samples and a wide variety of items (indicators) as a database, but also has a dramatically improved accuracy (correct answer rate) compared to the original main database 100.

In addition, when evaluation C is smaller than evaluation B, the data of the validity indicators is re-extracted and the carefully selected donor database 103 is re-generated, and the accuracy of the re-generated carefully selected donor database 103 is re-evaluated, so that a more accurate integrated database 102 can be automatically generated.

Furthermore, when the accuracy (rating D) of the generated integrated database 102 is less than the accuracy (rating A) of the original main database 100, the data of the valid indicators is re-extracted and the carefully selected donor database 103 is re-generated, and the accuracy of the re-generated carefully selected donor database 103 is re-evaluated (see steps S8 and S9 in Figure 3), so that an even more accurate integrated database 102 can be automatically generated.

In addition, if the evaluation D is higher than the evaluation A, the database generation process of the first embodiment is terminated and the contents of the integrated database 102 are confirmed (see step S9 in FIG. 3: YES), so that an integrated database 102 with higher accuracy than the main database 100 can be automatically generated.

Furthermore, each evaluation by the evaluation unit 10 is performed using a cross-validation method that uses a confusion matrix, so that the accuracy of each database can be accurately evaluated.

Furthermore, since the extraction unit 11 extracts effective indicators using at least one of the principal component analysis method, the variable importance method, or a method using the SHAP library, it is possible to extract data on effective indicators that are more practical.

In addition, the degree of match between the data contained in the generated integrated database 102 and the data contained in the donor database 101 is evaluated and the evaluated degree of match is output (see steps S10 and S11 in Figure 3), so that the degree of match between the finally generated integrated database 102 and the original donor database 101 can be easily recognized.

Furthermore, since the evaluation of the degree of match in step S10 is performed using at least one of the mean/variance method, the histogram method, the statistical distribution method of aggregated data, the S/N method, or an evaluation method using Cronbach's alpha coefficient, the degree of match can be recognized more accurately.
(II) Second embodiment

Next, a second embodiment, which is another embodiment of the present invention, will be described with reference to FIG. 6. FIG. 6 is a flowchart showing the database generation process of the second embodiment.

In the database generation process of the first embodiment described above, the main database 100 and the donor database 101 are integrated to generate the integrated database 102. In contrast, in the database generation process of the second embodiment described below, the integrated database 102 generated above is further expanded in a manner similar to the database generation process of the first embodiment, to generate a database that can be applied to various markets (including so-called virtual markets on the Internet).

The hardware configuration of the database generation process of the second embodiment is basically the same as the hardware configuration of the database generation device S of the first embodiment, so in the following explanation, the same components as those of the database generation device S are given the same component numbers and detailed explanations are omitted. Furthermore, among the database generation process of the second embodiment, the same processes as those of the database generation process of the first embodiment described above are given the same step numbers and detailed explanations are omitted.

As shown in FIG. 6, the database generation process of the second embodiment executed by the database generation device of the second embodiment is started, for example, when the power switch of the database generation device of the second embodiment is turned on, similar to the database generation process of the first embodiment.

When the database generation process is started, first, data of the integrated database 102 generated by the database generation process of the first embodiment is obtained. At this time, the integrated database 102 provided to the database generation process of the second embodiment may be one in which one main database 100 and one donor database 101 are integrated by the database generation process of the first embodiment, or it may be an integrated database generated by integrating one or more main databases 100 and one or more donor databases 101 by repeating the database generation process of the first embodiment multiple times in succession.

Next, the evaluation unit 10 of the processing unit 1 of the second embodiment evaluates the accuracy of the integrated database 102 based on the acquired data of the integrated database 102 using an evaluation method similar to that of the database generation process of the first embodiment, and temporarily records the evaluation result as "evaluation a" in the recording unit 2 of the second embodiment (step S20).

Next, the integration unit 13 of the processing unit 1 of the second embodiment integrates the data of the integrated database 102 and the data of the connection database 124 of the second embodiment in a manner similar to that of the conventional method, generates a high-precision integrated database 120, and temporarily records it in the recording unit 2 (step S21).

The connection database 124 is a highly versatile database that may function as a sort of "glue" for connecting and integrating two databases, or that may be expected to improve the accuracy of the original database by integrating using the connection database, and that includes various items (indicators) and a predetermined number of samples.

Next, the evaluation unit 10 evaluates the accuracy of the high-precision integrated database 120 using the evaluation method described above based on the generated and recorded data of the high-precision integrated database 120, and temporarily records the evaluation result in the recording unit 2 as "evaluation b" (step S22).

Next, the processing unit 1 judges whether the evaluation c recorded in the recording unit 2 is equal to or greater than the evaluation b (step S23). If the judgment in step S23 is that the evaluation c is less than the evaluation b (step S23: NO), in order to improve the accuracy of the connection database 124, the extraction unit 11 of the processing unit 1 extracts data corresponding to effective indicators from the data in the connection database 124 at that time using the extraction method described above, and generates a new connection database 124 (with carefully selected items (indicators)) using the extracted data and temporarily records it in the recording unit 2 (step S27). This new connection database 124 is then used for the processing in the above step S21.

On the other hand, if the judgment in step S23 indicates that the evaluation c is equal to or higher than the evaluation b (step S23: YES), then the extraction unit 11 extracts data corresponding to the effective indicators from the recorded data of the high-precision integrated database 120 by the above-mentioned extraction method in order to improve the accuracy of the high-precision integrated database 120, generates the carefully selected high-precision integrated database 121 using the extracted data, and temporarily records it in the recording unit 2 (step S24). Here, the generation of the carefully selected high-precision integrated database 121 (step S24) includes the generation of items (indicators) corresponding to a predetermined virtual market whose attributes or characteristics are similar to those of the integrated database 102, and the generation of a model corresponding to the items (indicators). After that, the evaluation unit 10 evaluates the accuracy of the carefully selected high-precision integrated database 121 by the above-mentioned evaluation method based on the data of the generated carefully selected high-precision integrated database 121, and temporarily records the evaluation result as "evaluation c" in the recording unit 2 (step S25).

Next, the processing unit 1 judges whether the evaluation c recorded in the recording unit 2 is equal to or greater than the evaluation b (see step S22 above) (step S26). If the judgment in step S26 finds that the evaluation c is less than the evaluation b (step S26: NO), the extraction of the effective indicators in the connection database 124 in step S27 above is insufficient, and the process returns to step S27 again, where the extraction unit 11 further extracts effective indicators and then provides them to step S21. On the other hand, if the judgment in step S26 finds that the evaluation c is equal to or greater than the evaluation b (step S26: YES), the generation unit 12 of the processing unit 1 then generates samples (data) for the carefully selected high-precision integrated database 121 using the generation method described above (step S28).

Then, the processing unit 1 uses a predetermined market statistics database 122, which is a market statistics database containing statistical information in a real (non-virtual) market and has attributes or characteristics similar to those of the integrated database 102, to approximate the data of the carefully selected, high-precision integrated database 121 after the sample generation to the data of the market statistics database 122 (step S29), and generates a sample generation carefully selected, high-precision integrated database 123 using the approximated data and temporarily records it in the recording unit 2 (step S30).

Then, the evaluation unit 10 evaluates the accuracy of the sample generation, carefully selected, high-precision integrated database 123 using the evaluation method described above based on the data of the generated sample generation, carefully selected, high-precision integrated database 123, and temporarily records the evaluation result in the recording unit 2 as "evaluation d" (step S31).

Next, the processing unit 1 judges whether the evaluation d recorded in the recording unit 2 is equal to or greater than the evaluation c (see step S25 above) (step S32). If the judgment in step S32 is that the evaluation d is less than the evaluation c (step S32: NO), the processing unit 1 returns to step S28 again and repeats the subsequent steps, assuming that the accuracy of the processes such as sample generation and approximation to the data in the market statistics database 122 in steps S28 to S30 was insufficient.

On the other hand, if it is determined in step S32 that evaluation d is equal to or greater than evaluation c (step S32: YES), then the processing unit 1 evaluates the degree of match between the data in the sample generation carefully selected high-precision integrated database 123 at that time and the data in the connection database 124 and outputs the evaluation result in the same manner as in steps S10 and S11 in the database generation process of the first embodiment.

Then, the processing unit 1 judges whether or not to end the database generation process of the second embodiment, for example, by an end operation by the operation unit 3 (step S33). If the judgment in step S33 is that the database generation process is to be ended (step S33: YES), the processing unit 1 ends the database generation process as is. On the other hand, if the judgment in step S33 is that the database generation process is to be continued, for example, with another integrated database 102 as the target (step S33: NO), the processing unit 1 returns to the above step S20 and continues the above-described process with the other integrated database 102 as the target.

The database generation process of the second embodiment described above can achieve the same effect as the database generation process of the first embodiment.

That is, the accuracy of the integrated database 102 is rated a, the accuracy of the high-precision integrated database 120 is rated b, and when rated b ≧ rated a, a carefully selected high-precision integrated database 121 is generated, and when rated c ≧ rated b, the accuracy of the carefully selected high-precision integrated database 121 is rated c, and when rated c ≧ rated b, a sample generated carefully selected high-precision integrated database 123 is generated by approximating it to the data of the market statistics database 122 (see steps S20 to S30 in FIG. 6). Thus, a sample generated carefully selected high-precision integrated database 123 that has the number of samples and items corresponding to the integrated database 102 and also corresponds to the real market can be automatically generated.

In addition, when evaluation b<evaluation a (see step S23 in FIG. 6: NO) or when evaluation c<evaluation b (see step S26 in FIG. 6: NO), data on valid items is extracted from the connection database 124 and is made available for integration with the integrated database 102 (see step S27 in FIG. 6), so that a more highly accurate sample generation carefully selected high-precision integrated database 123 can be automatically generated.

Furthermore, when the accuracy evaluation d of the sample generation carefully selected high-precision integrated database 123 is less than the evaluation c (see step S32 in FIG. 3: NO), sample generation (data generation) is executed again as step S28, so that a sample generation carefully selected high-precision integrated database 123 with even higher accuracy can be automatically generated.

Furthermore, the degree of match between the data contained in the generated sample generation carefully selected high-precision integrated database 123 and the data contained in the connection database 124 is evaluated (see step S10 in FIG. 6), and matching information indicating the evaluated degree of match is notified (see step S11 in FIG. 6), so that the degree of match between the finally generated sample generation carefully selected high-precision integrated database 123 and the original connection database 124 can be easily recognized.

As explained above, the present invention can be used in the field of database integration, and particularly when applied to the field of integrating databases with different numbers of samples and/or items (indicators), it can produce particularly remarkable effects.

REFERENCE SIGNS LIST 1 Processing unit 2 Recording unit 3 Operation unit 4 Display 10 Evaluation unit 11 Extraction unit 110 Principal component analysis extraction unit 111 Variable importance extraction unit 112 SHAP extraction unit 12 Generation unit 13 Integration unit 100 Main database 101 Donor database 102 Integrated database 103 Carefully selected donor database 104 Sample generation carefully selected donor database 120 High-precision integrated database 121 Carefully selected high-precision integrated database 122 Market statistics database 123 Sample generation carefully selected high-precision integrated database 124 Connection database S Database generation device

Claims (8)

A database generating device which further integrates another database, which is an integrated database obtained by integrating and expanding an integrated database related to product purchases using an integrating database, and which is different from the integrated database related to product purchases in at least one of the number of samples or items as a database , comprising:
an integration means for integrating a general-purpose connection database, which is a database that may be expected to improve accuracy from the accuracy of the original database by integration, and which includes various items or indicators and a predetermined number of samples, into the integrated database to generate a second integrated database;
an extraction means for extracting data of effective items actually used for integration with the integrated database from the second integrated database to generate an extracted second integrated database when a second accuracy based on the accuracy of the generated second integrated database is equal to or greater than a first accuracy based on the accuracy of the integrated database;
a data generating means for generating data so as to increase the number of samples in the extracted second integrated database to match the number of samples in the integrated database when a third accuracy, which is an accuracy based on a rate of correct answers of the generated extracted second integrated database, is equal to or greater than the second accuracy;
a generating means for approximating data of the extracted second integrated database including the generated data to data of a market statistics database including statistical information of a real market, thereby generating an extracted second integrated database with an increased number of samples;
A database generating device comprising: 2. The database generating device according to claim 1,
A database generation device further comprising a second extraction means for extracting data of the valid items from the connection database and providing it for the integration when the second accuracy is less than the first accuracy or when the third accuracy is less than the second accuracy . 3. The database generating device according to claim 1,
A database generation device characterized in that when a fourth accuracy, which is an accuracy based on the accuracy rate of the generated sample number-increasing extracted second integrated database, is less than the third accuracy, the data generation means regenerates the data to increase the number of samples in the extracted second integrated database . 4. The database generating device according to claim 1 ,
A database generating device , characterized in that the extraction of data on the effective items is performed using at least one of a principal component analysis method, a variable importance method, and a method using a SHAP (SHapley Additive exPlanations) library . 4. The database generating device according to claim 1 ,
a matching degree evaluation means for evaluating a matching degree between the data included in the generated sample number-increased extraction second integrated database and the data included in the connection database;
a notification means for notifying a match degree information indicating the evaluated match degree;
The database generating device further comprises: 6. The database generating device according to claim 5 ,
The database generation device is characterized in that the matching evaluation means evaluates the matching degree using at least one of the mean/variance method, the histogram method, the statistical distribution utilization method of aggregated data, the S (Signal)/N (Noise) method, or an evaluation method using Cronbach's alpha coefficient . A database generating device further integrates another database, which is an integrated database obtained by integrating and expanding an integrated database related to product purchases using an integrating database, and which is different from the integrated database related to product purchases in at least one of the number of samples or items as a database, and which is executed in the database generating device comprising an integrating means, an extracting means, a data generating means, and a generating means,
an integration step of integrating a general-purpose connection database, which is a database that may be expected to improve accuracy from the accuracy of the original database by integration, and which includes various items or indicators and a predetermined number of samples, into the integrated database by the integration means to generate a second integrated database;
an extraction step of extracting, by the extraction means, data of effective items actually used for integration with the integrated database from the second integrated database to generate an extracted second integrated database when a second accuracy based on the accuracy of the generated second integrated database is equal to or greater than a first accuracy based on the accuracy of the integrated database;
a data generating step of generating data by the data generating means so as to increase the number of samples in the extracted second integrated database to match the number of samples in the integrated database when a third accuracy, which is an accuracy based on a correct answer rate of the generated extracted second integrated database, is equal to or greater than the second accuracy;
a generating step of approximating data of the extracted second integrated database including the generated data by the generating means to data of a market statistics database including statistical information of a real market, thereby generating an extracted second integrated database with an increased sample number;
A database generating method comprising : A computer included in a database generating device which further integrates another database, which is an integrated database obtained by integrating and expanding an integrated database related to product purchases using an integrating database, and which is different from the integrated database related to product purchases in at least one of the number of samples or items as a database,
an integration means for integrating a general-purpose connection database including various items or indicators and a predetermined number of samples into the integrated database, the general-purpose connection database being a database that may be expected to improve accuracy from the accuracy of the original database through integration, to generate a second integrated database;
an extraction means for extracting data of effective items actually used for integration with the integrated database from the second integrated database to generate an extracted second integrated database when a second accuracy, which is an accuracy based on a correct answer rate of the generated second integrated database, is equal to or higher than a first accuracy, which is an accuracy based on a correct answer rate of the integrated database;
a data generating means for generating data so as to increase the number of samples in the extracted second integrated database to match the number of samples in the integrated database when a third accuracy, which is an accuracy based on a correct answer rate of the generated extracted second integrated database, is equal to or greater than the second accuracy; and
a generating means for approximating data of the extracted second integrated database including the generated data to data of a market statistics database including statistical information of a real market, thereby generating an extracted second integrated database with an increased number of samples;
A database generating program that functions as a database generating program .

Images